"If you are unsure which computer/laptop/tablet to purchase
and are considering spending a lot of money then please email me
- I can offer personal advice on how to target the sweet spot
between cost and performance (and screen size)."
So not only is he giving away this amazing software but he is also offering free, personal advice on what you need to buy working with your budget!
It's not a panacea. Open source software is nice when it doesn't put contibutors into the poor house while corporations depend on the code at large-scale and don't support what they take. Open source software is great for SDKs and other supporting materials, but Github and Google aren't going to make their core infrastructure available to all... it defies common sense.
Furthermore, companies which would lose more profit than hard-to-monetize goodwill gained, by releasing something their competitors can use against them, has as much chance as an airplane made of lead.
Open source can help sell small products get initial traction but it makes them easy to knock-off and easy to steal (unless you make the build process hard and fragile)... It just won't scale because human nature.
Art has been quite open source in its workings since centuries. Look at how artists were joining masters to learn the art and how they "forked" masters' techniques to create theirs. Personal vision is not precluded from open source, to the contrary it is encouraged.
What's amazing is that I could see a cottage industry around this with respect to google glass like overlays that allow for visual navigation of the overlay... I think we have our first entry into the cyberpunk visual navigation future being seeded by this guys great work.
The way posts work on HN is a bit puzzling and not immediately obvious - I'll post something and then refresh the front page and won't even see it. The Reddit model seems simpler - I post something and it appears.
I think you're correct. A Show HN that goes nowhere fast will disappear and be seen by virtually no one. An equivalent in r/startups or similar will still be seen by a few people even if it doesn't attract comments or upvotes.
This is a good point - how do copyrights protect intellectual property from open source alternatives? I think that this is a really good thing to open-source, but the law can be tricky to navigate. Almost like you need a degree in it...
I wonder if you can combine this with Vim so that you could use it for lightning fast cursor movement and selections and still use the keyboard for anything else.
The problem with gaze tracking is that achieving the kind of accuracy you are thinking about isn't possible right now.
The trackers he listed have an accuracy of about 0.3-0.5 degree which at a distance of 50cm to the screen still maps to a fairly large area of about 50x50 pixel (depending on screen resolution and PPI). That's far too big to guess where you want your cursor to be placed.
Moreover, the fact that you'll see the cursor moving creates a loop in which you will follow the delayed cursor around with your gaze.
I wonder how well the hardware would work with an input system like Dasher. Dasher was experiment in statistical inference and tries to be an accessibility tool as well. http://www.inference.phy.cam.ac.uk/dasher/. I found it really fun to play around with and could get decent speeds but not great. It's true about eye tracking being prohibitively high. When we order an eye tracker for the lab, it ended up costing us £10K.
In the demo, he was quick and he didn't even seem to use much autocompletion. However, I wonder if using a different keyboard layout would be helpful. Dvorak a Colmak, for example, place the more frequently used letters on the home row so you'll spend more time with your eyes there. You can evaluate different layouts with a tool like this:
I wonder if it really matters with your eyes vs. your hands and fingers. Flicking your gaze around the keyboard seems much easier than moving your fingers around - you'll never have two fingers hitting each other.
And on the other end of the speed spectrum, it seems like it's unlikely that this will ever quite be as quick as operating even a QWERTY keyboard with fully functional hands.
If we're talking about what's optimized for eyes and the movement of them, you wouldn't even want a keyboard layout, you'd want something more like a dart board or oval shaped dartboard where the spaces are letters and the most common letters are towards the center. QWERTY is fine for someone familiar with it, like an experienced typist whose already mapped those keys mentally, but I expect someone with a lifelong disability would have no true familiarity with QWERTY (since they can't type) and would have to learn any layout from scratch anyway, so you might as well start from scratch.
There were keyboard layouts optimized for palmos pen input; fitaly and atomik. The most used letters are clustered in the middle so movements is minimized, theoretically speeding up the eye typing as well.
It seems to be a faster entry method then using a switch like Stephen Hawking does or using a single switch morse type system at perhaps 20 wpm, but indeed much slower then having 10 fingers (switches) and perhaps two feet available at the same time.
I tried to eye out "Batmobile" and had a surprisingly hard time compared to Colemak or QWERTY layouts. (I tested by making fake layouts in Photoshop.)
The issue being that while I can certainly find the letters when they are alphabetical - clusters of commonly used letters are further apart. Having "th" and "ng" visually close to each other aids a lot.
Something like the radial keyboard used in Steam's big picture mode could be really useful here. It would reduce potential error surface by quite a bit.
This is amazing software and I'm glad something like this is open source. Accessibility hardware often costs a fortune (for understandable reasons) and software isn't typically cheap either. This seems to be very high quality and it has the potential to really improve people's lives. I'm glad things like this exist.
I have been using voicecode.io (with dragon naturally speaking) and IR head tracking so that I don't have to use my hands for anything now with pretty good success. The head tracking mouse has a much smaller learning curve, but programming by voice has been rewarding as well.
voicecode.io is far ahead of anything else in terms of out of the box programming by voice. Everything else requires a lot of up front effort and essentially designing your own language before you can be productive.
It's curious that Optikey was posted today, because yesterday I wrote a post about the promise of iris scanning technology to deliver a seamless payments experience in VR headsets.
Unfortunately, none of the VR headsets shipping in the next six months have iris trackers (much less scanners). The only headset I found was Fove, which was successfully kickstarted and ships in 2016. Fove did a promotion with a Japanese school for wheelchair-bound children, in which a child played a piano using the iris tracking function of the headset.
There are some iris scanner-equipped phones shipping in the near future, but none with iris trackers as far as I could tell.
I can share the post link if people are interested.
Was thinking the same thing. The inevitable trend with VR is that it gets miniaturized to slimline eyewear, contact lenses, or embedded directly wetwear.
As it progresses, input devices will need to evolve quite a bit since it won't always be convenient to have a full-size keyboard, and part of the value of having a screen on your eyes vs. in your hand is that it frees up your hands for other things. Being able to use slight muscle gestures (ala "Rainbow's End" by Vernor Vinge) in combination with eye-tracking, audio controls, etc. seems like a logical leap until we can decode brainwaves sufficiently to use that as input.
I imagine something that shows a swipe-style keyboard overlaid on screen with your current finger position is also a likely solution.
That said, my big beef with this seems to be forcing a traditional keyboard layout on screen that takes up a massive amount of screen real estate. Would much prefer thinking outside of the traditional keyboard layout and making better use of space.
Assume you have a bunch of HUDs/AACs/Glasses/Whatever that are using this tracking tech. Assume that they are ONLY looking at the real world, not some online data/webpage etc...
There is a camera that either is fwd looking or also 360 looking.
Use the tech to eye track exactly everything that MANY people are looking at to train an AI as to what items in the real-world are important.
i.e. "the ground exists and we know its there, thus its priority in information is low" however "these signs that are being looked at have a higher context priority, and require understanding"
By doing this on a fair scale... you could train AI to use the information of "whats visually important to human navigation" to train them to navigate. This augments what other ML/AI stuff has already been going on...
I do not know if this is basically how self driving cars were developed -- but now that you have a seed of this tracking tech in open source -- it could blossom.
I don't think this is stupid, but probably won't happen any time soon. A project to correlate what humans look at to objects in the real world is probably possible, but would be a significant effort to get it to work reliably and accurately. Scaling it to a large population and processing the data to the point of providing useful insight about how we behave and look at things would be another hurdle.
Basically this probably won't happen until Google (or Facebook, or Apple, etc) decides that knowing what people look at is worth the effort/cost.
I'd be happy if all those time-and-space-stamped photos uploaded to facebook could be stitched together to make a photoessay of my vacation. That way I wouldn't even have to take a camera with me; facebook could use Other People's Pictures that I just happened to be in proximity to.
Great work. I wonder if the author had heard of the Eyewriter project [1]? Similar system, fully open source, but I'm not sure how active development is nowadays. It's been up since ~2011, though, so quite old by software standards (uses openFrameworks/openCV). Still, it worked impressively well when I built a derivative system.
The accessibility space needs as much open-source development as possible - most of the commercial tech, if you can find it, is locked down and outdated.
Tobii bundles some (Windows only) integration software with their EyeX. It allows you to click on whatever you are looking at with the press of an action button (left ctrl key for me), along with a few other features (eye tracking the alt-tab switcher is what I really like). It works okay, but eye tracking isn't quite as accurate as a mouse and you either need to make your icons big and widely spaced or you have to zoom in (with an action click) before selecting the icon precisely.
If eye-tracking is not a replacement, eye-tracking can still be used to initially teleport your cursor near your target, and then you can use the mouse to precisely place the cursor (Tobii EyeX (and Sentry?) has the feature).
I use a trackIR and linux-track with good success. The only issue is you have to have a reflective dot (or 3) on your head, ideally on the bill of a hat. It is accurate enough that I have to use the trackpad maybe once a day. I don't think it would work well for drawing or anything like that though ...
Mmmm... that was a great deal slower than me using the latest Android swiping support, even counting errors. Can't speak to the other mobile tech stacks, but if you're typing on a phone is slower than that you might want to poke around, you may be missing a technology already installed on it.
(Not a criticism of the excellent work here. It's a fundamental bandwidth problem.)
Concretely: In the time the demo took to write "Meet OptiKey. Full computer control and speech using only your eyes.", I wrote in an Android text area: "I am just typing words as quickly as I can in the android interface. I'm even having to think a bit about it more that I've written so much. uh, hello world? and other stuff. I'm still going and going and going." Including the missing "now" between "more" and "that", and you can't see it but I had to correct 3 words, too. That's about three times faster on the phone I have in my pocket. And let me just say... that is impressive performance for a free eye tracking suite, in my opinion, to still be that fast with just eyes.
You'd perhaps need something like this to come closer:
>Microsoft patents eye-tracking keyboard software
The idea’s just like swipe-based keyboard software, but instead of tracking the motion of your fingertip, the system tracks eye movement.
As your glance moves from key to key, darting around and changing direction, a devices camera records its gaze, and with a little algorithm magic, Microsoft tries to work out just which keys you were looking at – and ultimately, what word you were attempting to construct.
No you're right. Dexterous fingers beats eye tracking for text input, but my target audience can't use their hands so this is (hopefully) a good alternative.
If you reach out to the author, I'd imagine he'll get back to you as soon as he's able. He's answered a few questions over on Reddit, and his email address is on the page linked in the HN OP.
The author seems to be responsive, but the Reddit thread got /r/bestof'd so he's probably RIP Inbox'd. And he's a new dad. Might be worth a followup in a few days if he doesn't get back to you beforehand.
I have a Tobii EyeX and it has the option to track a single eye. You would basically need to set it to track just your right eye, which is a piece of cake in the config (on Windows at least).
OptiKey will listen to whatever it's told to! If you're using a Tobii tracker you should be able to change your configuration to track just one eye. OptiKey just cares about the coordinates that the Tobii engine spits out so it should be fine. Best of luck.
Truly an inspiring person.