Table of Contents
MultiTouch
~~META: status = dormant &relation firstimage = :project:multitouch.jpg ~~
We would like to create a large multi-user interactive surface, commonly known as “multi-touch table”, and experiment with hardware enhancements, creating better software support and enjoying the general coolness.
- Classic inspiration: Jeff Han started it all - http://www.perceptivepixel.com/
- Wikipedia's take on this: Surface_computing
- The NUI group has a lot of howtos, videos, and software: http://nuigroup.com/
- One of the nice construction guides: http://lowres.ch/ftir/
- A _lot_ more is googlable, huge amount of prior art exists.
- hak5 about LCDMultiTouch.com http://www.youtube.com/watch?v=zFmvM_yw72o
- Touché software http://gkaindl.com/software/touche
pQpr3W-YmcQ
Another Approach: Laser Multitouch
There are holographic lenses (it's basically engraved glass) which can diffract the laser beam, so it will not form a dot, but some other shape or image when it's projected on some surface (we've seen those tiny glasses in replacable tips/caps of cheap laser pointers from china which can project different images on wall with different tips). Main goal of diffraction is that it does NOT only filter-out unwanted beams (like regular LCD projector does), but it also diffracts those beams to directions where we are wanting them. So it can turn one straight laser beam to set of diverging beams forming some shape or image when projected. For multitouch purposes we can especialy use diffraction pattern which will form line when projected on wall - which means that it will produce triangular plane in 3D space, which can be used as input device.
Imagine it as some thin IR laser film covering some desk or table, which will be reflected by your hand, finger or something else when you'll touch that desk.
- http://en.wikipedia.org/wiki/Speckle_pattern (explanation why the red dot projected by pointer looks bit strange and noisy)
Mouseless
Goals of the Project
We would like to create a usable multi-touch table that is able to display stuff reasonably well in darkened room and can detect touches and gestures with reasonable reliability. We can then try to explore and implement software applications using this novel user interface.
Table Construction
We need a transparent surface on which we can detect touches and project images. Pasky has 1x2m polyacrilic board bought for exactly this purpose (thickness is 2cm I think, not sure now). We need a construction to put it on since all interesting stuff will be below - at the same time, it must not be too high so that it can be used comfortably. We do not have good solution for this yet; we have a relatively large cardboard box, but it might not be big enough. Some flexible construction that can be adjustable in all three dimensions would be great.
(Alternatively, we could make a vertical wall, but that takes more space and IMO is not really comfortable to work with.)
We need some projection screen (“diffuser”) attached to the acrylate board. Tracing paper works ok for very simple experiments on small sizes, but likely won't do for the real thing.
Unsurprisingly, we need an LCD projector. Pasky can arrange borrowing one for temporary periods - if the project goes far enough, he is willing to contribute significantly to buying one. Alternatively, we could build our own LCD projector from an old LCD screen (can probably be arranged).
We need a way to make the projected image big enough to cover the surface in the not-so-big space below the surface. Maybe special lens could be used, maybe we need a system of mirrors.
When fingers touch the surface, this is detected by IR light spots triggered at the surface. To register this, we will need an IR webcam. Pasky has a low-end one, the FPS is not high but can suffice for starters. Later, we can buy a wiimote, which has high FPS IR camera inside. To shine the IR light, we need an IR source; it is possible to buy special IR reflectors, Pasky also has few tens of IR LEDs.
Finally, we need to implement the way the IR light spots appear when fingers touch the surface. There are three ways:
-
- Front DI: we rely on ambient IR light, detecting the IR shadows casted by fingers on the surface. This is suitable only for extremely rudimental setups, since even fingers not touching the screen at all will cast shadows and the results are very noisy.
- Rear DI: we have IR illumination source next to the webcam, desperately try to evenly illuminate the surface from below and hope the IR light will reflect from the fingers. We can detect also hovering hands (reduced intensity) and placed objects (special IR-reflecting marks). It is tricky to achieve even illumination and it is sensitive on background light.
- Frustrated Total Internal Reflection (FTIR): The IR light is shined _into_ the acrylic plate, and reflects within unless a finger touches the surface, changing refraction index and causing the light to shine out. This is most accurate and reliable method, but cannot detect hovering, placed objects and it is quite laborous to polish the edges and mount the LEDs to a frame around the board.
If this is to be useful also for the Reactable, we _need_ diffused illumination.
Software
The NUIGroup above has a set of software. Unfortunately, it is based on Processing which is pretty awful, but is fine enough for initial testing. Since the construction is so tricky, I would prefer to ponder more on the software only when we have the hardware side covered.
Status and Roadmap
As described above, we have various parts, and Pasky has done some rudimental experiments with all methods, but not very successful. One needs the LED frame for FTIR and polished edges; front DI is nearly unusable; and for rear DI, Pasky has not been able so far to set up the IR illumination so that it is (i) strong enough (ii) diffused evenly over the whole surface. Also, we do not have suitable construction scaffolding so far, and the projection surface is not solved completely.
Pasky has tried to do this several times in the past, but always gave up for the time being due to his severely lacking mechanical construction skills. He will happily join forces with others who actually know their way around constructing stuff and have good instinct for suitable materials etc.
Snapshot of last state of Pasky's efforts: http://log.or.cz/?p=41
Hacking Point
All parts mentioned above are at Letnany, and we have some (limited) space to hack there. Anyone is welcome to visit! Please write a short note to the mailing list if you are interested and we will arrange some hacking sessions.