Ubi turns any wall into a touchscreen using Microsoft's Kinect

Ubi turns any wall into a touchscreen using Microsoft's Kinect"We can turn any surface into a 3D touchscreen," explained Anup Chathoth, one third of Munich-based startup Ubi Interactive. Such claims typically conjure up images of floating Minority Report-style touchscreens made from curved glass, but that's exactly what this three-person team has developed.

Ubi's system uses a Microsoft Kinect sensor to turn a regular projector into a multi-touch PC projection system, where regular PowerPoints, web pages, even games no longer require clickers or wireless mice to be navigated. By using the motion-tracking and depth-perception cameras in the Kinect, Ubi is able to detect where a user is pointing, swiping and tapping on a surface and interpret these gestures as if they were being performed on a giant touchscreen or interactive whiteboard.

Ubi Interactive

All well and good in principle, but does it actually work? Wired.co.uk visited Ubi Interactive at Microsoft's Westlake offices in Seattle this week for a hands-on demonstration. And the answer was a resounding yes.

A conventional boardroom projector lit up a pane of frosted glass that was suspended in the centre of a low-lit office. On the other side of the pane was a Kinect sensor, which was capturing the movements and gestures of our hands in front of the glass and sending the data to Ubi's software, running on the same Windows PC that was sending the live image to a projector.

Responsiveness was excellent, with only a split second delay between performing a gesture and action happening on-screen. We played with a 3D model of Earth (as used on Microsoft's Surface), using two hands to zoom in and out of the virtual planet, spin the globe around and locate ourselves in downtown Seattle. Naturally, this was followed by a successful test of Rovio's AngryBirds.

Universal compatibility

Being able to play a PC version of Angry Birds highlights an important aspect of Ubi's software-based system—it works with Windows' built-in touchscreen support and works with any PC application.

"It's all Windows touch-based gestures," Chathoth explained. "We wanted to start with an experience everyone knows, but we can open up our API for 3D gestures. It knows exactly how far your fingertip is from the surface—when you actually touch it, that's a click; when you're not touching, it becomes a hovering motion."

This depth-based action was also demonstrated. Hovering a finger over a strip of book covers projected onto a wall let you gesture left and right to browse titles from an ebook download store's app. Actually tapping one of the covers selected that book for download.

Ubi is one of 11 startups that won $20,000 (£12,300) of funding and support from Microsoft as part of its Kinect Accelerator programme, but it remains independent as a startup. Its business model has several aspects: selling the software to individuals who already have a Kinect and a projector; selling a unit that bundles together a projector with a Kinect, as well as the software; retrofitting office spaces with Kinect sensors and software; working with advertisers and outdoor agencies who want to add interactivity to public displays and surfaces.

"We are already deploying some advertising displays," Chathoth explained. "It's attractive to a customer because of the cost, the ease of deployment; it's portabl—an 80-inch screen you can carry inside a laptop case—it's flexible and it's also an engaging experience."

The software itself, Chathoth noted, would cost an individual "close to $500" (£320), and the company is keeping its options open for working with companies who want to integrate richer 3D applications that require custom gestures and software APIs.

He also highlighted the potential uses within educational establishments. "Kids can go and scratch on a wall, but nothing happens," Chathoth said, highlighting that by using a regular wall instead of a costly smart whiteboard means there's nothing a particularly aggressive pair of young hands could break.


Ubi Interactive is the name under which its three employees—Anup Chathoth, Chao Zhang and David Hajizadeh—operate. Chathoth and Zhang were school friends who wanted to develop an alternative to smart pens such as LiveScribe; Hajizadeh was a friend of a friend who came on board to develop the business side of the startup.

The original system the trio built was not based on the Kinect, however. Instead it was a bespoke device with custom software and an in-house operating system.

"We built a prototype [and] then the questions came up like, 'Yeah, the content is not there. You have your own OS, but we don't like it because we want to run PowerPoint on it', and the projector lacked the intensity," explained Chathoth. "But we we're a small start-up; we can't build a huge hardware business. So we were working with David at that time as he was helping us with the business side of things, and that's when we started using Kinect.

"We thought it could probably do everything we wanted to do—we just have to make the software. So we started playing with Kinect for Xbox when the beta software development kit (SDK) came out and then a couple of months ago we came across the Windows version and we thought this was the right business model for us—what we want to sell today is software for people who can buy Kinect for Windows from some third-party, or from us."

The result is a low-cost technology that can be retrofitted into existing projection systems without any technical experience. Without an effective demonstration it might appear to be a bit too good to be true, but this young German startup has proved its system works. What it needs to prove next is that it can scale itself well enough to support the demands of major clients. It'll likely be an interesting business to watch over the next couple of years.

Source: Ars Technica

Tags: Kinect, Microsoft

Add comment

Your name:
Sign in with:
Your comment:

Enter code:

E-mail (not required)
E-mail will not be disclosed to the third party

Last news

Galaxy Note10 really is built around a 6.7-inch display
You may still be able to download your content
Facebook, Messenger and Instagram are all going away
Minimize apps to a floating, always-on-top bubble
Japan Display has been providing LCDs for the iPhone XR, the only LCD model in Apple’s 2018 line-up
The 2001 operating system has reached its lowest share level
The entire TSMC 5nm design infrastructure is available now from TSMC
The smartphone uses a Snapdragon 660 processor running Android 9 Pie
The Samsung Galaxy A5 (2017) Review
The evolution of the successful smartphone, now with a waterproof body and USB Type-C
February 7, 2017 / 2
Samsung Galaxy TabPro S - a tablet with the Windows-keyboard
The first Windows-tablet with the 12-inch display Super AMOLED
June 7, 2016 /
Keyboards for iOS
Ten iOS keyboards review
July 18, 2015 /
Samsung E1200 Mobile Phone Review
A cheap phone with a good screen
March 8, 2015 / 4
Creative Sound Blaster Z sound card review
Good sound for those who are not satisfied with the onboard solution
September 25, 2014 / 2
Samsung Galaxy Gear: Smartwatch at High Price
The first smartwatch from Samsung - almost a smartphone with a small body
December 19, 2013 /

News Archive



Do you use microSD card with your phone?
or leave your own version in comments (15)