Pages Navigation Menu

Video: Microsoft Kinect powers augmented-reality magic show

Maybe Microsoft Kinect could replace Microsoft PowerPoint someday?

That was my first thought while watching this TED presentation by high-tech magician Marco Tempest, known for his iPhone and iPod magic tricks.

It has been fascinating to see all the unexpected applications for Kinect springing up as Microsoft’s sensor technology expands from the Xbox 360 to Windows. But I’ve never before seen anything quite this advanced.

This latest video from TED was just recently made available online. I spotted it on Long Zheng’s iStartedSomething blog. Tempest explains his approach in this post on the TED blog.

In this piece I try to use magic to prototype or sandbox a possible future where 3D sensors and gestural sensing are intelligent and ubiquitous. Imagine the possibilities of a system like this for day to day communication or education. A digital storytelling tool for the future?

Technically I am combining a HD video camera with the depth camera of a Kinect. I myself become a virtual object on a virtual stage filmed by a virtual camera and projected behind me in real-time. All visuals are generated in real-time and the sleight of hand and augmentation are blended seamlessly to tell the story of storytelling and magic.

The main task was to bring together a very long list of demanding interactive real-time effects – such as hand-tracking, face-tracking, face substitution, particle systems, elements in 3d space – with the exact course of a six minutes show, timed by the given soundtrack. For the development onformative together with checksum5, the developers of vvvv. Likewise Tebjan Halm and Enrico Viola provided additional vvvv development and preliminary studies done in openFrameworks.

The aim in »Magic and Storytelling« was to find a magical and engaging way to enhance a live performance with interactive graphics. The live component could be anything from a lecture to a performance. The augmentation layer is not pre-rendered. It is generated by a data flow system in real time and in reaction to the live elements. I think that this type of magic can help us to map out what that part of the future might feel like, where 3D projection and gestural sensing are intelligent and ubiquitous. The collaboration with onformative and checksum5 created the perfect synergies to explore my vision. vvvv proofed to be the ideal tool to develop this graphics and computational intensive app. Magic and illusion are a good sandbox for creativity. Adding the element of illusion to an existing technology allows us to create things that are otherwise not possible. An illusion takes a technology beyond its natural boundaries. Magic is a unique tool for pre-visualization. - Marco Tempest

Hardware: One PC, Kinect, HD Camera, Two Monitors.
Software: openFrameworks for preliminary studies, vvvv for final.

Leave a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>