Wlad had the idea to use a Kinect to track a performer onstage that plays each of Henry VIII's wives as ghosts.. so an appararition, tracked automatically onstage projecting textures as costumes onto the performer. It's a tricky build however, since the Kinect can only work over a short range of approx 15ft, placed at the front of the stage on the ground looking up at the performers, whereas the video projector has to be placed at the rear of the auditorium up high looking down at the stage from a distance of 100ft or so away. This means that the images do not scale correctly upstage and downstage and there are also roving keystone issues.
We worked on some settings to correct for the difference between the two positions that would allow the image to scale automatically while remaining 'locked' to the performer as she moves about on the stage.
The Kinect uses two Infra Red stereoscopic cameras and is depth sensitive. It is thus able to both see in the dark and to see in '3D' ie to see depth, unlike an ordinary single lens RGB visible light spectrum camera.
The Kinect also had to deal with two rear scrims and a rear projection screen interfering with the image, so we had to tweak the Quartz Composer patch to improve the outline of the performer that it detects with its two stereoscopic infra red cameras.
Anyway, its working now. Not perfect... it is video tracking after all... and there is a slight delay, but fun and certainly works well enough for a ghostly apparition. Good work, Wlad!
Here is Wlad's thread in the Isadora Forum
Will be doing more Kinect experiments and along with Rob Scharein (mathematician) we will be testing a custom Isadora Actor written in C++ (instead of a freeframe plugin) to better handle some of the scenarios that occur when a camera and a projector cannot be set up near each other. There are some useful third party user actors here by Matthew Haber, that we also looked at, but the Cornerpin actor was too heavy on the CPU for our patch. If its written as a standalone actor rather than a nested user actor it might be less heavy on the CPU.
Apologies if there are any missing credits here. Will add more info as it arises.
jamie