In the hands of some visionary people, my little project can be a medium for art. Communicating big ideas that steal hearts and move people via an installation art exhibition.

Just like this inspiring piece by the geniuses at teamLab.

But in my hands, it’s for giggles. Also an MSc degree.

My baby project combines 3D illusion art with anamorphic optical illusion. I borrowed a fancy 3D projector from the Institute of Neuroscience Newcastle University. I don’t know if the guys at teamLab used stereoscopy as well, but considering the scale of the screen–the whole room; I don’t think they need a 3D projector. Probably need a whole lot of motion detectors, though.

I used Unity not only because I was a total noob but also it works as a streamlined prototyping tool. The program draws a scene into a screen at an angle to create the anamorphic effect. A Kinect Sensor made sure the image follows where the viewer’s point of view.

At first, I was tempted to use maths to calculate the correctly projected image. The formula looked fancy but I couldn’t make it ran fast.

T = V A (P B)^-1

T = transformation matrix that needs to be calculated
V = viewer’s projection matrix
A = centroid of viewer’s view plane on the screen
P = projector camera’s projection matrix
B = centroid of projector camera’s view plane on the screen

That matrix inverse calculation every time the viewer moves? Nah. I’m not sure I could make it work in time for submission dedline.

Finally, I went with the trust-me-I’m-an-engineer method. Multiple rendering layers and two projected images from two cameras. Hurrah! I made a floating ghost cube.

Still looked weird because a slight error in measurement compounded. After a lot of double-check, it looked better but the latency was still jarring because I needed to smooth out the input from Kinect. I had to choose between slow response and jittery image.

I also made some other scenes but even the cube looked weird without 3D glasses.

I learned things about stereoscopy from this. I’m so thankful that I got a lot of guidance from my supervisors from both the Institute of Neuroscience and the School of Computing Science. I dream someday I’ll have a chance to use this and create some trippy installation art and people would come and see it for giggles.