I was really lazy here. There happened alot since November 2010. The biggest and coolest thing was the development of our face tracking and Augmented Reality SDK ‘Beyond Reality Face’ v2.0 for Flash/Actionscript 3.
In December 2010 we had ‘LISA’, a nice, but not very user friendly approach of face tracking. You had to fit yourself into a given mask, hold this position and click ‘start’. One wipe of your hand in front of your face or rotating your head too much resulted into a loss of tracking. The reset algorithm was good, but the tracking was not robust enough.
So we decided to look for a better approach, something smart, which starts right away, no clicking, no fitting into a mask and more robust. Two days before Christmas we started the development of an ‘Active Shape Model’ algorithm. We invested all the time we could afford beside our daily projects and finished the development on 20th of April 2011.
So what’s inside of the SDK? We offer you 2 (3) versions.
The actual ‘Beyond Reality Face’, coming with
face recognition – finding the rough position of a face in an image
face estimation – finding the actual face shape/features
pose estimation – calculating the 3D head pose of a found face shape.
‘BRF Face Estimation’
This is the face estimation part, no recognition, no pose estimation.
‘BRF Point Tracking’
This algorithm was used in ‘LISA’, our first face tracking approach.
All downloadable trial versions come with documation and examples. The big SDK has classes to show the usage for:
single user image + 2D image of an object
single user image + 3D model of an object
face recognition + face estimation
face recognition only
webcam stream + 2D image of an object
webcam stream + 3D model of an object
Our showcase app shows you the last option: http://www.beyond-reality-face.com/app
For other than personal/private use contact us via http://www.beyond-reality-face.com/contact
What are the next steps?
Currently we are preparing a small bunch of Molehill (Stage3D) demos. We are just waiting for Flare3D to launch the next version of their 3D engine.
If you have any questions about BRF or if something is missing in the SDK, just drop me a line. I would love to see your experiments with the SDK.
Thanks for reading.
Time for some playing.
As we tried to figure out how to get an accurate tracking using a webcam we stumbled upon different methods.
The first approach was using Haar Cascades. Trying to optimize them ended up in x-thousand lines of unrolled code in one function. I wrote an AIR app which transformed cascade xmls to a function AS3 class. Execution time was really good. If I remember right it was about 6ms for finding 2 eyes. But it was not accurate, found the eyes randomly. So we decided to skip Haar Cascades and tried to find another method which was more accurate.
We found OpenCV and the included lkdemo was what we thought would fit our needs. We did a complete rewrite in AS3 and optimized the hell out of it.
Optical Flow Demo (SWF) (webcam required)
On my machine it needs about 1ms per point + 2ms function overhead.
For optimisation we used Joas Apparat (http://blog.joa-ebert.com/) .