Updated Beyond Reality Face to v2.2.0

Hello everbody!

Today we want to introduce Beyond Reality Face v2.2.0 to you. This update solves some issues we had with the whole structure of the SDK. Now it’s an All-In-One package including all known features:
+ Point Tracking: Follow a certain point over a playing video/webcam stream.
+ Face Detection: Find faces in images or webcam streams.
+ Face Shape Estimation: Analyse found faces and estimate their features (eg. eyes, lips, brows)
+ 3D Pose Estimation: Get the 3D position/rotation out of the analysed face shape.

We also added more example code:
+ Flash Player 11/Stage3D container for Flare3D v2.0 – example app: http://www.beyond-reality-face.com/example?type=fp11
+ Shape Mask exporter/viewer – example app: http://www.beyond-reality-face.com/example?type=masks
+ Point Tracking – example app: http://www.beyond-reality-face.com/example?type=pt

By the way: The FP11 example comes with occlusion. It’s an ‘invisible’ head hiding all things (we want) behind it (eg. good to hide for bows of glasses).
Use your own FP11 compiling settings in your IDE (eg. in FDT) to start the FP11 example. The current setting is Flex 4.1 for FP10.1. We change that to Flex 4.5.1, when we compile to FP11.

If you have any questions or hints, please comment on Facebook: http://www.facebook.com/BeyondRealityFace

You can get the new SDK here: http://www.beyond-reality-face.com/getBRF

Cheers
Marcel

Halloween fun apps using BRF

Just to direct you to a nice Beyond Reality Face (BRF) Halloween fun!

Sweden based agency Stopp created a nice little Halloween app using BRF + 3D overlays. Create an image and use it as your creepy profile image:
http://www.stopp.se/halloween/

Tomek and me made a small Halloween app using BRF + face substitution like overlay. Take a snapshot and upload it to Facebook + use it as profile image:
http://7evenine.com/photoboo/
http://7evenine.com/gootube/

If you want to do a Christmas special using BRF, contact me via:
http://www.beyond-reality-face.com/contact/

I hope to find the time to make another blog post about the tech in PhotoBoo, since it was requested alot.

Thanks for reading!
Marcel

Edit:
At Flash on the Beach 2011 in September I had the  honour to do a 3 minutes session at the Elevator Pitches.
Here is the demo I put together: http://www.beyond-reality-face.com/fotb

Beyond Reality Face – Face Recognition/Tracking in Flash

I was really lazy here. There happened alot since November 2010. The biggest and coolest thing was the development of our face tracking and Augmented Reality SDK ‘Beyond Reality Face’ v2.0 for Flash/Actionscript 3.

In December 2010 we had ‘LISA’, a nice, but not very user friendly approach of face tracking. You had to fit yourself into a given mask, hold this position and click ‘start’. One wipe of your hand in front of your face or rotating your head too much resulted into a loss of tracking. The reset algorithm was good, but the tracking was not robust enough.

So we decided to look for a better approach, something smart, which starts right away, no clicking, no fitting into a mask and more robust. Two days before Christmas we started the development of an ‘Active Shape Model’ algorithm. We invested all the time we could afford beside our daily projects and finished the development on 20th of April 2011.

It was really hard to get a decent FPS. But thanks to Joa’s Apparat we optimized the hell out of this algorithm and got it from 30 seconds down to 30 to 40 ms. Hell yeah, That was heavy shit!

So what’s inside of the SDK? We offer you 2 (3) versions.

The actual ‘Beyond Reality Face’, coming with
face recognition – finding the rough position of a face in an image
face estimation – finding the actual face shape/features
pose estimation – calculating the 3D head pose of a found face shape.
http://www.beyond-reality-face.com/sdk

‘BRF Face Estimation’
This is the face estimation part, no recognition, no pose estimation.
http://www.beyond-reality-face.com/sdkFE

‘BRF Point Tracking’
This algorithm was used in ‘LISA’, our first face tracking approach.
http://www.beyond-reality-face.com/sdkPT

All downloadable trial versions come with documation and examples. The big SDK has classes to show the usage for:
single user image + 2D image of an object
single user image + 3D model of an object
face recognition + face estimation
face recognition only
webcam stream + 2D image of an object
webcam stream + 3D model of an object

Our showcase app shows you the last option: http://www.beyond-reality-face.com/app

For other than personal/private use contact us via http://www.beyond-reality-face.com/contact

What are the next steps?

Currently we are preparing a small bunch of Molehill (Stage3D) demos. We are just waiting for Flare3D to launch the next version of their 3D engine.

If you have any questions about BRF or if something is missing in the SDK, just drop me a line. I would love to see your experiments with the SDK.

Thanks for reading.
Marcel

Don’t use iframe with cookie (session) setting serverscripts

Today we ran into an issue with our movie tracker at www.lovelifefestival.com (click on ‘Star in your own condom movie’).

We use Rails3 as backend and the app is embedded with an iframe. Rails generates an auth token, which is validated when the user wants to upload the movie data.

The issue is about the users security settings. If the user does not allow third party cookies to be set, the backend fails to validate the auth token. So the conclusion is, not to use sessions with iframes. Make sure the user visit your site directly, so the session data is not third party.

Pure AS3 version of Lucas-Kanade Optical Flow

Time for some playing.

As we tried to figure out how to get an accurate tracking using a webcam we stumbled upon different methods.

The first approach was using Haar Cascades. Trying to optimize them ended up in x-thousand lines of unrolled code in one function. I wrote an AIR app which transformed cascade xmls to a function AS3 class. Execution time was really good. If I remember right it was about 6ms for finding 2 eyes. But it was not accurate, found the eyes randomly. So we decided to skip Haar Cascades and tried to find another method which was more accurate.

We found OpenCV and the included lkdemo was what we thought would fit our needs. We did a complete rewrite in AS3 and optimized the hell out of it.

Optical Flow Demo (SWF) (webcam required)

On my machine it needs about 1ms per point + 2ms function overhead.

For optimisation we used Joas Apparat (http://blog.joa-ebert.com/) .

So I opened up a blog

So I opened up a blog.

My intention is to write some articles about my daily work. What am I doing how, and why? It may be interesting for some people. But first of, who is Marcel Klammer:

I’m a Flash/Actionscript developer from Leipzig, Germany, freelancer since September 2007, 27 years old. My daily work is about agency work for big national and internatiol operating companies, mostly advertising campaigns using Flash/AS3.

If you are interested in hiring me, don’t hesitate to contact me. Find me on

LinkedIn: http://de.linkedin.com/in/marcelklammer
XING: https://www.xing.com/profile/Marcel_Klammer
Skype: Marcel Klammer (marcy_lh)

Thanks for reading,

Marcel