Facebook demos Prisma-like augmented reality app for live video

Facebook is working on new camera technology that overlays famous works of art on what you’re seeing. This feature was demonstrated on stage at the WSJD Live conference, but it’s not something that will released in the foreseeable future.

Chris Cox, Facebook’s chief product officer, said the company predicts that in the next five years, 70 percent of internet traffic will be video and this will be across all of its properties, be it around messaging — going from voice to video calls, Instagram — create stories, or any other application. He revealed that Facebook has been looking at a “perennial product problem for phones, which is how do you make it easy for someone to capture and create a video that’s nice using a mobile phone. This is something that most of us aren’t super comfortable doing, it’s really hard to create something you’re really proud of.”

One solution that Facebook is looking at involves the use of image style transfer, which came out of a white paper published in Germany this year. It lets you take an artist like a Monet or Rembrant and transfer the representation of the style of that painter onto an image using what’s called Convolutional Neural Networks and computer vision. Think of it like Prisma.

Facebook's AR filter prototype for live video

What Facebook is exploring is not applying filters after the fact, but live and in the moment experiences. Cox explained that the real-time implementation was the hardest challenge the company faced, as it functions at 24 frames per second. “It’s taking something that’s a known technology, but it was getting it to be fast on a phone and to be able to be done at a low-enough latency without dropping frames, stuttering, or with blurring.”

Don’t expect this feature to be available on Facebook properties in the near future, as it’s considered to be a prototype right now. However, the company is shifting investment towards the camera because it’s a great creative tool where you’ll use it for Instagram, Messenger, WhatsApp, and many other applications that Facebook could roll out in the future.

“Whether intimate moment or broadcast, this is the thing that will take the technology to take it to the next level,” Cox claimed.

Screen Shot 2016-10-25 at 2.49.32 PM

Get more stories like this on Twitter & Facebook

Previous Post
Next Post