Posts

If it keeps on rainin, the levee’s gonna break.

It is common knowledge that as technology evolves, individuals are becoming more informed and driven to create content aligned to their passions. Think about the content you are seeing in your social feeds – your colleague’s oddball memes, Aunt Sue with puppy’s ears and your old school mate Trevor Lam and his latest “work of art” – everyone around us is getting more and more able to express their creativity and publish it too. Facebook Augmented Reality (AR) is going to allow people to express themselves a whole lot more.

 

Give everyone the power to share anything with anyone.

– Mark Zuckerberg

 

What is F8 – and why should you care?

The Facebook Developers conference (F8) was first hosted in 2007 – where the team at FB presented the social graph – or a rendition of the concept of a social network. Subsequent editions of F8 hosted similarly theoretical principles behind the evolution of the behemoth that FB is today. Essentially, this is the conference where they announce their next plans, and given how integrated FB is in our lives today – you might want to be aware of what’s ahead.

 

F8 2017: Facebook Augmented Reality

Earlier this week, at F8 2017, Mark Zuckerberg (Zuck!) shared an update on the next phase of FB’s 10-year product plan, originally shared in at F8 2016. Commencing with some warm up jokes about the release of Fast and the Furious 8 (the “other” F8 ) this week, Zuck proceeded to give us a snapshot of how FB plans to integrate AR into camera functions in their apps.

 

Click to read: Business Insider’s article on the release of the FB 10-year plan

Mark Zuckerberg’s keynote from Day 1 of F8

 

FB, AR & Cameras – how do they come together?

In recent times, the FB family of apps (FB, Messenger, Instagram & Whatsapp) have seen the integration of camera icons across the board – enabling functions such as video conferencing. While these changes have gone unnoticed by some, it is estimated that the FB messenger app has 1.2 billion monthly users.

 

Facebook-Messenger-Video-Calling

FB Messenger with camera Icon (Source: Forbes)

Click to read: Forbes article on Facebook Messenger passing 1.2 billion users

 

And how does AR fit into this equation?

Zuck went on to share a common understanding that AR is essentially used for three key purposes;

  • The overlay of data onto the physical reality around us – such as messages or information
  • The ability to add digital objects into our surroundings – like a virtual television or gaming avatars
  • Enhancements to physical objects around us – like buildings or human faces.

Facebook Augmented Reality will work by aggregating these tools – the cameras within FB apps will allow users to create AR “experiences” – and they will seem quite familiar once you see them. AR is not rare – we’ve all seen it in one form or the other – quite possibly most recently in the form of Pokemon Go.

Now, we all know that Pokemon Go was a huge driver in bringing AR to the mainstream – even if it was little more than a temporary fad for most. You’ve also undoubtedly heard us go on about how the popularity of the game was influential in the voluminous cash injection industry players received in 2016. But this could very well be, to quote Led Zeppelin, when the levee’s gonna break.

 

Where might you have seen AR in action?

Some use cases we’ve found interesting

 

So how does it all work exactly? (A splash of technical jargon)

This vision comes to life with the incorporation of some technological building blocks;

  • Simultaneous localization and mapping (SLAM) – a technique borrowed from Artificial intelligence – enabling users to integrate digital elements into the reality in front of them
  • 3D effects – capturing and interacting with scenes that you can explore and effects that you can adjust
  • Object recognition – technology that can identify items around you, that can then be targets for the overlay of digital content

 

What about Virtual Reality?

AR and Virtual Reality align quite well, and in that vein, FB is launching a platform called Facebook spaces – where you can interact with people in a virtual environment through the Oculus Rift.

 

Facebook spaces (from F8 2017)

Meanwhile, we’ve been doing our own experiments with the Rift too!

image1 (1)

 

 

Early days in terms of adoption – and the plan for an open platform

Zuck reiterated a key message around AR: It is yet in a rudimentary phase of development – and most of the use cases around us are still evolving too. Don’t expect the world to change overnight.

That being said, in offering an open platform and leveraging the huge universe that lives on FB – users will be able to create AR experiences on their own, and share them online. In doing so, new users will have access to parallel creations by fellow users from around the internet.

This spike in available content will invariably help everyone around us find AR experiences that fit their fancy – especially if this punt from FB is a good one – and at Appearition, we certainly believe Facebook Augmented Reality is going to be something special for all of us.

 

Staff Blog: Mucking about with 360 VideoSphere VR Part 3

Interested in 360 VideoSphere (360 Virtual Reality)? This series shares what I learnt producing a short film aboard an old Sailing Ship for the Melbourne Fringe Festival[1].

In the last update, my team of volunteers were looking forward to shooting our first 360 VR short film. Then we hit a hurdle. Our cinematographer was unable to do any test shots or editing, due to incompatibility with his computer and the festival’s practice camera.

AWFUL AUDIO, OR AUDACIOUS?

The hurdle was followed by a face-first tumble into the mud when our sound engineer had an overseas job at the same time as the shoot. We could not reschedule; Melbourne Regatta Day aligned with our shooting window, and was too good to miss.

Plan B. We contacted other specialists, and they were keen to try 360 VideoSphere production, but there was not enough time to line up people and equipment. So we fell back to Plan C; use the Samsung Gear 360 camera’s inbuilt microphone. This would not be great if you’re recording a concert, or producing a narrative that requires directional sound to direct audience attention. Still, for our purposes we were pretty pleased with the quality.

body_image_1_article_3

NEW TECHNOLOGY? IF WISDOM FAILS TRY TENACITY

Conventional wisdom with emerging technology is test it early and iron out the inevitable problems. It’s wisdom for a reason.

Unfortunately, the festival’s production units were unavailable in advance. So, we could give up, or improvise and manage the risk. We decided to go for it, and as expected, encountered problems right away.

It is not possible to get behind a 360 camera and look through a viewfinder, but with the Samsung Gear you can use your phone as a remote viewer. A nifty feature, unless as in this case, Samsung block access to the app because you’re in a country where the Camera has yet to be officially released. The festival hadn’t identified this issue because like us, this was their inaugural spin on the 360 dancefloor.

Short of trying solutions like IP masking to make it look like we were in South Korea (where the camera was bought) we would have to shoot blind – so that’s what we did. For example, we climbed the mast and out onto a yardarm to attach the camera. Then, we recovered it after 10 minutes to physically connect it to my laptop and review the footage.

Sometimes even workarounds need a workaround. The case around the camera’s USB port was too small for our cable, and modifying the borrowed camera was out of the question. Our resident inventor, Andrew, borrowed the skipper’s knife and whittled away his own USB cable’s superfluous housing. I admit, I was sceptical but it fit neatly into the Camera’s port.

IN THE NEXT UPDATE

So after a long day shooting we had plenty of good footage.

However, 360 VideoSphere (360 VR) film is captured on multiple cameras. So, this composite footage must be “stitched” together before it can be edited. The results create some unique trials in the editing suit!

USEFUL LINKS

 

For an immersive experience and ease of use try using a Google Cardboard headset and selecting this icon in YouTube: google_cardboard_logo

 

CONTACT US






Staff Blog: Mucking about with 360 VideoSphere VR Part 2

All Goes Well, Until It Doesn’t

Interested in 360 VR (VideoSphere)? This series shares what I learnt producing one aboard an old Sailing Ship for the Melbourne Fringe Festival[1].

I was lucky enough to come across the opportunity at a VR (Virtual Reality) meetup to produce a VideoSphere short film, with the camera and expert guidance supplied[2]. I have produced traditional video, and was already intrigued after seeing two staff from ABC TV talk about their experience at an earlier Mixed Reality meetup[3].

When Amy Nelson and Astrid Scott explained how they produced the ABC’s first 360 production, I was struck by how accessible they made it[4]. They faced the challenge of placing their camera on a pole over an angry bull in the middle of a rodeo in outback Queensland. For other shots they had the camera operator hiding behind a barrel. Not because of the bull, but because hiding the crew behind the camera is not an option, when there is no “behind” the camera.

They were candid about accepting mistakes. They knew that many of the rules learned over the last century do not apply to this medium so new practices must be developed through experimentation.

So given this chance, my first thought was ‘brilliant!’ which is my reaction to anything related to Virtual and Augmented Reality. But I had no team, no 360 experience, and no time. So, it had everything going for it but common sense.

1830’s Cultural Heritage Meets 360 Video

The festival required a proposal. Cultural Heritage (i.e. history) kinda rocks and everybody loves old fashioned sailing ships, whatever their opinion of Johnny Depp’s (over)acting in Pirates of the Caribbean. However, not everybody can spend time aboard one, let alone to climb out onto a yard-arm high above the deck. I have been a volunteer aboard the Enterprize, an educational tourism ship, for the last few years, and this was a way to share the experience.

body_image_1_article_2

The first step was to create a storyboard (a sequence of shots, like panels in a comic book) and get the festival and the ship’s management on board.

The next step was to put together a team. For the cinematography I called on Andrew Gotts, an old friend who has worked in video production. He enjoys experimental technology, and has a good head for heights. He suggested an editor, Nadia Nusatea, so that made three. We still required an audio specialist so I approached Darius Kedros who runs a VR Audio special interest group[5].

There were a few short weeks to learn the technology, shoot and edit. But we now had a plan, a team, and something to film.

Cameras and Audio Equal Trouble and Strife

Then, bad news. The practice camera provided was incompatible with Andrew’s hardware, and we could not borrow the production camera until less than a day before the shoot. Our choices were to quit, or go in blind.

Worse news. Darius would be overseas for an extended period. Understandably he did not want to risk his very expensive audio capture equipment with somebody else; particularly when it would be suspended over salt water.

Next week

In next week’s edition: All At Sea But Problem Solving: We improvise to solve our production problems and capture our footage, but even editing 360 creates its own obstacles

Useful Links

Many of these are Melbourne (Australia) based, but you can find similar resources wherever you are.

  1. Part 1 of this Series: https://www.appearition.com/mucking-about-with-360-videosphere-vr/ 
  2. Nathan Beattie’s VR Developers’ Meetup: https://www.meetup.com/Melbourne-VR/
  3. Leah Bunny and Emily Harridge’s Real World VR Meetuphttps://www.meetup.com/Melbourne-VR-Virtual-Reality-Meetup/
  4. ABC TV’s first 360 VideoSphere production: http://mobile.abc.net.au/news/2015-12-03/vr:-the-life-of-a-bronc-rider/6966832
  5. Darius Kedros and Sally Kellaway’s VR Audio Group: http://www.meetup.com/Melbourne-AR-VR-Audio-Meetup/

 

CONTACT US






Staff Blog: Mucking about with 360 VideoSphere VR Part 1

I looked about Antarctica in all directions. From the snowy landscape rising towards the centre of the continent to the waves breaking against the beach; a beach made of large pebbles, clearly visible at my feet.

VideoSphere (360 video) is real world footage that enables you to look in all directions including up and down even on a regular computer. Use a VR (Virtual Reality) headset and it will fill your field of view and move with your head. This means you are no longer limited to a window predefined by somebody else. I’m a history geek so I used it for Cultural Heritage. However, if it is broadcast live, at your sister’s wedding or a music concert, you can be 1,000 miles away, but see exactly what you would see sitting in the audience.

My first experience was with its sibling, PhotoSphere (which are 360 still images). I was “standing” on the Antarctic beach, while physically located in my kitchen in Australia, thanks to Paul Pichugin[1].

screen-shot-2016-11-24-at-2-34-14-pm

360 Video in Practice 

If you have not come across VideoSphere it is only a matter of time. Mainstream news organisations, tourism and sporting bodies are already using it as more than a novelty. You can even produce it yourself for family and friends.

At the entry level, you can pick up a decent camera with respectable sound capture, such as the Samsung Gear 360[2], for under US$500 (plus software such as Adobe Premiere[3] and a computer with enough grunt to run it. If you’re a serious professional and have US$45,000 handy, you can pick up Nokia Ozo, a bargain down from its original US$60,000 price tag[4]

Another factor is sound. Capturing ambient noise is not difficult. If you would like to capture directional sound and edit layers of sound (e.g. voice, action, music, and ambient) to a high quality then you may need a sound technician with a directional sound rig, editing desk, and software.

Using VideoSphere Yourself 

If you’re interested in producing your own VideoSphere, having one done professionally, or are just curious about the technology, you can read this series. I’ll cover the lessons (and mistakes to avoid) that I learnt by producing this short film aboard an old-fashioned Sailing Ship[5] for the Melbourne Fringe Festival. You can watch the video below on Youtube with a Google Cardboard headset and selecting the google_cardboard_logo symbol. 

A quick aside on the furious debate – VR or not VR

You may encounter an argument that 360 should not be referred to as VR because you can look but not touch. I.e. you can’t interact the way you can with responsive computer generated (CG) content. It’s true, but it’s like arguing that a spider is not an insect. most people just don’t care. So, for the sake of simplicity I am bundling this into the Mixed Reality family.

Next week

Tune in next week for my next article: All Goes Well, Until It Doesn’t: Putting together a production in record time with technology so new it has not yet been released in the country has its risks

Useful Links
    1. Paul Pichugin’s Antarctica: http://immersiveimages.com.au/tours/antarctica/#s=pano10
    2. Samsung Gear 360 Camera: http://www.samsung.com/global/galaxy/gear-360/
    3. Adobe Premiere video editing software: https://helpx.adobe.com/premiere-pro/system-requirements.html
    4. Nokia Ozo 360 Camera: https://ozo.nokia.com/# 
    5. Melbourne’s Tall Ship Enterprize: http://www.enterprize.org.au/

 

CONTACT US