VR/MR/AR convergence ever-closer with Apple’s ARKit
Here at Zubr, we tend to allow ourselves a wide margin for error when it comes to defining the boundaries between augmented, mixed and virtual reality.
Perhaps more than any other company, we’ve had our AR/VR/MR boundaries mixed and mashed up right from the start – and I think that’s a good thing. Never have we concentrated on one format and then suddenly ‘discovered’ another – it has always been more like “AR/MR content that switches between mono and stereo view!” “AR-enabled positional tracking for VR content!” “A scene that starts in MR until you step through and it becomes VR!” “Mobile AR containing mini VR scenes!”
And that’s always felt very different to what we’ve heard from others in the industry in Bristol:
“We don’t think of Google Cardboard as a real VR device, because it doesn’t have six degrees of freedom/positional tracking.”
“AR and VR are two completely separate things”
“AR is dead. It’s all about MR now.”
2014: Bringing Vuforia and Google Cardboard together
When we started experimenting with the Vuforia AR engine in 2014, we weren’t really interested in making augmented reality. Despite previously working at the Bristol-based leading AR technology supplier Kudan, and being well-aware of the possibilities, our hearts were firmly set on exploring what we could do with the Google Cardboard. Excited by the presence of a hole for camera passthrough and even a ‘3D token‘ disc supplied with every cardboard, we started manually blending Vuforia’s AR and Google Cardboard’s VR capabilities together – something that, to our surprise, no one else was interested in at the time.
Google’s 2014 Cardboard design featured a camera passthrough hole and 3D tracker token
We arrived at a point where we would use AR image targets of different sizes and shapes to position 3D content in space, viewed in true stereoscopic 3D. Larger image targets could even be used to enable 6 degrees of freedom positional tracking for non-AR virtual reality scenes. An animated GIF of one of our early demos of this type posted on Twitter drew the attention of Google’s VP of Virtual Reality, Clay Bavor, and Vuforia’s Head of Marketing, Liz Philips, who was perhaps surprised that we hadn’t built it with Vuforia’s own VR compatability toolset that had only just been released.
One of Zubr’s 2014 experiments in combining augmented and virtual reality
2015: Augmentation with depth-sensing devices
To continue the success of the original low-cost Cardboard VR device, Google unveiled the Cardboard V2 in 2015. Though many aspects of the design were improved, such as the lens, build quality and portability; Google had removed the camera passthrough hole on the front. Through doing this, Google had effectively chosen to separate AR and VR, at least for the time being – perhaps in recognition that no developers had created any meaningful AR/MR content with the first Cardboard headset.
Running in parallel to the Google Cardboard, Google had a different team working on the Project Tango – a depth-sensing mobile device geared entirely towards augmented reality. Of course, it wasn’t long before people started exploring how they could bring augmented and virtual developments together.
This is where Microsoft comes in.
2016: Microsoft Hololens woos the corporate crowds
A new type of headset that creates visuals based on the centuries-old technique of Pepper’s Ghost, and depth-sensing abilities derived from the Kinect, the Microsoft Hololens very quickly became the ruler of a new segment of the industry – keenly pushed by Microsoft – called Mixed Reality.
It’s an impressive new direction for head-mounted hardware, and we’re certain that there’s a great deal of mileage in it, but sadly, the Hololens currently suffers from its’ very own Hype Train. Perhaps spurred by the business-class-feeling £3,000 price tag and relentlessly circulated stock images of ‘business users’ reaching out to touch ‘holograms’ in amazement, the Hololens has made incredible inroads to corporate environments, being demonstrated as an effective option for technical training and health & safety amongst other uses.
And, in a clichéd attempt to prove its’ worth to tech-savvy business users – despite having a field of view the size of a business card – most of those stock images you see will include a little OTT. No, not Over-the-Top – I’m talking about the Obligatory Turbine-Thing:
A Business Hololens User indulges in his OTT
Meanwhile, as we move into 2017, we can see increasing evidence that the previously-almighty VR hype train is slowing down fast. Sadly, some VR companies depended heavily on this hype translating to real business; and many are now struggling.
So what fresh innovations are popping up to keep everyone interested? Enter Apple.
2017: Apple unveils ARKit
There has been speculation going back over a year that Apple had been planning a major augmented reality toolset, with a number of relevant business acquisitions. In Spring 2017, the rumours turned out to be true – and Apple unveiled their awe-inspiringly-robust trackerless SLAM (simultaneous localisation and mapping) augmented reality system – ARKit.
We had already been testing our content with Kudan’s impressive SLAM system, which works across iOS and Android – so we were quick to try ARKit, and we not disappointed with the results:
— Zubr (@ZubrVR) July 17, 2017
“We’ve been waiting for ARKit for a long time, now our 4D scans can roam free…”
Naturally, we’re absolutely delighted that robust SLAM for mobile devices is finally here. Needless to say, it wasn’t long before we went full-circle and created a stereo split for ARKit demos, meaning you can experience the powerful AR engine through an immersive head-mounted display. In fact, we went full-on Microsoft, and created our very own Obligatory Turbine-Thing demo:
— Zubr (@ZubrVR) July 31, 2017
Keep your Hololens in its’ fancy hand-stitched bag, we’ve got this covered!
For many people, this is an exciting new development that has enticed them into the augmented reality world; simple and yet powerful enough for anyone to focus on creating fun and original content, instead of worrying about how to set up tracker images – which is great. Some VR developers have even hopped straight over to ARKit simply to explore its’ abilities for 6-degrees of freedom positional tracking, and they haven’t been disappointed.
2018: Standalone VR, Affordable MR, and Robust AR will bring convergence closer than ever
So, if we look at the big picture at this point; it’s not hard to see that we are on the cusp of a real AR, MR and VR melting pot – with technologies overlapping, developers crossing boundaries and even content that swings from one format to the other with the touch of a button.
VR Hardware will be reinvigorated by the release of new standalone headsets – independent from PC units and yet more capable than smartphone headsets, they hold the potential for a renewed push to bring VR to the mainstream.
Also, dedicated MR headsets will soon be affordable and versatile – for business users and entertainment purposes alike.
Finally, AR is likely to retain pole position out of the three technologies in regards to general uptake, at least for a while.
For us, it’s very exciting that hybrid experiences and an ‘XR’ approach are finally gaining traction. Although, that is business as usual in our studio – looking at our portfolio is sort of like taking an ‘Is it VR, MR or AR?’ quiz. For the big players – we’re all expecting Apple to cosy up more with VR technologies, especially now ARKit is out there. For Google, expect much-increased interplay between their VR and AR ventures, as they attempt to catch up with Apple on the AR side.
Oh, and Google – We knew you were making a mistake when you filled in that camera passthrough hole on the Cardboard V2 and Daydream headset. If you could please bring it back for your next headset, we won’t have to use craft blades so often!