Daydreaming about Cardboards

Virtual Reality in Android

🔧: headless CMS: contentful.com
🐦: @MarioBodemann
📄: bit.ly/mbvrhh
- Please interrupt 👄👅!

Please follow the slides at the given address.
Wait some seconds to see if someone is writing down the address...

Audience

Android programmer? 👋
VR programmer? 👋

Agenda

Introduced at Google I/O this year.
Soon™ first hardware available.

Google Daydream

based on Google Cardboard
light weight
in different styles/interactions
You just need a phone™

Controller

uses special hardware as controller
- can get emulated by any Android phone (>= 4.4)
- has four Buttons: Volume +/-, App(Programmable), Home
- clickable Touchpad. (x/y position, clicked state)(emulated by two touches)
Print out sheet to not get to disctracted by Phone
Connects via bluetooth to Headset Phone

Headset Phone

has to be a Nexus 6P running Android Nougat
start a Daydream app once, set it up by clicking on the Gear, enabling all VR settings
Reboot phone ... ;) .
Bluetooth to connect to controller.

Complete Package

Daydream Ready
- high performance sensors for high accuracy head tracking
- displays with fast response time to minimize blur
- powerfull mobile processor
- VR System notifications
will be available in fall
will be created by hardware partners
certified by Google
Google Play for VR and Daydream Home

How does Daydream work?

use 3D data, rendering two images
- one for the left and right eye
brain combines image to 3D scene

How do you program it?

GoogleVR SDK are available for

Android (OpenGL)
iOS (Objective C)
Unreal Engine (NEW)
Unity Engine (C# /JS) x (🍎, G,💻, 🐧)

What do those SDKs provide?

lens distortion correction
spatial audio
head tracking
3D calibration
side-by-side rendering

What else do they provide?

stereo geometry configuration
user input event handling
see https://vr.google.com/developers/

Google Daydream Technical Preview

special preview build from unity
Out of the box, basic functionality

or Google VR Plugin

Installer for Unity on 🍎+💻 (or 🐧)
Plugin gives full controller support and more

Unity demo

Create a new Project

Name it

See it

Main stage: see everything important happening here

Populate stage: Add a plane

Menu > GameObject > 3D Object > Plane

Show GameObject Plane

Open inspector on righthand side

Search for Inspector > Transform > Scale

See size changed to be screen filling.
Let's make it more exciting > more objects!

Let's add more objects

A Cube appears

Move that cube.

Take a look where the mouse cursor is pointing at.

Adding one of each: Cube/Sphere/Cylinder/Capsule

Take a look at the camera preview: It does not look to colourfull. Let's change that.

Overview of next steps: Let's add materials.

Click on Favorites\All Material

Click on Asset Store

Click on an asset which is interesting/looks nice (RedBeard_Bluestone WallV2)

Check Asset count: Defines how many different materials are included. the more the nicer looking.
Hit Import package.

Hit import after checking all checkmarks (should already be the case)

No change, we have to assign the material to different objects.

Use SHIFT to click select multiple objects.

Hit the little hidden button next to Inspector > ✓ Mesh Renderer > Materials > Element 0 > Default Material … ○

Select a material to be used.
Repeat for all objects.

Looks nice, let's run it.

Run the current program to see what the user would see.
(comparable to deploying it to an emulator, checking before deploying to phone)

Looks nice, but there is nothing happening, let's change that!

Overview of adding a gravity animation

Select all gravityable objects using SHIFT and left mouse click.
Do not add the plane, since the other objects need something to land on.

Add gravity to all selected objects by hitting menu option Component > Physics > Rigidbody.

Hit play.

Show stupidly simple animation
Nice, but we can do more ....

Copying objects by using Copy and Paste
Also: I moved the camera inside of the action

Now with fancy interactions and animations.
Cannot see the other parts, lets use a cardboard! :)

Import the Unity package by going to Assets > Import Package > Custom Package.

zoomed in view

Find the checked out folder from step 1
Select the GoogleVRForUnity.unitypackage
Hit Open.
Wait

Select all and hit ok again. You have now the complete sdk imported.

Please wait ... ;)

Updating is fine for simple projects, more complex might be an issue.
I Made a Backup. Go Ahead!

Sofar no changes
We need to add the GvrViewerMain to the camera of our scene.

Select Project > Assets > GoogleVR > Prefabs

Start drag and drop the prefab.

Drop it onto the Main Camera (indicated by round rect)
Sofar no changes
We need to add the GvrViewerMain to the camera of our scene.

Again no change :(
Let's run it.

We see changes
Image distortion, left and right eye
Can we simulate rotation of the head?

Yes, we can simulate by holding down ALT and moving the cursor.

Summary of Unity Demo

Using Unity is fast
Using Cardboard SDK is easy
Using internal renderer for simulating
GvrIntent start of interoperation Android - Unity

📄 Project 📱 APK

Difficulties in using Unity

Cost for non free version exists.
Integration with other (Java) libraries difficult at best.

How to use the controller

ControllerManager manager =
  new ControllerManager(this,
    new ControllerManager.EventListener() {

  public void onApiStatusChanged(int state) {
  }

  public void onRecentered() {
  }
});

ApiStatus changed: If controller gets invalid
recenter: Long press on home button on controller, should assume current rotation is straight forward ...

Controller controller = manager.getController();
if (controller != null) {
  controller.timestamp
  controller.orientation
  controller.isTouching
  controller.touch
  controller.clickButtonState
  controller.appButtonState
  controller.homeButtonState
  controller.volumeUpButtonState
  controller.volumeDownButtonState
}

orientation: Quaternion (xyzw)
polling every frame

VrView on Web/iOS/Android

VRView, available for

Android
IOS
Web

Not for Unity!

VrPanoramaView in Android

Run through the code

How to write an photosphere app like this

Sample equirectangular image

Dependencies

common.aar, commonwidget.aar and panowidget.aar as new module or directly in gradle.
compile 'com.google.protobuf.nano:protobuf-javanano:3.0.0-alpha-7'

see Google VR Getting Started

item_layout.xml

<?xml version="1.0" encoding="utf-8"?>
<com.google.vr.sdk.widgets.pano.VrPanoramaView
  android:layout_width="match_parent"
  android:layout_height="100dp"
  />

@Override public void onBindViewHolder(…) //{

final ResourceItemViewHolder viewHolder =
  (ResourceItemViewHolder) baseHolder;
final Resources resources =
  viewHolder.vrPanoramaView.getResources();
final Bitmap bitmap =
  BitmapFactory.decodeResource(resources,
    elements.get(position));
viewHolder.vrPanoramaView
  .loadImageFromBitmap(bitmap, null);

@Override public void onBindViewHolder(…) //{

  final ResourceItemViewHolder viewHolder =
    (ResourceItemViewHolder) baseHolder;

final Resources resources =
  viewHolder.vrPanoramaView.getResources();
final Bitmap bitmap =
  BitmapFactory.decodeResource(resources,
    elements.get(position));
viewHolder.vrPanoramaView
  .loadImageFromBitmap(bitmap, null);

@Override public void onBindViewHolder(…) //{

  final ResourceItemViewHolder viewHolder =
    (ResourceItemViewHolder) baseHolder;
  final Resources resources =
    viewHolder.vrPanoramaView.getResources();

final Bitmap bitmap =
  BitmapFactory.decodeResource(resources,
    elements.get(position));
viewHolder.vrPanoramaView
  .loadImageFromBitmap(bitmap, null);

@Override public void onBindViewHolder(…) //{

  final ResourceItemViewHolder viewHolder =
    (ResourceItemViewHolder) baseHolder;
  final Resources resources =
    viewHolder.vrPanoramaView.getResources();
  final Bitmap bitmap =
    BitmapFactory.decodeResource(resources,
      elements.get(position));

viewHolder.vrPanoramaView
  .loadImageFromBitmap(bitmap, null);

@Override public void onBindViewHolder(…) //{

  final ResourceItemViewHolder viewHolder =
    (ResourceItemViewHolder) baseHolder;
  final Resources resources =
    viewHolder.vrPanoramaView.getResources();
  final Bitmap bitmap =
    BitmapFactory.decodeResource(resources,
      elements.get(position));
  viewHolder.vrPanoramaView
    .loadImageFromBitmap(bitmap, null);
}

Result

Final result of our current demo

What did we do?

Loaded a equirectangular image into a bitmap
Let it be displayed by a VrPanoramaView
Use Daydream SDK to display it

Limitations of Daydream for Android SDK

using 3D models has to be done by
- either low level OpenGL or
- 3rd party 3d engine

What will you build?

🎥 General 🎥 Controller 🎥 Designing

📄 Google VR Github

🐦@MarioBodemann

Space	Forward
Right, Down, Page Down	Next slide
Left, Up, Page Up	Previous slide
P	Open presenter console
H	Toggle this help