AndroidAR

Introduction: Virtual inspection on mobile device

This project uses 2 infrared trackers and a tracking target that is attached to some android device. The data is collected from the trackers by CalVR in the form of a homogeneous 4x4 matrix which is processed by a CalVR plugin that sends packets using UDP to whoever sends a request packet. It is sent as a comma separated string and then unwrapped an parsed on the android device; stripping the position and axis column vectors.

On the android device, we are using the Min3D framework which is based on a scene graph and has excellent wavefront .obj file loading. When you first start the app, you a presented with a screen that allows you to choose which model you want shown. After picking a model, it should appear on the screen along with some information text in the upper left hand corner. Pressing the context menu button and then "Start tracking" will enable the tracking feature and continuously update the camera of the scene graph based on what we receive from the CalVR plugin.

Introduction: Multiple client drawing in virtual space

In this project we use the same tracking as before, but now must maintain state on a central server to let the clients know if points must be drawn in remote locations. Here we have decided on a simple 2-player scheme, as we are limited on our amount of tracking devices anyway. This can easily be changed to a more elaborate, robust and scalable structure, and research was made to see if a real-time turnbased [1]) approach would be feasible.

The objective of the project was to be able to draw in a virtual 3d space on a mobile device, and observe a partners drawings.

Equipment

3 TrackD infrared trackers

2 Android devices (Samsung Galaxy S2 and HTC Desire Z)

CalVR C++ Plugin. (To manage tracking targets and drawing information)

Goals

Rendering on Android (done)

Camera tracking (done)

Head tracking (done)

Multiple client drawing in virtual space (in progress)

Developers

Kristian Theilgaard Hansen

Mads Pedersen

Pictures

[2] Mads holding the final tablet and tracking the chair

[3] Xoom 2 running the Virtual Reality app with IR Tracking

[4] Xoom 2 running the Virtual Reality app

Future work

The project has not focused on various security aspects that may appear when communicating over an open network. No authentication is done with the mobile phones connected to the tracking server, but at worst, an attacker will be able to flood the server or find an offset location from tracker to our virtual origin in front of the desk computer.

Optimization (more clients than 2) and robust general implementation (the ability to move to other machines. Addresses are hardcoded) has been given a lower priority in favor of features.

Related work

Greg is working on doing all the tracking internally in the phone, using all its sensors and the camera(s).