AndroidAR

Introduction: Virtual inspection on mobile device

This project uses 2 infrared trackers and a tracking target that is attached to some android device. The data is collected from the trackers by CalVR in the form of a homogeneous 4x4 matrix which is processed by a CalVR plugin that sends packets using UDP to whoever sends a request packet. It is sent as a comma separated string and then unwrapped an parsed on the android device; stripping the position and axis column vectors.

On the android device, we are using the Min3D framework which is based on a scene graph and has excellent wavefront .obj file loading. When you first start the app, you a presented with a screen that allows you to choose which model you want shown. After picking a model, it should appear on the screen along with some information text in the upper left hand corner. Pressing the context menu button and then "Start tracking" will enable the tracking feature and continuously update the camera of the scene graph based on what we receive from the CalVR plugin.

Headtracking was implemented in a second iteration on this project with research being done on right triangles to affect the view of our virtual object. If a viewer is looking through a window displaying an object, this object must appear translated on the respective axis in which the eyes moved. A tracker on a viewers head monitors the position of the head to appropriately simulate the translation on the screen.

Introduction: Multiple client drawing in virtual space

In this project we use the same tracking as before, but now must maintain state on a central server to let the clients know if points must be drawn in remote locations. Here we have decided on a simple 2-player scheme, as we are limited on our amount of tracking devices anyway. This can easily be changed to a more elaborate, robust and scalable structure, and research was made to see if a real-time turnbased [1]) approach would be feasible.

The objective of the project was to be able to draw in a virtual 3d space on a mobile device, and observe a partners drawings.

Equipment

3 TrackD infrared trackers and infrared cameras.

2 Android devices (Samsung Galaxy S2 and HTC Desire Z).

CalVR C++ Plugin. (To manage tracking targets and drawing information).

Goals

Rendering on Android (done)

Camera tracking (done)

Head tracking (done)

Multiple client drawing in virtual space (in progress)

Developers

Kristian Theilgaard Hansen

Mads Pedersen

Pictures

[2] Mads holding the final tablet and tracking the chair

[3] Xoom 2 running the Virtual Reality app with IR Tracking

[4] Xoom 2 running the Virtual Reality app

Future work

Remove dependency on Min3d framework. OSG is used with other projects at Calit2, and too many features are half implemented or simply not working in Min3d.

The project has not focused on various security aspects that may appear when communicating over an open network. No authentication is done with the mobile phones connected to the tracking server, but at worst, an attacker will be able to flood the server or find an offset location from tracker to our virtual origin in front of the desk computer.

Optimization (more clients than 2) and robust general implementation (the ability to move to other machines. Addresses are hardcoded) has been given a lower priority in favor of features.

Related work

Greg is working on doing all the tracking internally in the phone, using all its sensors and the camera(s).