Difference between revisions of "ArtifactVis with Android Device"

From Immersive Visualization Lab Wiki
Jump to: navigation, search
(→‎Installing OSG for Android on Linux OS)
(→‎Developers)
 
(84 intermediate revisions by one user not shown)
Line 1: Line 1:
 
===Project Overview===
 
===Project Overview===
 +
 +
This project allow users to use the android device (as a client) to connect with CalVR (as a server) by the TCP protocol, then they can move any object and interact with it in the virtual world by using the android device.
 +
It allow users to move objects on the CalVR by using any kind of mode such as Move World, Scale, Drive and Fly, and then see the object movement changes on the android device. Also, they can use the android device (client side) to move the camera around the object to see the object from all sides. Finally, they can use the android device (client side) for picking objects to receive pictures about this object. They can use all these techniques with Artifact Archeology project to interact with the historical objects in the real world. Also they can use this plug-in with other projects.
 +
  
 
===Usage Instructions===
 
===Usage Instructions===
  
 
*'''Server'''
 
*'''Server'''
−
** This is the path to find the server code '''write the path here'''
+
** This is the path to find the server code '''/home/saseeri/CVRPlugins/calit2/MagicLens'''
 
** IP Address: Use this command line to find the IP address '''ifconfig'''
 
** IP Address: Use this command line to find the IP address '''ifconfig'''
 
** Use this PORT '''28888'''
 
** Use this PORT '''28888'''
−
** This is the path to use any object '''write the path here'''
+
** We can find objects on '''/home/saseeri/Development/OpenSceneGraph-Data''' folder
  
 
*'''Client'''
 
*'''Client'''
−
** This is the path to find the client code '''write the path here'''
+
** This is the path to find the client code '''/home/saseeri/CVRPlugins/calit2/MagicLens''', the name of the folder '''OSGTest1-AndroidClient'''
−
** IP Address: Use the same IP address on the server side and wirte it on the code   
+
** IP Address: Use the same IP address on the server side and put it on the code   
 
** Use this PORT '''28888'''
 
** Use this PORT '''28888'''
−
** Use the same object in client side and put the object on the android memory card and wrtie this path on the code '''write the path here'''
+
** Use the same object that is used in server side and put the object on the android memory card and wrtie this path on the code '''/mnt/sdcard/create_any_file name/object_name'''
  
 
'''To compile and run the code do the following:'''
 
'''To compile and run the code do the following:'''
  
 
Start with the server  
 
Start with the server  
−
*Go to this path on the terminal '''write the path here'''
+
*Go to this path on the terminal '''/home/saseeri/CVRPlugins/calit2/MagicLens'''
  
 
'''Write on the terminal'''  
 
'''Write on the terminal'''  
Line 26: Line 30:
 
   '''CalVR'''
 
   '''CalVR'''
  
−
After that Copy the client code to the eclips workspase and do compile to the Android ndk (C++) by using the following:
+
After that, Copy the client code to the eclips workspase and do compile to the Android ndk (C++) by using the following:
−
*Go to this path on the terminal '''write the path here'''
+
*Go to this path on the terminal '''/home/saseeri/workspace/OSGTest1-AndroidClient'''
  
 
'''Write on the terminal'''  
 
'''Write on the terminal'''  
 
   '''ndk-build'''
 
   '''ndk-build'''
 
Then go to the eclipse right click on the project then run it after you connect the android device
 
Then go to the eclipse right click on the project then run it after you connect the android device
 +
 +
*Click on Start Client
 +
*Then Connect Server
 +
 +
 +
[[Image:client.jpg]]    [[Image:plane.jpg]]
 +
  
 
===Software Tools and Equipment===
 
===Software Tools and Equipment===
Line 43: Line 54:
  
 
'''On Phone Device (Client):'''
 
'''On Phone Device (Client):'''
−
* Eclips
+
* Eclipse
 
* Linux-sdk
 
* Linux-sdk
 
* Android-ndk
 
* Android-ndk
Line 50: Line 61:
 
'''Equipment'''
 
'''Equipment'''
 
*Android Device (I used Galaxy Phone Samsung)
 
*Android Device (I used Galaxy Phone Samsung)
 +
  
 
===Installing OSG for Android on Linux OS===
 
===Installing OSG for Android on Linux OS===
  
−
@Build the environment for OSG on Android
+
This is the path to find installation of OSG for Android on Linux OS
−
1. download and install "android sdk"
+
−
 
+
−
2. download "android ndk" and do following steps.
+
−
*unzip the ndk and put the ndk folder into the directory you want (we use the path "/home/username/" as example for the following steps)
+
−
*open the directory "/home/username/" and press Ctrl+H to show hidden files
+
−
*open the hidden file ".cshrc" if the shell is "tcsh"
+
−
*add the following 2 lines in ".cshrc"
+
−
#android
+
−
setenv PATH /home/username/android-ndk-rX  #(X is the version of ndk, depends on what version of ndk you download)
+
−
*open a terminal, cd into "/home/username/" and enter the command "source .cshrc"
+
−
 
+
−
3. install OpenSceneGraph(OSG):
+
−
*create a folder for OSG(e.g. create a folder "OSGAndroid" under /home/username/, full path: /home/username/OSGAndroid)
+
−
*under the folder "OSGAndroid", open a terminal and enter the command "svn co http://www.openscenegraph.org/svn/osg/OpenSceneGraph/tags/OpenSceneGraph-3.0.1" (here is OSG version 3.0.1)
+
−
*the following steps are based on the website:http://forum.openscenegraph.org/viewtopic.php?t=10076
+
−
*create 2 folders "build", "osginstall" under the main OSG folder (e.g. /home/username/OSGAndroid/OpenSceneGraph-3.0.1)
+
−
*download the 3rdparty package from this website: "http://www2.ai2.upv.es/difusion/osgAndroid/3rdpartyAndroid.zip"
+
−
*unzip the 3rdparty package and put it into the main OSG folder
+
−
*cd into the "build" folder and open a terminal
+
−
*in the terminal, enter this:
+
−
cmake .. -DOSG_BUILD_PLATFORM_ANDROID=ON -DDYNAMIC_OPENTHREADS=OFF -DDYNAMIC_OPENSCENEGRAPH=OFF -DOSG_GL_DISPLAYLISTS_AVAILABLE=OFF -DOSG_GL_MATRICES_AVAILABLE=ON -DOSG_GL_VERTEX_FUNCS_AVAILABLE=ON -DOSG_GL_VERTEX_ARRAY_FUNCS_AVAILABLE=ON -DOSG_GL_FIXED_FUNCTION_AVAILABLE=ON -DOSG_CPP_EXCEPTIONS_AVAILABLE=OFF -DOSG_GL1_AVAILABLE=OFF -DOSG_GL2_AVAILABLE=OFF -DOSG_GL3_AVAILABLE=OFF -DOSG_GLES1_AVAILABLE=ON -DOSG_GLES2_AVAILABLE=OFF -DJ=4 -DCMAKE_INSTALL_PREFIX=/home/username/OSGAndroid/OpenSceneGraph-3.0.1/osginstall
+
−
 
+
−
be aware of the last 2 parameters, -DJ=number of CPU cores in your computer, and the install path
+
−
+
−
*after it configure, enter the command "make" in terminal, after it finished, enter the command "make install" (make and make install will take some time)
+
−
 
+
−
4. assume you downloaded and installed "Eclipse IDE for Java Developers" and installed android sdk with it. The we install CDT for Eclipse (allow you to edit .cpp, .h files in Eclipse)
+
−
*in Eclipse, go to "Help" (on the top)--> "Install New Software..." --> "Add...", enter "CDT" in the field "Name", enter "http://download.eclipse.org/tools/cdt/releases/galileo" in the field"in "Archive...". Then just check all items and next->next..->finish to install CDT
+
−
+
−
 
+
−
 
+
−
@Compile native (c++,c) code for Android
+
−
1. open a terminal and cd into the Eclipse workspace then cd into the classes folder (e.g. /home/username/workspace/project_name/bin/classes)
+
−
2. enter the command "javah -jni package.name.ClassThatHasNativeFunctions"
+
−
3. open a terminal and cd into "/home/username/workspace/project_name", you can see the header file for java native functions is generated (named as "package_name_ClassThatHasNativeFunctions.h")
+
−
4. in Eclipse, create a folder named "jni" under your Android project, copy the header file (package_name_ClassThatHasNativeFunctions.h) to the jni folder
+
−
5. create another .cpp file (say "native.cpp") to include the header (package_name_ClassThatHasNativeFunctions.h) and implement the functions
+
−
6. create a file named "Android.mk" which is the make file for Android native code, edit Android.mk
+
−
an example of Android.mk:
+
−
----------------------------------------------------------
+
−
LOCAL_PATH := $(call my-dir)
+
−
 
+
−
include $(CLEAR_VARS)
+
−
include /some/libraries/you/need
+
−
 
+
−
LOCAL_MODULE    := NativeLib
+
−
LOCAL_SRC_FILES := native.cpp
+
−
LOCAL_LDLIBS +=  -llog -ldl
+
−
 
+
−
include $(BUILD_SHARED_LIBRARY)
+
−
-----------------------------------------------------------
+
−
 
+
−
optional:
+
−
you can add a file named "Application.mk"
+
−
an example of Application.mk
+
−
-----------------------------------------------------------
+
−
APP_STL := gnustl_static
+
−
APP_CPPFLAGS := -frtti -fexceptions
+
−
APP_ABI :=  armeabi-v7a #armeabi-v7a #specify which architecture(s) your application supports
+
−
-----------------------------------------------------------
+
−
 
+
−
7. to compile the native code, open a terminal and cd into "/home/username/workspace/project_name" and enter the command "ndk-build"
+
−
8. Press the "Run" button in Eclipse to run the application on Android
+
  
−
Note: Be sure to add the following lines in the java code where you declare the java native functions
+
'''/home/saseeri/CVRPlugins/calit2/MagicLens/Installing OSG for Android on Linux OS.pdf'''
−
static{
+
−
System.loadLibrary("NativeLib"); (must be the same as the module name you specified in "LOCAL_MODULE" in Android.mk
+
−
}
+
  
−
===Goals===
 
  
 
===Screen Shots===
 
===Screen Shots===
  
−
===Future Work===
+
[[Image:1.jpg]]    [[Image:2.jpg]]    [[Image:3.jpg]]
  
−
===Developers===
+
[[Image:4.jpg]]    [[Image:5.jpg]]    [[Image:6.jpg]]
  
  
 +
We can translate the object from the android device by touch screen
  
−
===Progress===
 
  
−
====Week 1====
+
[[Image:translat11.jpg]]    [[Image:translat22.jpg]]    [[Image:translat33.jpg]]
  
−
- look for tutorials on OSG                  (Done part of them)
 
  
−
- creating boxes and spheres                (Done)
+
Also, we can rotate the object from the android device
  
−
- displaying images from JPEG files          (Working on it)
 
  
−
- mouse control: move/rotate objects in 3D  (Done the rotation)
+
[[Image:rotate1.jpg]]    [[Image:rotate2.jpg]]
  
−
- understand the concept of a scene graph    (Done)
 
  
−
====Week 2====
+
===What is done so far ...===
 +
* load model into Android Device and CalVR with osg
 +
* Connect the Android Device and CalVR over wifi (by using TCP protocol)
 +
* Move objects on the CalVR by using any kind of mode such as Move World, Scale, Drive and Fly, and then see the object movement changes on the android device
  
  
 +
===Future Work===
 +
* Move the camera around the object to see the object from all sides
 +
* Picking objects to receive pictures about this object
  
−
'''First: OpenSceneGraph'''
 
  
−
The resources that I used to learn OSG
+
===Developers===
−
 
+
−
"OpenSceneGraph 3.0 Beginner's Guide book" and the OpenSceneGraph website.
+
−
 
+
−
I read the following chapters:
+
−
 
+
−
'''Chapter 1:''' The Journey into OpenSceneGraph
+
−
 
+
−
'''Chapter 3:''' Creating Your First OSG Program
+
−
 
+
−
'''Chapter 4:''' Building Geometry Models
+
−
 
+
−
'''Chapter 5:''' Managing Scene Graph
+
−
 
+
−
'''Chapter 6:''' Creating Realistic Rendering Effects
+
−
 
+
−
'''Chapter 9:''' Interacting with Outside Elements
+
−
 
+
−
And some tutorial related to the following topics:
+
−
 
+
−
* Geometry
+
−
 
+
−
* 2D Text write function to set camera position in osg from head node: for
+
−
testing, transfer 4x4 matrix of head pos
+
−
 
+
−
* Transformation (Translation, Rotation, Scaling)
+
−
 
+
−
* Shapes (Box, Cone, Sphere, Capsule, Cylinder)
+
−
 
+
−
* Polygon mode
+
−
 
+
−
* 2D Texture (load image from images file)
+
−
 
+
−
* Interaction with outside element (keyboard and mouse)
+
−
 
+
−
I read the previous topics and ran them on my PC and made some changes to the code to understand what each method does in the code.
+
−
 
+
−
 
+
−
'''Second: CalVR
+
−
'''
+
−
 
+
−
I read the CalVR website, and I will focus on it more on Monday
+
−
 
+
−
 
+
−
'''Third: Android'''
+
−
 
+
−
I started with android: installed it on my laptop, done the "Hello World" program and ran it in my android devise. I also read about the following topics:
+
−
 
+
−
* Camera
+
−
 
+
−
* Kind of sensor
+
−
 
+
−
* Activity
+
−
 
+
−
* Create list, buttons
+
−
 
+
−
 
+
−
'''roadblocks I have encountered:'''
+
−
 
+
−
* Working with OSG installation
+
−
 
+
−
* Write a makefile
+
−
 
+
−
* Working with some linux commands.
+
−
 
+
−
* Understanding CalVR (but I did not give CalVR much time to work on)
+
−
 
+
−
* There were no simple Examples for OSG on the internet so I used the book "OpenSceneGraph 3.0 Beginner's Guide book" to practice some simple examples.
+
−
 
+
−
* I couldn't understand some mathematical equations in your slides. 
+
−
 
+
−
 
+
−
 
+
−
'''Summery'''
+
−
 
+
−
Look for tutorials on OSG and compile them on your PC in my lab           '''Done'''
+
−
 
+
−
Creating boxes and spheres           '''Done'''
+
−
 
+
−
Displaying images from JPEG files           '''Done''' (I used the image as texture)
+
−
 
+
−
Mouse control: move/rotate objects in 3D                              '''Done''' (move/rotate objects in 3D and Picking objects)
+
−
 
+
−
Understand the concept of a scene graph           '''Done'''
+
−
 
+
−
Go through the slides on this web site           '''Done''' (Lecture 1, 2, 3, 4, 5, 6, 10) I read in general and will continue reading
+
−
 
+
−
====Week 3====
+
−
 
+
−
Reading and understanding the code (Done 60%)
+
−
 
+
−
Read about socket programming (Done without any practical code)
+
−
 
+
−
====Week 4====
+
−
 
+
−
Install OSG on the Android device  (Motorola Xoom tablet)  '''Done'''    But it's doesn't work
+
−
Load any Object and I apply rotation, translation, and scaling it by touch screen  '''Done'''
+
−
 
+
−
====Week 5====
+
−
 
+
−
Continu install OSG on the Android device but on the Galaxy mobile  ''Done'''
+
−
Load any Object and I apply rotation, translation, and scaling it by touch screen  '''Done'''
+
−
load excavation site model (osg file) into mobile with OSG      '''Done'''    But the output is a black screen
+
−
 
+
−
====Week 6====
+
−
 
+
−
osg::MatrixTransform * getObjectTransform () returns the position of the data set in room coordinates; send this to phone; on phone, put it in a MatrixTransform just above the node with the cow
+
−
 
+
−
===TODO===
+
−
 
+
−
* load excavation site model (VRML file) into tablet with osg:
+
−
** osg::MatrixTransform * getObjectTransform () returns the position of the data set in room coordinates; send this to phone; on phone, put it in a MatrixTransform just above the node with the cow
+
−
** Goal: when flying around data set in CalVR, the view on the phone should change accordingly
+
−
** getHeadMat() returns the head position in room coordinates: this is constant at a desktop, but changes in the caves
+
−
 
+
−
- write function to set camera position in osg from head node: for
+
−
testing, transfer 4x4 matrix of head position over wifi to tablet:
+
−
this will effectively copy the view on the Calvr system to the tablet
+
  
−
- sense touch on artifact, then send touched artifact ID to Calvr
+
'''Software Developer'''
−
computer, which will send associated photograph to tablet; display
+
* Sahar Aseeri
−
photograph on tablet. This will allow the user to view pictures
+
−
associated with artifacts on the screen
+
  
−
- connect tracking system to tablet to allow moving tablet around
+
'''Project Advisor'''
−
starcave and view excavation site model from arbitrary perspectives on
+
*Jurgen schulze
−
tablet screen.
+

Latest revision as of 18:08, 27 August 2012

Contents

Project Overview

This project allow users to use the android device (as a client) to connect with CalVR (as a server) by the TCP protocol, then they can move any object and interact with it in the virtual world by using the android device. It allow users to move objects on the CalVR by using any kind of mode such as Move World, Scale, Drive and Fly, and then see the object movement changes on the android device. Also, they can use the android device (client side) to move the camera around the object to see the object from all sides. Finally, they can use the android device (client side) for picking objects to receive pictures about this object. They can use all these techniques with Artifact Archeology project to interact with the historical objects in the real world. Also they can use this plug-in with other projects.


Usage Instructions

  • Server
    • This is the path to find the server code /home/saseeri/CVRPlugins/calit2/MagicLens
    • IP Address: Use this command line to find the IP address ifconfig
    • Use this PORT 28888
    • We can find objects on /home/saseeri/Development/OpenSceneGraph-Data folder
  • Client
    • This is the path to find the client code /home/saseeri/CVRPlugins/calit2/MagicLens, the name of the folder OSGTest1-AndroidClient
    • IP Address: Use the same IP address on the server side and put it on the code
    • Use this PORT 28888
    • Use the same object that is used in server side and put the object on the android memory card and wrtie this path on the code /mnt/sdcard/create_any_file name/object_name

To compile and run the code do the following:

Start with the server

  • Go to this path on the terminal /home/saseeri/CVRPlugins/calit2/MagicLens

Write on the terminal 

  make
  ...  
  CalVR

After that, Copy the client code to the eclips workspase and do compile to the Android ndk (C++) by using the following:

  • Go to this path on the terminal /home/saseeri/workspace/OSGTest1-AndroidClient

Write on the terminal 

  ndk-build

Then go to the eclipse right click on the project then run it after you connect the android device

  • Click on Start Client
  • Then Connect Server


Client.jpg Plane.jpg


Software Tools and Equipment

Software Tools

On PC (Server):

  • Linux Operating System
  • OpenSceneGraph 3.1
  • CalVR

On Phone Device (Client):

  • Eclipse
  • Linux-sdk
  • Android-ndk
  • OpenSceneGraph 3.1 Android

Equipment

  • Android Device (I used Galaxy Phone Samsung)


Installing OSG for Android on Linux OS

This is the path to find installation of OSG for Android on Linux OS

/home/saseeri/CVRPlugins/calit2/MagicLens/Installing OSG for Android on Linux OS.pdf


Screen Shots

1.jpg 2.jpg 3.jpg

4.jpg 5.jpg 6.jpg


We can translate the object from the android device by touch screen


Translat11.jpg Translat22.jpg Translat33.jpg


Also, we can rotate the object from the android device


Rotate1.jpg Rotate2.jpg


What is done so far ...

  • load model into Android Device and CalVR with osg
  • Connect the Android Device and CalVR over wifi (by using TCP protocol)
  • Move objects on the CalVR by using any kind of mode such as Move World, Scale, Drive and Fly, and then see the object movement changes on the android device


Future Work

  • Move the camera around the object to see the object from all sides
  • Picking objects to receive pictures about this object


Developers

Software Developer

  • Sahar Aseeri

Project Advisor

  • Jurgen schulze
Retrieved from "http://ivl.calit2.net/wiki/index.php?title=ArtifactVis_with_Android_Device&oldid=5201"