Pose Estimation for a Mobile Device

Progress

Feb 13, 2013

• Succeed to transfer text from mobile device to the PC through LAN.
• Modify the program into multi-threaded. now the total processing time is 51 [ms], FPS: 19.75.

Feb 10, 2013

• Survey for implementation for TCP/IP communication between android device and the PC.

Feb 7, 2013

• Upload a demo video

 http://www.youtube.com/watch?v=6-dbbrgPk7U&feature=share&list=UUxO9pGQwtWujpAMGAoI21TA

Jan 31, 2013

• Porting feature detection code into android. Need to modify some SSE2 code into NEON.

Jan 30, 2013

• System timing result for onlineSFm per frame.

- correspondence time......[ms]: 31.488
- reprojection matchtime...[ms]: 1.87859
- pose estimate time.......[ms]: 15.13
- filter estimate time.....[ms]: 1.09726
- updateKeyFrames time.....[ms]: 4.43677
- BA time..................[ms]: 14.0742

• total running time is 68 ms on a single thread with CPU 2.2Ghz. A possible improvement is to separate the updateKeyFrame and BA into another thread so we can pipeline it when computing the correspondence.

Jan 29, 2013

• Modify the feature tracking to update the feature description at every frame. This essentially improve the robustness of the feature tracking result and the pose estimation quality.
• (Done) Remove features with inconsistent measurements.
• (Done) Redesign the mechanism of adding features at each key frame. (We should only add features that make the density of the features are equally spread.) Note: currently I used bucket with non-maximum suppression to achieve similar effect in equal density for features. Ideal approach would be using a quad-tree to efficiently compute the density of the features and then put new features at those areas.

Jan 24, 2013

• Add the point prior information into structure and pose estimation. Now the filtering result is kept in the optimization so the reconstruction result uses more measurements with slightly increasing the computation cost.

Jan 21, 2013

• Setup the android environment on Win7. Able to run sample programs on mobile devices.

 Environment
 - Android SDK with eclispe bundled for windows
 - Android NDK r8 for windows
 - OpenCV 2.4.3 for android. No need to recompile the library.

 Test device:
 - ASUS TF101
 - HTC EVO 3D

 Test program:
 - acceleration reader.
 - basic camera reader.

Jan 17, 2013

• Started project.

To Do List in a week

PC side:

• Add motion model of the camera into pose estimation cost function.

Mobile side:

• Perform feature detection on the mobile device and record its performance.
• Setup the connection between a mobile devise and the PC.

Goal

PC side:

• Refine the feature matching to consider the density of the features at a frame.

• Incorporate the motion model of the camera into optimization.

• Make the program to be multi-threaded.

Mobile side:

• Setup basic data capturing program

• Test on data transfer between mobile device and PC.

• Transfer part of preprocessing program to the mobile device.