Android Head Tracking

Project Overview

This project's main goal is to be able to create an application on the Android mobile platform that will use face detection to track the head movements of the user. The face detection is used with OpenCV and Androids Face Detection API, and with these algorithms, we are able to locate the face with the device's front facing camera. After face detection has been implemented, we would need to be able to track the faces X,Y and Z coordinates (relative to the camera). With this data, we would be able to create a 3D model view where the user would be able to look around the object by simply moving their head.

Significant Updates

• May 20 Removed most of the bloat from sample code and was able to create a new android app with core files.
• May 16 Got middle of eye position and was able to give a live display of values on view
• May 8-9 Discovered helpful Face Detection/Eye Tracking algorithm, and successfully set up environment and code to work as a base.
  • Android Eye Detection and Tracking with OpenCV
• May 7 - Got face detection samples running on android device (complied on development computer)
• April 30 - Found Error with Hardware on API on Development device.
• April 23 - Gained Control of the Camera

TO DO

• Using openCV for face detection
  • Testing face detection samples on development device.
  • Finding the range of Face Detection
• Display eye positions, and other helpful information
• Enable Front Camera
• Comb through Eye Detection code to create usable functions
• Centralize the coordinates to have the middle of the screen to be (0,0)
  • Figure out a way to do Z coordinate
• Remove the preview and still be able to output face coordinates.
• Create simple openGL interface

Known Bugs/Issues

• Android Face Detection API
  • Only works for Ice Cream Sandwich [API 14.0 4.0] and up
  • Not all Devices Support the FaceDetection API
• OpenCV Samples
  • Sample Face Detection Code is not the most accurate face detection
    • Need to do more testing on faces and different lighting and back ground
  • Face Detection works when user looks directly at the camera but has issues finding sides of the face.
• Eye Tracking
  • Pupal are not tracked correctly like in the tutorial but I will not being using that data for this project and will focus on bounding box around the eyes for coordinates
• The range of face detection is still unknown but at the same time, it is not very far.
• There are multiple issues with false positives that might cause some issues with the code.
  • When multiple faces are detected, I don't know how to keep track of the main face we are looking at.
  • Faces are kept in a array of faces
• The coordinates that are obtained are based upon the size of the screen of the device.

Additional Information

Android API

• The following code allows determines if the hardware of the device support Face Detection API

// if faces is less than zero then device does not support face detection 
int faces = params.getMaxNumDetectedFaces(); 

• Used within a check function

/** 
     * faceDetectionCheck Method
     * @param currentCamera	the current Camera used for the application
     * Checks to see if the current camera passed 
     * @return
     */
    private static boolean faceDetectionCheck(Camera currentCamera){ 
    	Camera.Parameters params = currentCamera.getParameters();
        System.out.println("Checking to see if face detection works on the Device");
        // start face detection only *after* preview has started
        int faces = params.getMaxNumDetectedFaces();
        if(faces > 0){
        		System.out.println("Android Face Detection is compatable on this Device");
        		return true;
        }
        System.out.println("Android Face Detection API is not compatable");
    	
    	return false;
        }
    

OpenCV Samples

• This the the result from OpenCV face detection sample.
  • 

• OpenCV Testing with Human Face
  • 

OpenCV Eye Tracking

• Base Tutorial 1 model running on Asus Transformer
  • 

Installation

Setting up Environment

Other than the normal Android Environment; the OpenCV Android environment must also be installed onto your workstation.

• Documentation for installation are here
  • My personal environment was set up on a Mac with the Eclipse IDE.
• OpenCV does have samples that you can run. Try to run the Face Detection sample to make sure all of the libraries have been installed correctly.

Additions to Activity

• This part of code needs to be added into your onCreate or your onResume since it will load your OpenCV libraries into your application so the rest of the program can use.

 
Log.i(TAG, "Trying to load OpenCV library");
if (!OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_2, this,mOpenCVCallBack)) {
	Log.e(TAG, "Cannot connect to OpenCV Manager");
}
 

Permissions

    <uses-permission android:name="android.permission.CAMERA"/>
    <uses-feature android:name="android.hardware.camera" />
    <uses-feature android:name="android.hardware.camera.autofocus" />