Homework4W13
Contents |
For this assignment you can obtain 100 points, plus up to 10 points for optional work.
The goal of this assignment is to create an application which uses exclusively the Microsoft Kinect for user interaction.
Microsoft Kinect
Every team is going to get to borrow a Kinect for Windows from the instructor for the remaining weeks of the quarter. If your team has two members, please take only one Kinect. The Kinects can be picked up from the instructor's office at Atkinson Hall, room 2125. The office door should be open during business hours, or you can email to make an appointment. If the instructor is not in the office, please sign one of the borrow agreements and leave it on the desk, then take a Kinect from the box on the floor by the wall with the whiteboard.
The Kinect drivers and SDK are already installed on the lab machines. For your home computer, you can find the SDK at Microsoft's Website.
Application (100 Points)
The goal is to create an application which allows the user to walk to EBU1, climb up the wall like Spiderman, then jump off it, wearing a wing suit and land in the Price Center courtyard. Use this 3D model of the campus for the visuals.
If you prefer, you can think of the assignment in which you gesture-control a flying monkey to do the same task.
Your application needs to have the following features:
- At the beginning of the application you should have a calibration function which registers a neutral position of the user. This will be needed to parse the navigation gestures more accurately.
- You will need to use the skeleton extraction functionality to find the positions of the user's limbs.
- The camera should provide a first person view. It is not necessary to render the user's avatar, or provide other views, but these can get you extra credit (see below).
- You will need to implement three modes of navigation:
- Walk to the building: the user starts in the engineering courtyard, by the entrance of the CS building. The user will need to walk in place to move forward, the direction can be changed by pointing with the torso an extended arm, or another gesture of your choice.
- Climb up the wall: Once the user reaches EBU1 (either anywhere at the building, or in a specific spot of your choice), the navigation mode automatically changes so that the user can climb up the wall. It is sufficient to track the arms and hands moving up and down to climb. Faster movement should make you climb faster.
- Soar to the Price Center: Once the user gets to the top of the building, the navigation mode changes to flying. The user can now hold their arms out and tilt them left and right like wings of an airplane to steer, and move both arms up or down to control the speed of moving forward. Adjust the rate of falling so that it is possible to reach the Price Center without much effort.
- You don't need to check for collisions with other buildings in any of the navigation modes, meaning that the user can walk or fly through other buildings as if they were not there.
- The navigation modes should switch automatically, once the user gets to EBU1 (or a designated spot at it), and when the user reaches the top of the building.
- You will need to measure the time it takes from when the user starts walking until they land at the Price Center. You don't need to continuously display the time (except for extra credit), but need to display the total time in the console window once the player has landed. Who will get the fastest time?
Grading
On the due date, you will need to demonstrate your application with a Kinect in the lab on a lab computer or your own laptop.
Extra Credit (10 Points)
You can achieve 10 points of extra credit if you implement one of the following additional features:
- Bird's eye view: Add a camera mode, switchable with a special gesture, in which the camera looks at the player from a distance, so that the entire player and the positions of their limbs can be seen. You will need to render an avatar where the player is, mapping the skeletal values from the Kinect to the avatar's geometry. The avatar needs to consist of three dimensional body parts, so lines don't count, but rectangular boxes or ovals would work, similar to the animated robot task in CSE 167.
- Dashboard: Display a campus map in a corner of the screen with an indicator for where the player is, show velocity and altitude when walking, climbing or flying numerically or with gauges, and show how much time has passed since the application started, and how long each of the three parts of the navigation task (walking, climbing, flying) have taken.
- Two-player mode: When two people are in front of the Kinect, it will return two skeletons and even keep track of who is who if the players switch places. Add support for two players with a split screen mode, in which each player sees the scene from their point of view. That way the players can compete for who will land at the Price Center first.
- Add voice commands: The Kinect has a built-in microphone array, which is supported by the Microsoft SDK. Add voice commands to your application, for instance to reset the player position to the starting point, or go to one of the two points at which the navigation mode changes.
Useful Links
Kinect SDK and Visual Studio 2010 OSG camera control with a matrix
