Project3S17

CAVE Simulator

This project is currently under construction, until we post a note on Piazza that it is complete. Regardless, you are welcome to start writing code for it. We may still change details about the project but the general idea won't change.

In this project you will need to create a VR application for the Oculus Rift, which simulates a virtual reality CAVE. Like in projects 1 and 2, you're going to need to use the Oculus SDK, OpenGL and C++.

In the discussion sections we're going to go over off-center projection with OpenGL and off-screen rendering in greater detail.

Starter Code

You can start with your code from project 1 or 2, or go back to the Minimal Example. We will re-use the calibration cube and the sky box from project 2 so at least those parts of that project will be useful to you.

Project Description (100 Points)

This picture shows what a 3-sided virtual reality CAVE can look like:

Looking closely at the image, you will find that the yellow cross bar at the top does not look straight in the image, although it does to the primary user. In the image it is bent right where the edge between the two vertical screens is. This is typical behavior of VR CAVE systems, as they normally render the images for one head-tracked viewer. All other viewers see a usable image, but it can be distorted.

In this project you are tasked with creating a simulator for a 3-sided VR CAVE. This means that you need to create a virtual CAVE, which you can view with the Oculus Rift. In this CAVE you will need to display a sky box and the calibration cube from project 2. You will need to be able to switch between the view from the user's head and a view from one of the Oculus controllers.

The virtual CAVE should consist of three display screens. Each screen should be square with a width of 2.4 meters (roughly 8 feet). One of the screens should be on the floor, the other two should be vertical and seamlessly attached to the screen on the floor and to each other at right angles, just like in the picture. This application should be used standing, with the viewpoint set to your height so that it appears as if you're standing on the floor screen. The initial user position should be with their feet in the center of the floor screen, facing the edge between the two vertical screens. In this position, the user should see the calibration cube from project 2 in front of them, again 20cm wide and at a distance of about 30cm. The sky box should also be visible. You can use the stereo sky box from project 2, or any other sky box even if it's mono. In this project it is not necessary that the user can turn the boxes off individually.

The big difference between this project and project 2 is that you need to render the scene to the CAVE walls first, instead of directly to the Rift's display. You need to do this by rendering the scene six times to off-screen buffers. Six times because there are three displays, and you need to render each separately from each eye position. To render from the different eye positions of a head-tracked user, you are going to need to render with an asymmetrical projection matrix. This link provides a great explanation (including source code) of how to create an off-center projection matrix as you will need it in this project. This picture illustrates the skewed projection pyramids originating from the two eyes:

To see the effect of rendering from the point of view of another user, you need to be able to switch the viewpoint to one of the controllers. The switch should happen when the controller's middle finger trigger is pulled, and remain active as long as it is held. Once the trigger is released, the viewpoint should go back to the user's head. Note that for stereo, you'll need to create two camera positions at the user's hand. These should be offset to the left and right of the controller position, respectively.

To make the calibration cube more useful, you need to be able to move it in all three dimensions, and make it bigger or smaller. Use both thumbsticks to allow for this. For instance, the left thumbstick could be used to move the cube left/right and forward/backward. The right thumbstick could move it up/down and the other direction makes it smaller/larger.

To help with debugging, you should be able to turn on and off debug mode with the A button. In debug mode, the user sees not only the images on the virtual CAVE screens, but also all six viewing pyramids (technically they're sheared pyramids). These pyramids start in each of the eye positions and go to the corners of each of the screens. Three screens times two eyes is six pyramids. You can visualize the pyramids in wireframe mode with lines outlining them, or use solid surfaces. In either case, you should draw those pyramids that indicate the view from the left eye in green, those from the right eye should be red.

Extra Credit (10 points max.)

TBD