Revision as of 10:21, 9 May 2017

CAVE Simulator

In this project you will need to create a VR application for the Oculus Rift, which simulates a virtual reality CAVE. Like in projects 1 and 2, you're going to need to use the Oculus SDK, OpenGL and C++.

In the discussion sections we're going to go over off-center projection with OpenGL and off-screen rendering in greater detail.

Resources:

• Off-screen rendering with modern OpenGL is explained here.
• This link provides a great explanation (including source code) of how to create an off-center projection matrix as you will need it in this project.
• The original SIGGRAPH paper on the CAVE from 1993 provides an in-depth explanation of CAVE technology, including as part of it off-axis projection.

Starter Code

You can start with your code from project 1 or 2, or go back to the Minimal Example. We will re-use the calibration cube and the sky box from project 2 so at least those parts of that project will be useful to you.

Project Description (100 Points)

This picture shows what a 3-sided virtual reality CAVE can look like:

Looking closely at the image, you will find that the yellow cross bar at the top does not look straight in the image, although it does to the primary user. In the image it is bent right where the edge between the two vertical screens is. This is typical behavior of VR CAVE systems, as they normally render the images for one head-tracked viewer. All other viewers see a usable image, but it can be distorted.

In this project you are tasked with creating a simulator for a 3-sided VR CAVE. This means that you need to create a virtual CAVE, which you can view with the Oculus Rift. In this CAVE you will need to display a sky box and the calibration cube from project 2. You will need to be able to switch between the view from the user's head and a view from one of the Oculus controllers.

The virtual CAVE should consist of three display screens. Each screen should be square with a width of 2.4 meters (roughly 8 feet). One of the screens should be on the floor, the other two should be vertical and seamlessly attached to the screen on the floor and to each other at right angles, just like in the picture. This application should be used standing, with the viewpoint set to your height so that it appears as if you're standing on the floor screen. The initial user position should be with their feet in the center of the floor screen, facing the edge between the two vertical screens. In this position, the user should see the calibration cube from project 2 in front of them, again 20cm wide and at a distance of about 30cm. The sky box should also be visible. You can use the stereo sky box from project 2, or any other sky box even if it's mono. In this project it is not necessary that the user can turn the boxes off individually.

The big difference between this project and project 2 is that you need to render the scene to the CAVE walls first, instead of directly to the Rift's display. You need to do this by rendering the scene six times to off-screen buffers. Six times because there are three displays, and you need to render each separately from each eye position. To render from the different eye positions of a head-tracked user, you are going to need to render with an asymmetrical projection matrix. This picture illustrates the skewed projection pyramids originating from the two eyes:

The pictures below illustrate what an off axis view looks like on the screen and from the user's point of view:

To see the effect of rendering from the point of view of another user, you need to be able to switch the viewpoint to one of the controllers. The switch should happen when the right controller's middle finger trigger is pulled, and the viewpoint should remain at the controller and update while the controller is moving, until the middle finger trigger is released. When the trigger is released, the viewpoint should switch back to the user's head. Note that because we're rendering in stereo, you'll need to create two camera positions at the controller, one offset to the left, one to the right of it by half an average human eye distance, which is 65 millimeters. When switching the viewpoint, this should only affect the viewpoint used for rendering on the CAVE displays. The HMD's viewpoint should always be tracked from its correct position for a view of the space the CAVE is in.

We also need a way to freeze the viewpoint the CAVE renders from, regardless of whether it's rendering from head or controller location. For this we'll use the 'B' button. When it is pressed, the viewpoint the CAVE renders from should freeze. When the 'B' button is pressed again, the view should unfreeze.

To make the calibration cube more useful, you need to be able to move it in all three dimensions, and make it bigger or smaller. Use both thumbsticks to allow for this. For instance, the left thumbstick could be used to move the cube left/right and forward/backward. The right thumbstick could move it up/down and the other direction makes it smaller/larger.

To help with debugging, you should enable debug mode with the 'A' button (activate debug mode while 'A' is depressed, disable upon release). In debug mode, the user sees not only the images on the virtual CAVE screens, but also all six viewing pyramids (technically they're sheared pyramids). These pyramids start in each of the eye positions and go to the corners of each of the screens. Three screens times two eyes is six pyramids. You can visualize the pyramids in wireframe mode with lines outlining them, or use solid surfaces. In either case, you should draw those pyramids that indicate the view from the left eye in green, those from the right eye should be red.

Extra Credit (10 points max.)

TBD