Revision as of 11:48, 19 April 2020

Levels of Immersion

THIS PROJECT IS UNDER CONSTRUCTION. DO NOT START YET.

In this project we are going to explore different levels of immersion with your headset. We are using Unity with C# for this project again like in project 1.

We recommend starting with your code from project 1 and add the relevant sections for the cubes and the sky box.

Milestones

• Week 1: sky box and control buttons
• Week 2:
• Week 3:

Project Description (100 Points)

You need to add the following features to your project:

1. 3D Sky box (15 Points) The sky box in Unity is monoscopic by default, which makes it look flat like a poster on a wall. This is fine for far away objects such as mountains and trees but doesn't look right for a closer environment. Create a seteroscopic sky box using these Skybox textures) and the custom shader provided here.
2. Custom Object (5 Points) Create or add a custom asset that will be used in your project. You can simply create a cube and apply a texture or create your own. You will need to use a custom shader for this project though, so you would need to convert the materials to work with this shader. You can place this wherever you wish in the scene.
3. Gaze Interaction Menu (10 Points) Create a menu that you can interact with using the gaze system. This will be the controls for the following options. Here is a texture you can apply on a plane if you don't feel like creating your own or adding text. You can place this wherever you wish in the scene.
4. Stereo Modes (15 Points) Implement the functionality to change the rendering mode from Stereo to Mono, Left Eye Only, and Right Eye Only. The provided class here and material shader here will be required to implement this. The scene should render in only one of these modes at a given time and should support other functionality while rendering in any mode.
5. Rendering Scale (15 Points) Implement the functionality to change the rendering scale of your custom object. Support at least half size (0.5x scale) to double size (2.0x scale) as well as a reset button (1.0x scale). Make sure the object doesn't move (position and rotation) and only scales in size.
6. Head Tracking (10 Points) Implement the functionality to disable and enable head tracking (position and rotation).
7. Variable IOD (15 Points) Implement the functionality to change the default interocular distance (IOD). Each eye has an offset from the central position, support at least the range of -1.0x to 2.0x distance with a reset button. This is not a supported operation by default in Unity so it's okay if things don't look exactly right or break some other aspects (such as the reticle or gaze system).
8. Rendering Lag (15 Points) Implement the functionality to simulate rendering lag. This is what it would feel like if a frame look longer than the allotted time to render and same image was used for more than one frame. Support 1/2 FPS (30FPS usually), 1/4 FPS (15FPS usually), and 1/10 FPS (6FPS Usually) as well as a reset button.

Tips for Rendering the Skybox

There are six cube map images in JPG format in the ZIP file. Each is 2k x 2k pixels in size. The files are named nx, ny and nz for the negative x, y and z axis images. The positive axis files are named px, py and pz. Here is a downsized picture of the panorama image:

And this is how the cube map faces are labeled:

The panorama was shot with camera lenses parallel to one another, so the resulting cube maps will need to be separated by a human eye distance when rendered, i.e., their physical placement needs to be horizontally offset from each other for each eye.

Notes:

• Here is a great tutorial on 3D skyboxes in Unity.
• Here are higher resolution skybox textures.

Extra Credit (up to 10 points)

There are three options for extra credit, for a total of 10 points maximum.

1. Viewmaster Simulator: Take two regular, non-panoramic photos from an eye distance apart (about 65mm) with a regular camera such as the one in your cell phone. Use the widest angle your camera can be set to, as close to a 90 degree field of view as you can get. Cut the edges off to make the images square and exactly the same size. Use your custom images as the textures on a rectangle for each eye. You may have to shift one of the images left or right see a correct stereo image that doesn't hurt your eyes. (5 points)
2. Custom Sky Box: Create your own (monoscopic) sky box: use your cell phone's panorama function to capture a 360 degree panorama picture (or use Google's StreetView app, which is free for Android and iPhone). Process it into cube maps - this on-line conversion tool can do this for you. Texture the sky box with the resulting textures. Note you'll have to download each of the six textures separately. Make it an alternate option to the Bear image when the 'X' button is pressed. (5 points)
3. Super-Rotation: Modify the regular orientation tracking so that it exaggerates horizontal head rotations by a factor of two. This means that starting when the user's head faces straight forward, any rotation to left or right (=heading) is multiplied by two and this new head orientation is used to render the image. Do not modify pitch or roll. In this mode the user will be able to look behind them by just rotating their head by 90 degrees to either side. Get this mode to work with your skybox and calibration cubes, tracking fully on, and correct stereo rendering. This publication gives more information about this technique. (5 points)
4. Inverted Stereo Rendering Extend your stereo modes to include an inverted stereo mode, this means that the image that is usually rendered to the left eye will now be displayed on the right eye and vice versa. (5 points)