CalVR Plugins

Overview

In CalVR, new applications are added in the form of plugins. These plugin modules implement a virtual base class in the CalVR core and compile into a separate shared library. At runtime, this library is loaded and the plugin receives callbacks from the CalVR core through the implemented virtual functions.

Plugin Development

It is recommended to build the doxygen documentation for the CalVR core for use during plugin development. You can turn this on with a flag in the ccmake config. On the lab machines, the pre-built Doxygen documentation can be accessed by:

firefox /home/calvr/CalVR/doc/html/index.html

Adding a new Plugin for CalVR

Create a new directory named <YourPluginName>. Each source code directory must have a file "CMakeLists.txt".
For your plugin, use (CVRPlugins Dir)/calit2/ModelLoader/CMakeLists.txt as an example.
Change the LIB_NAME variable to be set to <YourPluginName> and replace the source and header files with those for your plugin.
If you need compile against other libraries, add entries for them with INCLUDE_DIRECTORIES and TARGET_LINK_LIBRARIES options.

In order for cmake to generate a makefile for the new plugin, we need to add an entries to the "CMakeLists.txt" in the parent directory.

Under the parent directory, e.g., (CVRPlugins Dir)/calit2/CmakeLists.txt, add the following lines:

OPTION(PLUGIN_YOURPLUGINNAME "YourPluginName" OFF)

IF(PLUGIN_YOURPLUGINNAME)
 ADD_SUBDIRECTORY(YourPluginName)
ENDIF(PLUGIN_YOURPLUGINNAME)

Go to the root CVRPlugins directory and run:

ccmake .

Type 'c' to configure the project again.
You should now see an option to build your new plugin. Set this option to ON, re-configure, then type 'g' to generate the new makefiles.
You can now build your plugin by typing make.

Now we need to modify the CalVR Config File to turn on the new plugin. Now run CalVR, in the terminal we should see the init message of the new plugin.

The Plugin Base Class

Your plugin project will need to have a class the inherits from the CalVR plugin base class. This is the CVRPlugin class defined in:

#include <kernel/CVRPlugin.h>

The class may implement any needed virtual functions in this class to receive calls from the CalVR core during runtime.
There are doxygen comments for these functions, but I will note them here as well:

virtual bool init()

Called once right after the plugin is created. A return of true means the init went ok and false means there was some failure and the plugin should be unloaded. This is a good place to do one time setup operations like creating menus or a base scene graph structure.

virtual void preFrame()

Called once each frame right before the scene is rendered. This is where things like scene graph changes could take place and affect the current frame.

virtual void postFrame()

Called once each frame after rendering and buffer swap (for double buffered rendering). This call may have a situational use. Implement it if you feel you need it.

virtual bool processEvent(InteractionEvent * event)

Function that allow the plugin to use events generated by the CalVR core. These events are things like button presses, keyboard presses, valuators, etc. See doxygen or below for more on the InterationEvent class types. If true is returned the event is consumed, otherwise it continues through the event pipeline.

virtual void message(int type, char * & data, bool collaborative = false)

Called when the plugin is sent a message from another plugin or a collaborative session. TODO: create and link to message passing documentation

virtual int getPriority()

Returns a priority value for this plugin. This function is called once when the plugin is loaded. It is used to order all loaded plugins. A plugin with a higher priority gets function calls and events before lower priority plugins. The default is 50. This value really SHOULD NOT BE CHANGED unless you have a good reason to do so. This is designed to allow you to be able to enforce a relative order of plugin execution or event handling.

All plugins must also include a macro in the main .cpp file:

CVRPLUGIN(PluginName)

Where PluginName is the name of the class implementing the CVRPlugin class. This creates a small function that is used as an interface into the plugin shared library.

Using the Config File

For info on the config file itself, see CalVR Config File.
To access values from the config file you must first include the ConfigManager class.

#include <config/ConfigManager.h>

This class provides static functions that return values listed in the config file. An example would be something like:

int myInt = ConfigManager::getInt("size","Plugin.MyPlugin.MyTag",100);

The ConfigManager looks for the given attribute "size" in the given tag search path "Plugin.MyPlugin.MyTag". If the tag/attribute is not found, the default value is returned, in this case 100. Optionally, you can include a bool pointer as a final argument in the function call. The bool value is filled with true if the tag/attribute is found and false otherwise. If this tag/attribute appears more than once in the config structure, the first one found is returned.
This is the xml structure the example is searching for:

<Plugin>
 <MyPlugin>
  <MyTag size="100" />
 </MyPlugin>
</Plugin>

There is a different function call for each type of value you wish to find, bool/int/float/etc. See the doxygen comments or header files for more info.

Menus

CalVR allows for use of menus to allow a user to dynamically interact with a plugin. You may create a structure of menu items and add them to one of the menu systems.
The two main types of menu items are sub menus and everything else.

SubMenu

The SubMenus create the structure of the menu. You may add any type of MenuItem into a SubMenu. You also add SubMenus to SubMenus.
To create a SubMenu, include:

#include <menu/SubMenu.h>

Other MenuItems

Into the SubMenus you can add various types of items. I will list some of the main ones here:

#include <menu/MenuButton.h>

A clickable button item.

#include <menu/MenuCheckbox.h>

A toggleable checkbox.

#include <menu/MenuRangeValue.h>  | <menu/MenuRangeValueCompact.h>

Allows user to change a value within a range.

#include <menu/MenuText.h>

Allows insertion of non clickable text.

Using Menu Interaction

A plugin receives menu actions in the form of callbacks. In order for this to work, a few things need to be done. First, the class that is getting the callback must implement the MenuCallback interface:

class MyClass : public MenuCallback
...

The class must also define the function:

virtual void menuCallback(MenuItem *)

Now when the MenuItems are created, this class must be registered as the callback for the item:

MenuButton* button = new MenuButton("MyButton");
button->setCallback(this); // only if this class implements the MenuCallback interface
// add button to menu

Now when the user interacts with the item, the callback fuction will be called. Within the callback function, you can look at the MenuItem * that is passed in and see if it equals one you have created. At this point, you know what the item is and you can properly handle the interaction.

Menu Systems

There are a few places you can add menu item to let the user interact with them.

Main Menu

The main menu in CalVR includes options for navigation as well as a few other general things. Your plugin is able to add to this menu if needed. Normally, you create a SubMenu that is named the same as the plugin and place all needed items inside of it. To add items to the main menu, use the PluginHelper class.

Popup Menu

You may also create your own personal menus and add whatever you want to them. These can take to form of PopupMenus:

#include <menu/PopupMenu.h>

You can title these menus and set their size/position. The user is able to drag the menu by clicking on the title bar.

Context Menu

See SceneObjects

PluginHelper

The PluginHelper class is designed to collect useful functions to make plugin development easier. The class is include with:

#include <kernel/PluginHelper.h>

All the fuctions in this class are static. Therefore, you do no need to create and instance of it.

Frequently Used Functions

 static osg::MatrixTransform * 	getScene ()

Get the root scene transform (world space).

 static const osg::MatrixTransform * 	getObjectTransform ()

Get node containing the current object space position/orientation.

 static const osg::Matrix & 	getObjectMatrix ()

Get matrix containing the current object space position/orientation.

 static const osg::Matrix & 	getWorldToObjectTransform ()

Gets the matrix transform from object space to world space.

 static const osg::Matrix & 	getObjectToWorldTransform ()

Gets the matrix transform from world space to object space.

 static osg::Matrix & 	getHandMat (int hand=0)

Get the hand to world space transform.

 static osg::Matrix & 	getHeadMat (int head=0)

Get the head to world space transform.

 static int 	getNumButtons (int system=0)

Get the number of buttons in a given tracking system.

 static unsigned int 	getRawButtonMask (int system=0)

Get the mask repesenting the current button state.

 static unsigned int 	getHandButtonMask (int hand=0)

Get the mask repesenting the current button state for each hand.

 static void 	addRootMenuItem (MenuItem *item)

Add an item to the main menu.

 static double 	getLastFrameDuration ()

Get the time taken for the last frame, in seconds.

 static void 	registerSceneObject (SceneObject *object, std::string plugin="")

Register a SceneObject with the SceneManager.

 static void 	unregisterSceneObject (SceneObject *object)

Unregister a SceneObject with the SceneManager.

 static void 	sendMessageByName (std::string plugin, int type, char *data)

Sends a message to a plugin with the given name through the message() callback in the plugin interface.

Plugin-specific Data

Any data other than source code that is specific to a plugin, such as 3D widget files, sound effect files, etc. should not be put in the plugin's source code directory, but under data/plugins/<PLUGIN_NAME>. It is very important that such files are not committed to the GIT repository, so that the repository remains small enough to download fast.

Standard Plugins

CollaborativePlugin

This plugin allows connecting two CalVR sessions over the network. The plugin's configuration file entry looks like this:

  <CollaborativePlugin value="on">
    <Server value="starcave.ucsd.edu" />
    <Port value="11050" />
  </CollaborativePlugin>

Make sure that the default port of 11050 is available on all participating machines, and that it is not blocked by the firewall.

For the communication to work, the standalone command-line tool CollabServer (to be found at CalVR/bin) needs to be run on one of the participating computers, or on another machine on the network. Upon startup CollabServer should display the message "Socket setup ok". When CalVR instances connect, it displays "Found connection" and shows which machine connected and what type of events are being received.

CalVR instances connect once the user opens the CollaborativePlugin's main menu and clicks "enable". Then the user can see the other connected users' head and wand positions.

OSGPDF - PDF Viewer

Plugin OsgPdf will allow you to run "CalVR filename.pdf" and will load the PDF to the screen. It requires ReaderWriterPDF, which is an option under CVRPlugins called PLUGIN_OSGPDF. Set its flag to ON using ccmake. Once built enable the plugin module in the configuration file:

<Plugin>
  <OsgPdf value="on" />
</Plugin>

PanoViewLOD

This is a viewer for stereo panorama images.

To convert left/right TIFF images to PanoView format:

• Bob Kooima's conversion tool is at: /home/calvr/CVRPlugins/calit2/PanoViewLOD/panoproc/panoproc
• Documentation for panoproc: panorama.html
• Standard usage syntax: panoproc -o outputfilename.tif -d 3 inputfilename-left.tif
• Conversion can take several minutes.
• Above syntax works well for 24k x 12k pixel images, for bigger images increase -d parameter.

To register an image pair to load in CalVR:

• Open CalVR configuration file (eg, config.xml) in editor.
• Search for PanoViewLOD section.
• Add entry to list of files, for instance NewImage.
• Create file NewImage.xml in PanoView config directory.
• Copy existing configuration file content into it.
• Open NewImage.xml in editor.
  • Make sure entry for depth matches that used in the conversion step above.
  • Edit file name and path for images.

Run CalVR, make sure image loads correctly.

WaterMaze

To run the WaterMaze demo in the NexCAVE do this:

• Log in as user 'neuro'
• Run CalVR
• Don't use the WaterMaze menu, but keyboard shortcuts instead
• Press the L key to load a paradigm
• Press the S key to start the paradigm
• Wait until subject finds the target tile, the trial times out, or press the S key to stop
• Wait for grey screen to come up
• Press the N key to load the next paradigm
• Press the S key to start the paradigm
• To navigate use the joystick
• If the joystick doesn't work fix permissions of USB port (needs to be done upon each reboot)

PointsOOC

This plugin allows out of core rendering of large point clouds. It requires Points2Archive, which will convert large point files to .osga paged loaders, which the plugin requires.

Run "Points2Archive -h", it will explain the required parameters.

Plugin Overview

• AcoustiMaze: ??? DELETE
• AlgebraInMotion: by Dan Sandin, under construction
• AndroidNavigator: by Brooklyn? allows using an Android device as a VR controller for navigation, written for the Xoom tablet
• AppSwitcher: by Neil Smith???
• ArtifactVis: by Kyle Knabb, Jurgen, ported by Andrew ; dig site at Khirbat-an-Nahas
• ArtifactVis2: by Neil Smith, uses Neil's data base, point cloud viz, loci viz; dig site in JordanBi
• BinauralReality: unknown author (Eric Hamdan?)
• CameraFlight: good concept, Google Earth style navigation, never worked, DELETE
• CaveCADBeta: COVISE plugin ported by Cathy, original by Lelin; works, uses sphere menu
• CrashDemo: by Andrew, car crash test simulation viz with exploding car, works
• CudaTest: test plugin to test memory copy to Cuda by Andrew
• CullingMultiGPUTRender: uses occlusion culling and multiple GPUs to render car crash tests rapidly
• DumpTracking: debug plugin to dump tracking information to the screen, works well, by Andrew
• ElevatorRoom: by Cathy, then Scott, then Philip. Eduardo's Alzheimer's project
• EOGCalibration: by Cathy, then Scott: stereo eye tracking with EOG electrodes
• EZViz: by Philip, easy graphics programming language, based on MUGIC
• FlowVis: by Andrew and Philip, CFD viz for converted VTK data
• FuturePatient: by Andrew, written for 2D tiled wall, biomarker and microbiome viz and analysis tool
• GML: by Philip, visualizes GML graphs in 3D
• GreenLight: by John and Alfred, NSF GreenLight project, no longer have data base or live streaming
• Hpwren: by Philip, osgEarth with live weather data and 2m elevation
• ImageSequence: plays image sequences like a video, 2D images only
• ImageViewer: displays 2D images, user can move images around in 3D space
• Interactors: ???
• KinectDemo: displays Kinect point clouds in 3D, needs update on interface with Kinect interface to newest version
• KinectHelper: demonstrates Kinect integration into CalVR
• LayoutManager: by John, component for something else, not standalone
• LightEditor: by John and David, provides controls to place light sources in 3D and manipulate their parameters
• LocationTracker: by Philip, DELETE
• Maze2: original by Lelin, ported by Cathy, part of Eduardo's project, generates random mazes for navigation studies with patients
• ModelLoader: by Philip, versatile tool to load 3D models in, supports all OSG 3D model files, requires files to be listed in CalVR config file, can also load models in by command line argument
• MUGIC: by Philip, developed out of project with Shahrokh Yadegari, simple graphics language for cluster graphics displays, multi-user support and regular expressions
• MultiGPURenderer: by Andrew, allows faster rendering of scenes, breaks up scenes
• MultiHandInteraction: by Andrew, demonstrates the use of two hand held controllers, provides walk, fly, scale functions for two hands
• MultiViewSimulator: by Andrew, code for paper on Democratizing Head Position, allows two viewers to each have equally good/bad views and distortions
• MVSim: same as MutliViewSimulator; need to merge code and delete one of them
• OsgChromosome: honors thesis by Yixin, displays chromosomes in 3D at two levels of detail, supervised by Dr. Ren
• OsgEarth: by Philip, loads osgEarth, has special navigation mode for earth scale motion, uses DepthPartition feature of CalVR to correctly display earth and menus with z-buffer
• OsgMovie: by Philip, plays back a video, supports stereo videos, allows rendering video on screen in auditorium when ModelLoader loads a model; can save position and orientation of film clip
• OsgPathRecorder: by Philip and Andrew, records a camera path to a file and allows playback, interpolates between control points at playback time, was used for Exodus demo, supports OSG path data type
• OsgPdf: PDF viewer, by Philip, used by FuturePatient plugin
• OsgVnc: VNC viewer, by Philip, used by FuturePatient plugin
• OssimPlanet: by Philip, uses OssimPlanet library for planet rendering, defunct needs major updating to latest OssimPlanet version
• PanoView360: original pano viewer without LOD support, slow loading times, by Andrew, loads pairs of non-pyramidal TIFF or JPEG images
• 'PanoViewLOD: by Andrew and Bob Kooima, loads panos fast with LOD support, requires pyramidal TIFFs created with Bob Kooima's conversion tool panoproc
• ParticleDreams: interactive art piece by Dan Sandin, original written for Thumb, uses CUDA for particle simulation, allows interaction with particles, consists of several sections some of which have water effects
• PathRecorder: by Lelin, records camera paths by recording camera matrix; use OsgPathRecorder instead, DELETE
• PlanarLayout: related to LayoutManager, by John
• PluginTest: demonstrates how to build a plugin, menus, data, various CalVR widgets
• Points: by Philip, basic point rendering, supports XYZ, PLY and XYB files (XYB is custom binary version of XYZ)
• Points2: DELETE
• PointsOOC: by Philip, out of core point renderer with GLSL shaded points, BSP tree for optimization, view frustum culling
• PointsWithPans: uses Points plugin for point rendering and PanoViewLOD for panos, by Andrew
• SampleEarth: by Philip, demo plugin to show how to interface with the osgEarth plugin, uses osgEarth plugin for terrain rendering
• Site: by Philip, project with Albert Lin, visualization of Mongolian data, fuses points, terrain, images
• Sketch: sketch in 3D, written by Cathy, supports lines, ribbons, points
• SMV2Settings: multi-viewer support
• SoundPlayer: Supercollider sound support, by Eric Hamdan
• SoundTest: test plugin for SoundPlayer plugin, by Eric Hamdan
• SpaceNavigator: supports 3dConnextion Space Navigator for CalVR navigation, by Andrew
• StructView: original plugin was written for COVISE by Structural Engineering student Andre Barbosa, data from CalTrans, provided as Bentley data, converted to 3ds Max, then VRML
• TouchDesigner: by Andy Muehlhausen, may never have been completed?
• TourCave: by Andrew for KAUST, cell phone controlled canned demos
• TrackServer: ???
• TwitterEarth: by Philip, twitter feed viz on osgEarth
• Video: superceded by OsgMovie, DELETE
• VirvoVolume: volume renderer for CalVR, uses Virvo library to display volume
• Volume: by Philip, volume renderer with segmentation support, requires ITK and VTK
• WaterMaze: by Cathy, Scott, Philip; models the Morris Water Maze for VR for Alzheimer's patient studies, PI Macagno