VRDPI
From Immersive Visualization Lab Wiki
Contents |
Project Overview
The VRDPI is intended to be a tool for deep packet capture and analysis in Virtual Reality. Using standard virtual reality controls (Oculus Touch, Vive Motion Controllers) users will be able to select a network interface to capture and a duration of the capture. Packet capture will be completed in the background (seperate thread) and the results will be displayed inside VR.
Project Goals
- Render deep packet inspection results in a virtual reality environment. Utilizing the WinPCap library and Unreal Engine 4 with 99% C++ UE4 API.
- Renders each TCP/UDP session in the form of a packet column. The packet columns representing each session’s packet stream are arrayed in front of the user after capture in the form of a semi-arc.
- Users will utilize standard VR motion controller interface patterns to navigate the VR environment. Including VR teleportation, direct controller to object manipulations, and controller to remote object at a distance manipulations.
- User can also resize a packet to make the data displayed on its front face larger and more clear.
- Information inside the packet is divided into layers which represent each layer of the network stack. Frame layer, Ethernet layer, IP layer, TCP layer, and finally the application layer.
- Each layer will be color coded. Each layer will display its data in hexadecimal format where appropriate.
Developer
- Marlon West
Project Adviser
- Jurgen Schulze
Technologies
- Unreal Engine v4.15.1
- WinPCap Library v4.1.2
- Oculus Rift
- HTC Vive
TODO
-
Add PacketBox decals for each Packetbox, indicating packet #, timestamp and if it is an ACK. -
Setup PacketBox information display widget. Widget will have multiple collapsible fields that are inside a scrolling area. -
Setup packet resizing, scaling the height and width of the box while maintaining the same depth. -
Setup conversation recognition, divide packet column into session columns. -
Implement decoding of IPv6, UDP, and Application header types. -
Implement lookup table for socket types, protocol types and DNS lookup for IP addresses. -
Break columns down by session instead of conversation. -
Create session info decals at the head of each column with IP address of both parties, the tcp port numbers or application protocol in use. - Attempt a real-time packet capture animation
- Add a statistical output board close by the podium.
- Add a dim to black/grey feature when loading a new packet dump
Project Development Blog
March
3/23/17
- Created VRDPI Unreal Engine 4.15.1 Project using the C++ template.
- Imported relevant class files from DemoA project.
- Created large (5km x 5km) floor.
- Created "Target" material in Photoshop (2000x2000) for origin floor indicator. Imported into UE4 as texture.
- Created blender model for podium. Imported into UE4 as static mesh.
3/24/17
- Created PacketHelper class and linked the WinPCap static library into the project.
- Extended PacketHelper as a child of FRunnable UE4 class in order to make the packet capture multithreaded.
- Successfully tested packet capture in VR.
3/25/17
- Researched how to properly integrate C style callbacks with UE4.
- Reconfigured PacketHelper class to push data back to PlayerController object.
3/26/17
- Created PacketHolder class to contain individual packets.
- Added static methods for retrieving adapter configuration to PacketHelper.
- Created Podium class and podium widget dialog to let the user setup and initiate the packet capture.
3/27/17
- Debugged and improved the Podium widget
- Created AdapterButton class to contain the buttons which display the adapter list on the podium.
3/28/17
- Finished widget functionality implementation. Capture packet button will be disabled during a packet captured and re-enabled after.
- Created PacketBox actor class which will hold the virtual representation of a packet in VR.
- Created Blender models for the PacketBox and the handle. Imported into UE4 as static meshes.
3/29/17
- Fleshed out PacketBox class adding the materials and meshes. Added event handling.
- Created a PacketBox generation method in Podium, aligns the packets linearly, pointing in the direction specified.
3/30/17
- Added local and remote movement capabilities to PacketBox.
3/31/17
- Created Decal images for Cargo Container font. Imported images into UE4
- Created Decal materials for all the Cargo Container font images.
- Created DecalHelper class to store the Decal material instances, and use them to return new decal instances upon demand.
- Added CreateDecalsFromString to Podium class to convert String->Decal actors.
April
4/1/17
- Created algorithm to arrange decals at regular intervals in a Packetbox chain, given a direction vector and origin.
4/2/17
- Debugged process for taking multiple captures, one after the other. VRDPI now can capture more than once and arranges the captures in a line, one after the other.
4/3/17
- Researched FStreamableManager to facilitate asynchronis loading of assets.
- Created Loading widget.
- Adapted Podium class to use pool of pre-allocated actor objects.
- Tested 10,000 packets, but allocated 16gb+ and crashed. With 1,000 packets the operation took ~2 seconds.
- Opted for 1,000 packet soft limit, which the user can add to with additional captures.
- Created packet capture indicator animation on the Podium widget.
4/4/17
- Created algorithm for arranging decals on the side of a packet box.
4/5/17
- Tested and debugged algorithm for placing packet box decals on the PacketBoxes. Added variable scaling to packet box labels.
- Filled in PacketBox widget, creating buttons and collapsible text areas.
- Added alphanumeric characters to decal alphabet.
- Aligned PacketBox widget with front of packet.
4/8/17
- Researched how to dynamically resize text areas in a widget. Found the Size widget component which allows you to specify the size.
- Setup the PacketBox widget to display five deep packet analysis layers. The layers are as follows: Frame Layer, Ethernet Layer, IP Layer, TCP Layer and Application Layer.
- Created a method for calculating the lines used by a given frame analysis.
- Created 3 typedefs for storing packet header data, for ethernet, ip, udp and tcp layers.
- Created generateEthernetFrameString(), generateIPFrameString() and generateTCPFrameString(). These methods copy raw values from the packet buffer to their respective typedefs, as well as building a string output of their respective packet headers.
4/9/17
- Performed first deep packet analysis in VR by inspecting captured packets with the PacketBox widget.
4/10/17
- Created first video demoing simple workflow for inspecting the contents of a packet in VRDPI.
-
VIDEO LINK: https://youtu.be/3YF8zjWYFG8 - Fixed timestamps and added Frame layer packet data to the PacketBox info widget.
- Fixed bug with remote manipulation of Packetboxes with the left motion controller.
- Added Sticky mode for remote movement of Packetboxes. You can now set a packetbox to stay in remote movement mode after you release the trigger. Then you can use the other motion controller in teleport mode and you effectively have a way to teleport into the sky to get a birds eye view of the packet data.
4/11/17
- Resolved issues preventing packaging of the project for use on other computers.
- Created longer (4:27) Demo video which shows off sticky grabbing as well as packet manipulation, deep packet inspection and teleportation flight.
- VIDEO LINK: https://www.youtube.com/watch?v=3YF8zjWYFG8
4/18/17
- Created algorithm for performing conversation analysis on the IP level.
- Created algorithm for packet column layout, packets now layout in columns by conversation. A conversation constitutes all traffic between two IP addresses, in both directions. The layout creates a 300 unit circle and arrays each of the lines radiating out from the center of this circle.
- Added sound FX for teleportation, sticky activation and music during the packet capture.
4/20/17
- Refined algorithm for packet column layout, made it so that multiple packet captures are laid out contiguously, each conversation being concatenated with the newly captured packets.
- Created column moving algorithm to re-organize existing columns of packets to allow for more packet columns on a wider ring.
4/26/17
- Added floor-level decal repositioining into the column moving algorithm. All decals now reposition properly as successive packets are added to each chain.
- Based on end user feedback removed controller mode locking when creating a new spawnbox.
- Added packet box scaling based on packet size. Packets will scale up by a factor of 1x if they are the largest permissible packet size (1514bytes).
4/27/17
- Changed default control mode of left controller from create to teleport to improve usability.
- Added headers defines in Packet Holder for IPv6, ICMP, IPv6 IP addresses, and ICMP Echo.
- Added support for printing out header types for IPv6, ICMP and UDP protocols.
May
5/3/17
- Added application layer printing. Created print function for entire hex-dump of the packet, as well as the ascii printout of the packet.
- Debugged IPv6 layer printing. IPv6 adresses now properly stored.
- Color coded IPv6 packets purple.
5/4/17
- Changed the control podium, removed the podium model, made the podium widget large and double sided to improve usability. Made podium widget background transparent.
- Researched UE4 library for solution to hostname resolution feature. Determined that UE4 has no rdns lookup capability. Researched using winsock to perform the name resolutions.
5/10/17
- Created new mesh and texture for podium backing. Set the texture to be translucent and have a blue emissive color.
- Modified remote position manipulation to subtract the yaw of the controller from the yaw of the packetbox rotation, now packetboxes more easily line up with the viewers perspective when doing distance manipulations.
- Created DNSHelper class to perform dns lookups and caching.
- Wrote method for basic rdns lookup and tested successfully.
5/11/17
- Updated DNSHelper with a DNS cache to cache already queried addresses.
- Added hostname precaching
- Attempted to add Portmapping for UDP and TCP via data structures through sourcecode, but took to long to compile.
- Converted both datasets to CSV and converted to JSON.
- Researched api commands for importing JSON data structures via UE4.
- Created PortHelper class to facilitate loading of the json files and querying the data structure created by them.
5/14/17
- Resolved bug with portmapping system causing crashes.
5/18/17
- Started refactoring session parsing and packetbox placement algorithm. Implementing session-by-session breakdown of packets into chains, increasing the complexity of the data structure needed to contain them by one dimension.
- Rewrote PerformSessionAnalysis() and RepositionPacketCollumns()
5/19/17
- Re-evaluated the requirements of the algorithm and opted to create a new custom data structure which included the new class SessionContainer. A SessionContainer includes additional flags for whether each individual packet in any given column has already been placed.
5/20/17
- Refactored PerformSessionAnalysis(), RepositionPacketCollumns() and created CreatePacketRing() to work with the new SessionContainer data structure.
5/21/17
- Continued the refactoring by adapting CreateLinearPacketChain() to work with the newly created session storing data structures.
- Finished refactoring and began debugging the new session handling methods.
5/22/17
- Added new character textures and materials for additional decal letters.
- Refactored CreateDecalsFromString() to include scale and letter spacing arguments.
- Refactored CreateLinearPacketChain() to create a decal set for the head of the session column.
- Meta data decal set now prints the IP and Hostnames as well as the port number and port name of both the destination and source addresses of the packet column.
5/23/17
- Debugged meta data decal, added automatic line chopping for long hostnames.
5/31/17
- Debugged invalid packet box issue, PerformSessionAnalysis() method was generating some invalid PacketBoxes for packets which corresponded to packets which were not Ethernet packets. These packets will be ignored for now.
June
6/1/17
- Added scaling to the meta data decal generation. Meta data decals are now scaled up when there are enough packet chains to allow for a larger area for each meta data decal.
- Performed performance profiling to diagnose low frame rates with larger packet captures. Determined that a bug with DX11 implementation in UE4 and running the application in the PIE environment might be contributing to a slowdown of the Mip Mapping algorithm in UE4. This is a known bug that is not going to be fixed until UE4 migrates to a DX12 render engine.
- Tested performance by removing complicated material from the packet box, removing the large floor and removing widget generation on the PacketBoxes. None of these steps substantially effected the frame rates.
6/6/17
- Created final demo video for Quarter Project, which can be found here: https://www.youtube.com/watch?v=a0NXoQDPWy4
