Roadmap

Currently working on bold

---

Current Focus

Visual Cone Detection

• setup AirSim
• add cone position information
• dataset generation pipeline
• augmentation
  • noise
  • low light
  • weather
• generate dataset
• train YOLOv3-tiny
  • see here
• test inference
• test inference performance on Xavier
• annotate real data
• retrain the network
• test in real-world
• improvements:
  • add sub-pixel refinement
  • add tracking

Visual SLAM Exploration

• test orbslam 2
  • discarded as not robust enough
• test orbslam_dwo
  • discarded since no RT branch available, also slower than original orbslam
• test rovio
  • works OK on servo data
• test servo on provided data
  • bugfixes - doesn't compile
  • compile
  • test ROS node on provided data - works fine
• conclusion: continue working with servo

Visual SLAM System

• integrate servo with the main system
• calibrate our cameras and IMU
• test on our data
• performance optimization
  • integrate with orbslam2-gpu
  • add binary vocabulary
  • remove Pangolin GUI
• publish points and poses to ROS
• test cone detection
• implement cone detector and descriptor
• implement data fusion in orbslam
• test on the car

---

Additional Directions

System

• define ROS node topology
• add documentation

Simulator

• eufs_sim - gazebo simulator
  • compiled
  • bugfix - no car
• AirSim - Microsoft's framework
  • test on linux
  • add lidar, see here
  • add IMU, see here
  • integrate with ROS, see here

Hardware

• camera
• LIDAR
• Nvidia Jetson setup
• capture with rosbag
• install and run on the car
• capture demo content
• capture real content
• Nvidia Drive PX2 setup
• after meeting with Nvidia decided to use Jetson AGX Xavier
• improve capture
  • debug ZED low fps
  • find good IMU
• automate capture - launch files
• capture new content
• Nvidia Jetson Xavier setup
• install and run on the car
• TODO: update regarding the electric car
  • sensors
  • control

LIDAR Setup

• install ROS velodyne drivers
• test and store data
• control LIDAR speed from ROS, see here
• filter LIDAR by FOV and distance, search for ring information, see here
• LIDAR-camera calibration
  • lidar_camera_calibration doesn't compile, todo: debug
  • try this
• fusion with camera

LIDAR Cone Detection

• test the data
• add node to ROS
• ground removal
• cone segmentation and clustering
• cone tracking
  • probably redundant
• additional filtering
• color from intensity
• color from camera

LIDAR SLAM Exploration

• test off the shelf LIDAR SLAM solutions
  • loam_velodyne - not good enough for tracking cones
  • hdl_graph_slam - doesn't compile
  • LeGO-LOAM - worse than loam_velodyne
  • cartographer - compiled, looks promising, but integration is difficult
• next will unlikely work
  • mrpt
  • gmapping
  • hector_mapping - similar to gmapping
• conclusion: general-purpose LIDAR SLAM is not good enough

LIDAR Landmark Based SLAM

• explore directions
  • Kalman Filter vs Graph Optimization
  • g2o vs gtsam

Model Predictive Control

• explore different options:
  • MPCC Model Predictive Contouring Controller (MPCC) for Autonomous Racing
  • MPPI Model Predictive Path Integral Controller (MPPI)
• start working on simulator