Skip to content

Commit

Permalink
cleaned up factor adders package and readmes
Browse files Browse the repository at this point in the history
  • Loading branch information
rsoussan committed Jun 7, 2024
1 parent c62fcaf commit 42d72a6
Show file tree
Hide file tree
Showing 8 changed files with 33 additions and 131 deletions.
27 changes: 4 additions & 23 deletions localization/factor_adders/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@ find_package(Eigen3 REQUIRED)

# include ff_nodelet to get ff_util header files since these aren't exposed elsewhere
catkin_package(
LIBRARIES ${PROJECT_NAME} ${GLOG_LIBRARIES}
INCLUDE_DIRS include ${GLOG_INCLUDE_DIRS}
CATKIN_DEPENDS graph_factors localization_common localization_measurements node_adders vision_common
)
Expand All @@ -55,17 +54,6 @@ include_directories(
${GLOG_INCLUDE_DIRS}
)

file(GLOB cc_files
"src/*.cc"
)

# Declare C++ libraries
add_library(${PROJECT_NAME}
${cc_files}
)
add_dependencies(${PROJECT_NAME} ${catkin_EXPORTED_TARGETS})
target_link_libraries(${PROJECT_NAME} ${catkin_LIBRARIES})

if(CATKIN_ENABLE_TESTING)
find_package(rostest REQUIRED)
# Loc factor adder test
Expand All @@ -74,44 +62,37 @@ if(CATKIN_ENABLE_TESTING)
test/test_loc_factor_adder.cc
)
target_link_libraries(test_loc_factor_adder
factor_adders ${catkin_LIBRARIES}
${catkin_LIBRARIES}
)
# Single measurement based factor adder test
add_rostest_gtest(test_single_measurement_based_factor_adder
test/test_single_measurement_based_factor_adder.test
test/test_single_measurement_based_factor_adder.cc
)
target_link_libraries(test_single_measurement_based_factor_adder
factor_adders ${catkin_LIBRARIES}
${catkin_LIBRARIES}
)
# Standstill factor adder test
add_rostest_gtest(test_standstill_factor_adder
test/test_standstill_factor_adder.test
test/test_standstill_factor_adder.cc
)
target_link_libraries(test_standstill_factor_adder
factor_adders ${catkin_LIBRARIES}
${catkin_LIBRARIES}
)
add_rostest_gtest(test_vo_smart_projection_factor_adder
test/test_vo_smart_projection_factor_adder.test
test/test_vo_smart_projection_factor_adder.cc
)
target_link_libraries(test_vo_smart_projection_factor_adder
factor_adders ${catkin_LIBRARIES}
${catkin_LIBRARIES}
)
endif()

#############
## Install ##
#############

# Mark libraries for installation
install(TARGETS ${PROJECT_NAME}
ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
RUNTIME DESTINATION ${CATKIN_GLOBAL_BIN_DESTINATION}
)

# Mark cpp header files for installation
install(DIRECTORY include/${PROJECT_NAME}/
DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
Expand Down
19 changes: 0 additions & 19 deletions localization/factor_adders/src/test.cc

This file was deleted.

4 changes: 3 additions & 1 deletion localization/graph_vio/readme.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
\page graphvio Graph VIO

# Package Overview
Performs sliding-window graph based visual-intertial odometry (VIO) using a VOSmartProjectionFactorAdder and StandstillFactorAdder. Uses a CombinedNavStateNodeAdder for creating combined nav state nodes (containing a pose, velocity, and IMU bias) at required timestamps using IMU measurements.
Performs sliding-window graph based visual-intertial odometry (VIO) using a VOSmartProjectionFactorAdder and StandstillFactorAdder. Uses a CombinedNavStateNodeAdder for creating combined nav state nodes (containing a pose, velocity, and IMU bias) at required timestamps using IMU measurements. Optionally adds depth image correspondences as point-to-point between factors.

# Background
For more information on the theory behind the GraphVIO and the factors used, please see our paper:
Expand All @@ -13,9 +13,11 @@ For more information on the theory behind the GraphVIO and the factors used, ple
</p>

## Factor Adders
* DepthOdometryFactorAdder
* VoSmartProjectionFactorAdder
* StandstillFactorAdder
## Graph Factors
* Point BetweenFactors (for depth odometry correspondences)
* CombinedIMUFactor
* RobustSmartProjectionFactor
* Standstill Factors (zero velocity prior and identity relative transform between factors)
Expand Down
59 changes: 2 additions & 57 deletions localization/parameter_reader/readme.md
Original file line number Diff line number Diff line change
@@ -1,59 +1,4 @@
\page graphoptimizer Graph Optimizer
\page parameterreader Parameter Reader

# Package Overview
The GraphOptimizer class provides a sliding window optimizer using the GTSAM library that slides in time. Several objects are utilitized to track state parameters (nodes) and error functions (factors) as described below: NodeUpdater, FactorAdder, GraphValues, and GraphActionCompleter. The GraphOptimizer contains a set of NodeUpdaters which handle node insertion and deletion for each state parameter type. Each of these NodeUpdaters contains a GraphValues object for storing a type of state parameters.

<p align="center">
<img src="./doc/images/graph_optimizer.png" width="330">
</p>

FactorAdders push factors to the GraphOptimzer where they are buffered until they are able to be added to the graph.

<p align="center">
<img src="./doc/images/measurement_adding.png" width="520">
</p>

If the factors are too old they are discarded. Otherwise, if the factors are more recent than the state parameter nodes in the graph they depend on, the factors remain buffered until this is no longer the case. This can occur, for example, if a factor is created using a sensor measurement that arrives at a higher frequency than the sensor data required to link subsequent state parameter nodes. For more information on linking state parameter nodes, see the NodeUpdater section below.<br/>
The GraphOptimizer uses the latest and oldest timestamps of nodes stored in its NodeUpdaters to maintiain a sliding window of the correct duration.



# Background
The GraphOptimizer uses the GTSAM library (_Dellaert, Frank. Factor graphs and GTSAM: A hands-on introduction. Georgia Institute of Technology, 2012._) to optimize a nonlinear weighted least-squares problem where the weights are the measurement uncertainties and the least-squares errors are measurement errors (factors).

# Important Classes
## GraphOptimizer
The GraphOptimizer maintains sets of NodeUpdaters and GraphActionCompleters. It adds factors provided by FactorAdders and CumulativeFactorAdders and handles optimization of the graph. When factors are added to the graph, the NodeUpdaters responsible for creating new nodes are called and the factors are updated with Keys for the newly created nodes. The GraphOptimizer also maintains the required sliding window duration by calling each of its NodeUpdaters with the new desired window size. Marginal factors can optionally be added for factors removed during a sliding window operation. Additional functions are provided that allow for functions to be called after the window is slid and after each Update() call which includes the nonlinear optimization of the graph. The Update() call is illustrated below:

<p align="center">
<img src="./doc/images/update.png" width="200">
</p>

## FactorAdder
FactorAdders are responsible for outputting factors given a certain measurement type. The output factors from a FactorAdder should be added to the GraphOptimizer using its BufferFactors() member function.

## CumulativeFactorAdder
CumulativeFactorAdders provide an alternative interface for adding factors to a GraphOptimizer than FactorAdders. Whereas FactorAdders add factors given a single measurement, CumulativeFactorAdders add factors for a set of measurements. CumulativeFactorAdder creation is triggered at the beginning of each GraphOptimizer Update() call so that factors are created once the maximum number of measurements have been accumulated. An example of a use for CumulativeFactorAdders is for SmartFactors (_Carlone, Luca, et al. "Eliminating conditionally independent sets in factor graphs: A unifying perspective based on smart factors." 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2014._). These create a set of factors using a set of measurements depending on a state parameter and subsequently marginalize out the state parameter from this factor set, creating a single resulting factor containing the information of the factor set in the process. Since this can only occur once all the measurements are provided, the normal FactorAdder interface that creates factors given a single measurement would not suffice for this type of factor.

## GraphValues
GraphValues offer an interface on top of the gtsam::Values class that adds several helper functions for accessing values. Additionally, OldestTimestamp() and LatestTimestamp() functions are provided so the values can be used for sliding window optimization.

## NodeUpdater
NodeUpdaters are responsible for adding and removing state parameters when new factors are added or the graph optimizer window slides. For time varying state parameters, new nodes should be created when a new factor is added whose timestamp is not contained in the current set of state parameter nodes that it depends on. Typically this occurs when a factor is more recent than previously added factors or when an out of order measurement arrives and a state parameter needs to be added between previously existing state parameters. The NodeUpdater maintains its own copy of GraphValues for the state parameter being tracked and updates this with newly added state parameters. Additionally, the NodeUpdater is responsible for adding required relative factors for new state parameter nodes.

<p align="center">
<img src="./doc/images/factor_updating.png" width="500">
</p>

For a new state parameter node that is more recent than any previous nodes, this simply requires connecting the new node with the previous latest node using some type of factor that provides a relative constraint. Typically these are in the form of a motion model or using sensor measurements such as an IMU when dealing with a robot body pose as the state parameter. Additionally, NodeUpdaters are responsible for providing a SlideWindow() member function that handles removing old state parameter nodes. Each unique state parameter set tracked in a GraphOptimizer should have its own NodeUpdater to handle insertion and deletion of its nodes. NodeUpdaters must be added to the GraphOptimizer object using its AddNodeUpdater() function.

## GraphActionCompleter
The GraphActionCompleter provides the option of completing a graph action operation after a set of factors from a FactorAdder that were buffered have been added to the GraphOptimizer. A graph action is any operation modifying the newly added factors or other factors in the graph. Since factors from a FactorAdder are not immediately added to the graph optimizer (see Overivew section for more details), this enables checks on factors once required state parameter nodes are created. For example, checks can see if a factor measurement is an outlier and remove the factor. GraphActionCompleters should be pushed to the GraphOptimizer object using the AddGraphActionCompleter() function.

# Delayed Insertion of Factor and Key Objects
As factors are first buffered in the GraphOptimizer before being added (see Overview section for more details), not all required information for factors are available when they are created in FactorAdders. For example, GTSAM requires Keys for the state parameter nodes that factors depend on. Since these may not have been created yet, the FactorToAdd and KeyInfo classes are used to allow for the delayed insertion of factors.

## FactorToAdd
The FactorToAdd object contains a set of factors and KeyInfo objects that contain the required information for subsequent key and node creation in the GraphOptimizer.
## KeyInfo
KeyInfo objects contain the required information to create a gtsam::Key. They take a KeyCreatorFunction (typically gtsam::symbol_shorthand::X, where X should be unique for each state parameter type), the NodeUpdaterType so the graph optimizer knows which NodeUpdater to use for node creation, and an optional timestamp. KeyInfos can be static or dynamic, depending on if they vary in time or not.
The parameter reader package loads parameters from config files for various localization packages.
16 changes: 12 additions & 4 deletions localization/ros_graph_localizer/readme.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
\page rosgraphlocalizer Ros Graph Localizer

# Package Overview
Ros wrapper for the graph localizer. The graph localizer wrapper takes ros messages, convers these to localization measurements, and passes these to the graph localizer. The ros graph localizer nodelet handles live subscribing and publishing to ros topics.
Ros wrapper for the graph localizer. The graph localizer wrapper takes ros messages, convers these to localization measurements, and passes these to the graph localizer. The ros graph localizer nodelet handles live subscribing and publishing to ros topics. It runs the ros_graph_localizer, ros_graph_vio, and depth_odometry correspondence estimation sequential and passes input messages to each of these.

## RosGraphLocalizerWrapper
Contains the GraphLocalizer. Converts ROS messages to localization measurements and provides these to the localizer.
Expand All @@ -10,9 +10,17 @@ Contains the GraphLocalizer. Converts ROS messages to localization measurements
Subscribes to ROS messages for online use and publishes GraphVIO messages and TFs. Contains a RosGraphLocalizerWrapper and passes messages to this.

# Inputs
* `/graph_vio/state`
* `/hw/depth_haz/extended/amplitude_int`
* `/hw/depth_haz/points`
* `/hw/imu`
* `/loc/ar/features`
* `/loc/of/features`
* `/loc/ml/features`
* `/loc/depth/odom`
* `/loc/depth/odom`
* `/mob/flight_mode`

# Outputs
* `graph_loc/graph`
* `graph_loc/state`
* `/graph_loc/state`
* `/graph_vio/state`
* `/loc/depth/odom`
8 changes: 6 additions & 2 deletions localization/ros_graph_vio/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,14 @@ Ros wrapper for the graph VIO. The graph VIO wrapper takes ros messages, convers
## RosGraphVIOWrapper
Contains the GraphVIO. Converts ROS messages to localization measurements and provides these to the VIO object.

## ImuBiasInitializer
Assumes standstill and estimates IMU biases and stddevs from a sequence of IMU measurements.

# Inputs
* `/hw/imu`
* `/loc/of/features`
* `/loc/depth/odom`
* `/mob/flight_mode`

# Outputs
* `graph_vio/graph`
* `graph_vio/state`
* `/graph_vio/state`
8 changes: 4 additions & 4 deletions localization/ros_pose_extrapolator/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# Package Overview
Extrapolates localization estimates using relative odometry and IMU measurements.
The ros pose extrapolator wrapper takes ros messages, convers these to localization measurements, performs the required extrapolation. The ros pose extrapolatornodelet handles live subscribing and publishing to ros topics.
The ros pose extrapolator wrapper takes ros messages, converts these to localization measurements, and performs the required extrapolation. The ros pose extrapolator nodelet handles live subscribing and publishing to ros topics.

## RosPoseExtrapolatorWrapper
Converts ROS messages to localization measurements and extrapolates localization estimates using relative odometry and interpolated IMU measurements.
Expand All @@ -12,8 +12,8 @@ Subscribes to ROS messages for online use and publishes localization messages an

# Inputs
* `/hw/imu`
* `graph_localizer/state`
* `graph_vio/state`
* `/graph_localizer/state`
* `/graph_vio/state`

# Outputs
* `gnc/Ekf` (message named Ekf for historical reasons)
* `/gnc/Ekf` (message named Ekf for historical reasons)
23 changes: 2 additions & 21 deletions localization/tutorial_examples/readme.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,4 @@
\page graphvio Graph VIO
\page tutorialexamples Tutorial Examples

# Package Overview
Performs sliding-window graph based visual-intertial odometry (VIO) using a VOSmartProjectionFactorAdder and StandstillFactorAdder. Uses a CombinedNavStateNodeAdder for creating combined nav state nodes (containing a pose, velocity, and IMU bias) at required timestamps using IMU measurements.

# Background
For more information on the theory behind the GraphVIO and the factors used, please see our paper:
* Ryan Soussan, Varsha Kumar, Brian Coltin, and Trey Smith, "Astroloc: An efficient and robust localizer for a free-flying robot", Int. Conf. on Robotics and Automation (ICRA), 2022. [Link](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9811919 "Link")

# Graph Optimization Structure
<p align="center">
<img src="../doc/images/graph_vio.png" width="550">
</p>

## Factor Adders
* VoSmartProjectionFactorAdder
* StandstillFactorAdder
## Graph Factors
* CombinedIMUFactor
* RobustSmartProjectionFactor
* Standstill Factors (zero velocity prior and identity relative transform between factors)
## Node Adders
* CombinedNavStateNodeAdder
Examples for using the AstroLoc state estimation library. See the tests for various examples and reference the astroloc_library_quickstart.pdf in the localization/doc directory for more information on the library.

0 comments on commit 42d72a6

Please sign in to comment.