Skip to content

Commit

Permalink
Merge pull request #75 from StanfordVL/readme_update
Browse files Browse the repository at this point in the history
updated to reflect 1) new api and 2) replacement of IG 2.0 with OG
  • Loading branch information
sanjanasrivastava authored Sep 21, 2023
2 parents 3bd15bd + fc9d7a6 commit 7afa310
Showing 1 changed file with 16 additions and 10 deletions.
26 changes: 16 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,40 +85,44 @@ The `:goal` section specifies the condition that the agent must satisfy to be su

You will typically want to use BEHAVIOR activities with a simulator. To use a BEHAVIOR activity without a simulator, use the following code.
```
from bddl.activity_base import BEHAVIORActivityInstance
from bddl.activity import Conditions
behavior_activity = "storing_the_groceries" # the activity you want to try, full list in bddl/bddl/activity_definitions
activity_definition = 0 # the specific definition you want to use. As of BEHAVIOR100 2021, this should always be 0.
simulator = "omnigibson" # this does not require an actual simulator, just a domain file (e.g. activity_definitions/domain_omnigibson.bddl). You can make your own if desired.
conds = Conditions(behavior_activity, activity_definition, simulator)
behavior_activity_instance = BEHAVIORActivityInstance(behavior_activity=behavior_activity, activity_definition=activity_definition)
# You can now use the functions in bddl/activity.py to interact with the conds object. This generally requires a backend that's based on the simulator; in this case, you can use a stub backend. You can create something similar to, or directly use, the DebugBackend and DebugObject found in tests/bddl_debug_backend.py.
```

### With simulator

To use a BEHAVIOR activity with a simulator, create a subclass of `BEHAVIORActivityInstance` for your simulator. Example for iGibson 2.0. This will require an implementation of sampling functionality or pre-sampled scenes that satisfy the activity's initial condition and implementation for checking each type of binary kinematic predicate (e.g. `ontop`, `nextto`) and unary nonkinematic predicate (e.g. `cooked`, `saturated`).
To use a BEHAVIOR activity with a simulator, create a subclass of `BDDLBackend` for your simulator. An example for OmniGibson is found [here](https://github.com/StanfordVL/OmniGibson/omnigibson/utils/bddl_utils.py#L169). This will require an implementation of sampling functionality or pre-sampled scenes that satisfy the activity's initial condition and implementation for checking each type of binary kinematic predicate (e.g. `ontop`, `nextto`) and unary nonkinematic predicate (e.g. `cooked`, `saturated`).

## Logic evaluator for goal

When using BEHAVIOR activities with a simulator, the goal condition is evaluated at every simulator step by calling `simulator_activity_instance.check_success()`, where `simulator_activity_instance` is some subclass of `BEHAVIORActivityInstance`. `bddl.logic_base` and `bddl.condition_evaluation` contain this functionality. Atomic formulae that interface directly with the simulator are implemented in `bddl.logic_base`. These require the simulator checking functions for various predicates to be implemented, and are the leaf nodes of the compositional expression making up a goal condition or the list of literals making up an initial condition. Logical operators are implemented in `bddl.condition_evaluation`, and form a compositional structure of the condition to evaluate.
When using BEHAVIOR activities with a simulator, the goal condition is evaluated at every simulator step by calling `bddl.activity.evaluate_goal_conditions(goal_conditions)`, where `goal_conditions` is the output of `bddl.activity.get_goal_conditions(conds, backend, scope)`, and `scope` is the output of `bddl.activity.get_scope(conds)` that has then been populated by mapping the string identifiers to actual objects from your simulator. `bddl.logic_base` and `bddl.condition_evaluation` contain the actual evaluation functionality. Atomic formulae that interface directly with the simulator are implemented in `bddl.logic_base`. These require the simulator checking functions for various predicates to be implemented, and are the leaf nodes of the compositional expression making up a goal condition or the list of literals making up an initial condition. Logical operators are implemented in `bddl.condition_evaluation`, and form a compositional structure of the condition to evaluate.

## Solver for ground goal solutions

`bddl.condition_evaluation` also contains basic functionality to generate ground solutions to a compositional goal condition, including one that may contain quantification. This functionality is much like a very simple, unoptimized logic program, and will return a subset of solutions in cases where the solution set is too large to compute due to exponential growth.
`bddl.condition_evaluation` also contains basic functionality to generate ground solutions to a compositional goal condition, including one that may contain quantification. This functionality is much like a very simple, unoptimized logic program, and will return a subset of solutions in cases where the solution set is too large to compute due to exponential growth. To enable this, set `generate_ground_options=True` when using `bddl.activity`.

# Using BEHAVIOR with a new simulator

Using BEHAVIOR activities with a new simulator requires implementing its functional requirements for that simulator, as has been done for iGibson 2.0 [3].
Using BEHAVIOR activities with a new simulator requires implementing its functional requirements for that simulator, as has been done for OmniGibson [4].

### Implementation of BDDL predicates as simulated object states

To simulate a BEHAVIOR activity, the simulator must be able to simulate every predicate involved in that activity. The full list of predicates is at [TODO add list of predicates to config]. For any one activity, the required predicates can be found by reading its BDDL problem (in activity_definitions/<activity_name>/.)

Implementing these requires 1) a simulator-specific child class of the `BDDLBackend` class ([link](https://github.com/StanfordVL/bddl/blob/654cfefb078dbdf264957a08a30571086a2aa726/bddl/backend_abc.py#L6-L9)) and 2) implementations of object states such as `cooked` and `ontop` that can both **instantiate** an object as e.g. `cooked` or `not cooked`, and **check** whether the predicate is true for a given object.
Implementing these requires 1) a simulator-specific child class of the `BDDLBackend` class and 2) implementations of object states such as `cooked` and `ontop` that can both **instantiate** an object as e.g. `cooked` or `not cooked`, and **check** whether the predicate is true for a given object.

**1. Child of `BDDLBACKEND`:** This class has one method, `get_predicate_class`. It must take string tokens of predicates from BDDL problems (e.g. `"cooked"`, `"ontop"`) and map them to the simulator's object states. Example: [iGibson's `BDDLBackend` child class](https://github.com/StanfordVL/iGibson/blob/ig-develop/igibson/task/bddl_backend.py).
**1. Child of `BDDLBACKEND`:** This class has one method, `get_predicate_class`. It must take string tokens of predicates from BDDL problems (e.g. `"cooked"`, `"ontop"`) and map them to the simulator's object states..

**2. Simulated object states:** For any object in a BEHAVIOR activity, it must be instantiated in certain simulated states and be checked for certain simulated states, as specified by a BDDL problem. `BDDLBackend` expects state implementations that are object agnostic, but the implementation is ultimately up to the user. Assuming object-agnostic states, each one should be able to take an object and instantiate that object with the given state if applicable, and check whether that object is in that state or not. Example: [iGibson's object state implementations](https://github.com/StanfordVL/iGibson/tree/ig-develop/igibson/object_states).
**2. Simulated object states:** For any object in a BEHAVIOR activity, it must be instantiated in certain simulated states and be checked for certain simulated states, as specified by a BDDL problem. `BDDLBackend` expects state implementations that are object agnostic, but the implementation is ultimately up to the user. Assuming object-agnostic states, each one should be able to take an object and instantiate that object with the given state if applicable, and check whether that object is in that state or not. Example: [OmniGibson's object state implementations](https://github.com/StanfordVL/OmniGibson/omnigibson/object_states).

*Note on binary predicates:* in BDDL, all binary predicates are kinematic (`ontop`, `nextto`, `touching`, etc.). Instantiating objects in the associated simulator states is more complex than instantiating objects in unary predicates' states due to potential for failure based on physical constraints of the scene and multiple possibilities for object pairing, especially when implementing scene-agnostic instantiation capable of generating infinite distinct episodes. Please look at the setter methods of kinematic states in iGibson 2.0 for a robust example capable of instantiating BEHAVIOR activities with many objects.
*Note on binary predicates:* in BDDL, certain binary predicates are kinematic (`ontop`, `nextto`, `touching`, etc.). Instantiating objects in the associated simulator states is more complex than instantiating objects in unary predicates' states due to potential for failure based on physical constraints of the scene and multiple possibilities for object pairing, especially when implementing scene-agnostic instantiation capable of generating infinite distinct episodes. Please look at the setter methods of kinematic states in OmniGibson for a robust example capable of instantiating BEHAVIOR activities with many objects.

# Testing

Expand All @@ -135,3 +139,5 @@ should evaluate true.
[2] PDDL Parser (2020). Version 1.1. [Source code]. https://github.com/pucrs-automated-planning/pddl-parser.

[3] C. Li*, F. Xia*, R. Martín-Martín*, M. Lingelbach, S. Srivastava, B. Shen, K. Vainio, C. Gokmen, G. Dharan, T. Jain, A. Kurenkov, C. K. Liu, H. Gweon, J. Wu, L. Fei-Fei, S. Savarese. iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks. CoRL 2021.

[4] C. Li*, R. Zhang*, J. Wong*, C. Gokmen*, S. Srivastava*, R. Martín-Martín*, C. Wang*, G. Levine*, M. Lingelbach, J. Sun, M. Anvari, M. Hwang, M. Sharma, A. Aydin, D. Bansal, S. Hunter, K.-Y. Kim, A. Lou, C. R. Matthews, I. Villa-Renteria, J. H. Tang, C. Tang, F. Xia, S. Savarese, H. Gweon, C. K. Liu, J. Wu, L. Fei-Fei. BEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation. CoRL 2022.

0 comments on commit 7afa310

Please sign in to comment.