[Question] How can I get which usd have been spawned in each env when using MultiUsdFileCfg?

[virgilio96upm]

Question

Hi,

I'm working on a manager-based RL project where I need to track which object has been spawned in each environment and include that information in the observation. Specifically, I need to determine which USD path is associated with each environment.

I have modified the FrankaLiftCube task to spawn different objects that the robot must learn to lift:

class FrankaCubeLiftEnvCfg(LiftEnvCfg):
    def __post_init__(self):
        # post init of parent
        super().__post_init__()

        # Set Franka as robot
        self.scene.robot = FRANKA_PANDA_CFG.replace(prim_path="{ENV_REGEX_NS}/Robot")

        self.scene.replicate_physics= False
        # Set actions for the specific robot type (franka)
        self.actions.arm_action = mdp.JointPositionActionCfg(
            asset_name="robot", joint_names=["panda_joint.*"], scale=0.5, use_default_offset=True
        )
        self.actions.gripper_action = mdp.BinaryJointPositionActionCfg(
            asset_name="robot",
            joint_names=["panda_finger.*"],
            open_command_expr={"panda_finger_.*": 0.04},
            close_command_expr={"panda_finger_.*": 0.0},
        )
        # Set the body name for the end effector
        self.commands.object_pose.body_name = "panda_hand"

        # Set objects as rigid objects
        self.scene.object = RigidObjectCfg(
            prim_path="{ENV_REGEX_NS}/Object",
            init_state=RigidObjectCfg.InitialStateCfg(pos=[0.5, 0, 0.055], rot=[1, 0, 0, 0]),
            spawn=MultiUsdFileCfg(
                usd_path= [
                    f"/home/mt/Desktop/STL_shapes/scaled_tools/simple_L.usd",
                    f"{ISAAC_NUCLEUS_DIR}/Props/Blocks/DexCube/dex_cube_instanceable.usd",
                    f"{ISAAC_NUCLEUS_DIR}/Props/Blocks/MultiColorCube/multi_color_cube_instanceable.usd",
                ],
                random_choice=True,
                scale=(0.8, 0.8, 0.8),
                rigid_props=RigidBodyPropertiesCfg(
                    solver_position_iteration_count=16,
                    solver_velocity_iteration_count=1,
                    max_angular_velocity=1000.0,
                    max_linear_velocity=1000.0,
                    max_depenetration_velocity=5.0,
                    disable_gravity=False,
                ),
            ),
        )

        # Listens to the required transforms
        marker_cfg = FRAME_MARKER_CFG.copy()
        marker_cfg.markers["frame"].scale = (0.1, 0.1, 0.1)
        marker_cfg.prim_path = "/Visuals/FrameTransformer"
        self.scene.ee_frame = FrameTransformerCfg(
            prim_path="{ENV_REGEX_NS}/Robot/panda_link0",
            debug_vis=False,
            visualizer_cfg=marker_cfg,
            target_frames=[
                FrameTransformerCfg.FrameCfg(
                    prim_path="{ENV_REGEX_NS}/Robot/panda_hand",
                    name="end_effector",
                    offset=OffsetCfg(
                        pos=[0.0, 0.0, 0.1034],
                    ),
                ),
            ],
        )

Using the debugger, I am inspecting object: RigidObject = env.scene[SceneEntityCfg("object")], which allows me to access different fields. The configuration (cfg) correctly assigns each USD path string an index corresponding to its position in the provided list.

However, I cannot find where the selected USD path for each environment is stored. I would like to retrieve a tensor of shape (num_envs, 1), where each row contains the usd_path_id, which corresponds to the index of the USD path in the configuration file.

How can I achieve this?

[RandomOakForest]

Thanks for posting this. A similar discussion was recently started here. We are looking into this and will write back as soon as possible.

[RandomOakForest]

Following up on this, the team suggests you can parse the references in a separate observation class. Let us know if you need a specific example.

[virgilio96upm]

Thanks for the follow-up! A specific example would be really helpful to understand the best way to implement this.

[RandomOakForest]

Ok, I'll move the post into our Discussions section for the team to follow up.

[virgilio96upm]

Hi, is there any update on this? Thanks!

[bamertsa]

This is how I solved it in my case.

[virgilio96upm]

Thanks for your message! I'll check it out. Also, yesterday I found another approach that does not modify the wrapper and instead retrieves the stage to access the prim stack to check which usd was actually loaded in each env using the identifier method.

    stage = omni.usd.get_context().get_stage()
    prim_paths = sim_utils.find_matching_prim_paths(object.cfg.prim_path)
    with Sdf.ChangeBlock():
        for prim_path in prim_paths:
            # spawn single instance
            prim = stage.GetPrimAtPath(prim_path)
            stack = prim.GetPrimStack()
            object_file = stack[1].layer.identifier