[Question] Passive joint and action space size (skrl)

[AndreaRossetto]

Question

Hi,
I've created a custom robot (kind of cassie robot) with some passive joints, in total I have 12 joints but only 10 are "useful" in the action state (the controllable joints and readable joint position are only 10). If in the DirectRLEnvCfg I set the action_space to 10 I get the error the inputs of the tensor is not matching

class RobotEnvCfg(DirectRLEnvCfg):
    # env
    decimation = 4                  # Number of time steps between actions
    episode_length_s = 20.0         # Episode time max in seconds
    action_scale = 0.5 
    action_space = 12               # Number of actions the neural net returns
    observation_space = 53 
    state_space = 0 

skrl yaml model:

models:
  separate: True
  policy:  # see gaussian_model parameters
    class: GaussianMixin
    clip_actions: False
    clip_log_std: True
    min_log_std: -20.0
    max_log_std: 2.0
    initial_log_std: 0.0
    network:
      - name: net
        input: STATES
        layers:
          - linear: [512]
          - linear: [256]
          - linear: [256]
        activations: ["relu", "relu", "relu"]
    output: ACTIONS
  value:  # see deterministic_model parameters
    class: DeterministicMixin
    clip_actions: False
    network:
      - name: net
        input: STATES
        layers:
          - linear: [512]
          - linear: [256]
          - linear: [256]
        activations: ["relu", "relu", "relu"]
    output: ONE

I wanna remove the two passive rod joints from the action_space, so the actor net output control only the main joins and not the rod joint. How can I do that?

[kellyguo11]

You can define a list of actuated joints, such as https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/direct/shadow_hand/shadow_hand_env_cfg.py#L148, and index the action buffer to only modify it at those indices.

[AndreaRossetto]

Thank you for the help, I'm changing like the example you posted so I'm doing like this:

Here I put only the actuated joints

    actuated_joint_names = [
        "hip_abduction_left",
        "hip_rotation_left",
        "hip_flexion_left",
        "knee_left",

        "hip_abduction_right",
        "hip_rotation_right",
        "hip_flexion_right",
        "knee_right",
    ]

I have to leave the 12 joint otherwise I get an error but I change the num_action to 8

action_space = 12
num_actions = 8

and I leave the ImplicitActuatorCfg as before like this

actuators = {
        "legs": ImplicitActuatorCfg(
            joint_names_expr = [ "hip_abduction_left", "hip_rotation_left", "hip_flexion_left", "knee_left", "tarsus_joint_left", "RevoluteJoint_left",
                                 "hip_abduction_right", "hip_rotation_right", "hip_flexion_right", "knee_right", "tarsus_joint_right", "RevoluteJoint_right"
                               ],
            effort_limit = { # N.m
                "hip_abduction_left":       8.0,
                "hip_rotation_left":        8.0,
                "hip_flexion_left":         8.0,
                "knee_left":                8.0,
                "tarsus_joint_left":        8.0,
                "RevoluteJoint_left":       8.0,

                "hip_abduction_right":      8.0,
                "hip_rotation_right":       8.0,
                "hip_flexion_right":        8.0,
                "knee_right":               8.0,
                "tarsus_joint_right":       8.0,
                "RevoluteJoint_right":      8.0,
            },
            velocity_limit = { # rad/s
                "hip_abduction_left":       32.0,
                "hip_rotation_left":        32.0,
                "hip_flexion_left":         32.0,
                "knee_left":                32.0,
                "tarsus_joint_left":        32.0,
                "RevoluteJoint_left":       32.0,

                "hip_abduction_right":      32.0,
                "hip_rotation_right":       32.0,
                "hip_flexion_right":        32.0,
                "knee_right":               32.0,
                "tarsus_joint_right":       32.0,
                "RevoluteJoint_right":      32.0,
            },
            stiffness = {
                "hip_abduction_left":       10.0,
                "hip_rotation_left":        10.0,
                "hip_flexion_left":         10.0,
                "knee_left":                10.0,
                "tarsus_joint_left":        0.0,
                "RevoluteJoint_left":       0.0,

                "hip_abduction_right":      10.0,
                "hip_rotation_right":       10.0,
                "hip_flexion_right":        10.0,
                "knee_right":               10.0,
                "tarsus_joint_right":       0.0,
                "RevoluteJoint_right":      0.0,
            },
            damping = {
                "hip_abduction_left":       0.05,
                "hip_rotation_left":        0.05,
                "hip_flexion_left":         0.05,
                "knee_left":                0.05,
                "tarsus_joint_left":        0.0,
                "RevoluteJoint_left":       0.0,

                "hip_abduction_right":      0.05,
                "hip_rotation_right":       0.05,
                "hip_flexion_right":        0.05,
                "knee_right":               0.05,
                "tarsus_joint_right":       0.0,
                "RevoluteJoint_right":      0.0,
            },
            friction = {
                "hip_abduction_left":       0.0,
                "hip_rotation_left":        0.0,
                "hip_flexion_left":         0.0,
                "knee_left":                0.0,
                "tarsus_joint_left":        0.0,
                "RevoluteJoint_left":       0.0,

                "hip_abduction_right":      0.0,
                "hip_rotation_right":       0.0,
                "hip_flexion_right":        0.0,
                "knee_right":               0.0,
                "tarsus_joint_right":       0.0,
                "RevoluteJoint_right":      0.0,
            },
        ),
    },

Is that correct or I'm missing something to change?

[RandomOakForest]

Following up on this. It seems you got this running, but let us know if you need further help. I can move this post into our Discussions section if so, for the team to follow up.

[AndreaRossetto]

I'm still having some problems with the joint and I don't know if is related to this or something else. During the training seems the joint are actuated but with low force and the robot collapse on the ground

[RandomOakForest]

Ok, I'll move this to our Discussions for the team to follow up. If you could post example videos, or share more on different configs you have tried, would be helpful.