Migrating legacy Dagster code to latest version - Trying to find a similar behavior for preset_defs

[kennycontreras]

Hello team! I need some help migrating legacy Dagster code to the latest version. Specifically, I'm struggling to find a similar behavior of a preset_defs that allows me to declare a list of configs for just one solid but when I go to the launchpad, I have the option to select all or select just one part of the jobs list, so for example key1, key2, and key3. Not sure if that's possible anymore

run_config = {
    "solids": {
        "my_solid": {
            "config": {
                "jobs": [
                    {
                        "key1": "value1",
                        "key2": "value2",
                        "key3": "value3",
                    },
                    {
                        "key1": "value4",
                        "key2": "value5",
                        "key3": "value6",
                    },
                    {
                        "key1": "value7",
                        "key2": "value8",
                        "key3": "value9",
                    },
                ]
            }
        }
    }
}

Not sure if I explained correctly, but I'll share a screenshot with the example.

Thanks 🙇🏻‍♂️

[benpankow]

Hi Kenny,

One common way to structure the same compute with multiple different config options is to create a single graph with multiple job definitions built from that graph. A simple example would be the following:

from dagster import op, Config, Definitions, graph, RunConfig


class Op1Config(Config):
    a_string: str
    a_bool: bool


@op
def my_op_1(config: Op1Config):
    pass


class Op2Config(Config):
    an_int: int


@op
def my_op_2(config: Op2Config):
    pass


@graph
def my_graph():
    my_op_1()
    my_op_2()


preset_1_job = my_graph.to_job(
    name="preset_1_job",
    config=RunConfig(
        ops={
            "my_op_1": Op1Config(a_string="foo", a_bool=True),
            "my_op_2": Op2Config(an_int=1),
        }
    )
)
preset_2_job = my_graph.to_job(
    name="preset_2_job",
    config=RunConfig(
        ops={
            "my_op_1": Op1Config(a_string="bar", a_bool=False),
            "my_op_2": Op2Config(an_int=2),
        }
    )
)


defs = Definitions(jobs=[preset_1_job, preset_2_job])

[kennycontreras]

Thanks @benpankow 🙇🏻‍♂️

Following your example, let's suppose I want to pass a list of a_string, a_bool to my_op_1 so when I go to the launchpad I have the option to 1. select the whole list, 2. select a sequence of the list

For example:

[{a_string="bar", a_bool=False}, {a_string="bar1", a_bool=True}, {a_string="bar2", a_bool=True}]

And finally, select the second one ({a_string="bar1", a_bool=True}) from that list for the execution. Is that possible?

[kennycontreras]

I think I found what I was looking for. I did a small test using @static_partitioned_config, and it helped me to achieve what I was looking for with a little bit of workaround.

With that, I defined the partition keys, and then created a function to change the config's remaining values based on the selected partition key. That's more than enough for now.

One point that could be helpful is to add more examples of @static_partitioned_config; should I create an issue for that?

Thanks a lot, @benpankow 🙇🏻‍♂️