How does development with dagster work?

[dagsir[bot]]

Hello newbie here. I've managed to deploy to Docker on my server. However, I am really struggling to grasp how to dev on the assets etc? Is the idea that you would SSH into your docker container? Use a mounted volume and SSH into that? Or have your code stored in another location? I'm sure i'm missing something obvious and any help would be greatly appreicated

---

The question was originally asked in Dagster Slack.

[benpankow]

Dagster is build to allow you to develop locally while deploying to a production environment like Docker, Kubernetes, or Dagster Cloud.

Typically, users will run Dagster locally using dagster dev so that they can make changes and test their code live. The pluggable config & resource systems allow you to point to different external resources (like databases or APIs) locally, for unit tests, and in production, meaning you can build and test your code against mock or staging data.

[EtienneT]

Something that is not mentioned anywhere in the doc but has been life changing for me is to add the following run profile in launch.json of your visual studio code:

{
    "name": "dagster dev",
    "type": "python",
    "request": "launch",
    "module": "dagster",
    "args": [
        "dev",
    ],
    "subProcess": true
}

Then, in vscode you can hit F5 which will run dagster dev locally in debug mode and will attach visual studio code debugger to it. You can then open the web interface here: http://localhost:3000/

Then you can add breakpoints anywhere in your dagster code inside Visual Studio Code. For example you could trigger the materialization of an asset from the web interface and then if you have a breakpoint inside your code for this asset, you could step in the code line by line in the visual studio debugger.

Without this you are left using context.log.info to try to print logs to understand bugs.

I would have learned dagster 100x faster if I would have known that since the beginning.

[philiporlando]

I found a similar approach to debugging Dagster within Docker containers in #25252.