Hi again, one more thing i am trying to figure out: Does someone have a hint how to debug with the VSCode debugger when running a pipeline via dagit? I am new to python and the best option i came up with so far is to use debugpy in a solid, but that only works if i start listening in the solid itself. Is there a better way to initialize debugpy globally and be able to set breakpoints in multiple solids to attach via VSCode debugger and then debug a whole pipeline?

Hey Mark - here’s my

launch.json

configuration to attach the vscode debugger to the dagit process

Thanks a lot!

I believe you will want to include

"subProcess": true

in the config section since

dagit

will use subprocess for interacting with your code and running pipelines

Hi, should this approach work for debugging sensor code or only for solids?

the sensor evaluation runs subprocess to the

dagster-daemon

so you would have to run that in a debugger which is not as simple since its not its own python module

Is the sensor also running as a subprocess when launched via

dagster sensor preview

? If not it should be possible to at least attach the debugger to a separate sensor launched that way, right?

@Mark does it work for you to change your sensor code to call

pdb.set_trace()

and then run

dagster sensor preview <sensor_name>

?

Hi @prha - i am not sure yet how extensive i will need to debug sensors, so i'll try if this approach to debug without breakpoints works for me. Thanks for the hint.

a debugger entry like this should work

{
            "name": "dagster sensor",
            "type": "python",
            "request": "launch",
            "module": "dagster",
            "args": [
                "sensor",
                "preview",
                // other args here
            ]
        },

Is this still a valid way to do debugging VSCode? This is my launch.json and it isn’t stopping on breakpoints. I’m running

DAGSTER_HOME=/Users/aaron/dagster_home/ dagster-daemon run --python-file repo.py

in a separate shell.

{
      "name": "Debug Dagit",
      "type": "python",
      "request": "launch",
      "module": "dagit",
      "cwd": "${workspaceFolder}/pipeline",
      "env": {
        "DAGSTER_HOME":"/Users/aaron/dagster_home/"
      },
      "args": [
              "-f",
              "repo.py"
      ],
      "console": "integratedTerminal",
      "subProcess": true

try

"justMyCode": false

Still doesn’t stop unfortunately. I’m triggering the job from a sensor, but the breakpoints are in the ops, not the sensor.

This worked for me recently:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Debug Dagit",
      "type": "python",
      "request": "launch",
      "module": "dagit",
      "cwd": "${workspaceFolder}",
      "args": [
        "-f",
        "repo.py"
      ],
      "console": "integratedTerminal"
    }
  ]
}

I’m triggering the job from a sensor

this goes through the daemon which doesn’t route anything through dagit, in the local case it has its own subprocess grpc server

use

dagster sensor preview

shown earlier in this thread

This worked, but only with breakpoints within the sensor

run daemon via vscode

Not sure what you mean by this, from above it sounded like you can’t run a debugger on the daemon itself. I’d also be ok using

breakpoint()

in places but that just causes the job to error when run.

stepping back, if you have the run config the sensor is submitting you can use the op/job testing APIs to directly execute in process https://docs.dagster.io/concepts/testing#testing

Oh, perfect, thanks!