https://dagster.io/ logo
d

Daniel Galea

12/14/2022, 9:27 AM
Hi everyone, is it possible to run Dagster in debug mode and use breakpoints? I am using VSCode
D 1
👀 1
3
p

Peter Davidson

12/14/2022, 9:59 AM
same q for PyCharm
m

Martin Picard

12/14/2022, 10:35 AM
yes breakpoint debug works for me with vscode, since it's just regular python and pytest works for me out of the box
d

Daniel Galea

12/14/2022, 10:39 AM
@Martin Picard so you can run Dagster and running a Job will trigger a breakpoint? Or do you mean that you Debug a specific Python file and it runs the default debugger?
m

Martin Picard

12/14/2022, 10:43 AM
ah sorry I misunderstood, in theory vscode can attach to any python process https://code.visualstudio.com/docs/python/debugging I was talking specifically about test debugging, which works great, would recommend debugging your unit tests or creating a test for your specific problem first
👍 1
👍 1
d

Daniel Galea

12/14/2022, 10:49 AM
Thanks for the links 🙂 I was hoping to have something like developing and debugging Azure Functions locally which lets you run a function (which is basically a web app) locally and then you can use breakpoints which are triggered when different parts of the app are called. Unit testing and debugging will work too 🙂
p

Peter Davidson

12/14/2022, 11:26 AM
code step through, running in debug mode, is a nice feature in pycharm too. idk how realisic it is to expect dagster to cater to this but it's nice.
b

Benedikt Buchert

02/13/2023, 12:44 PM
I tried attaching to the process ID no success. Would be really nice for debugging though. Currently I just use logging only in my development process.
a

Abhishek Agrawal

03/03/2023, 1:31 PM
So, we can't attach a debugger to dagster running on localhost and debug in PyCharm, right? Was anybody able to do this?
@sandy is there a way to do this?
s

sandy

03/03/2023, 4:20 PM
my suspicion would be that it's possible, but I'm not super familiar with PyCharm or attaching debuggers to running Python processes
a

Abhishek Agrawal

03/04/2023, 4:50 AM
I tried attaching to the process but it didn't work.. could you give it a go on your end?
s

sandy

03/04/2023, 7:25 PM
sorry but this isn't a setup that I have
b

Benedikt Buchert

03/04/2023, 8:29 PM
If you have vs code can you try 😄 I was not able to get it to work.
a

Abhishek Agrawal

03/04/2023, 10:19 PM
Not working in PyCharm too..
p

Peter Davidson

03/09/2023, 6:55 AM
hi - what we have got working, in PyCharm, is runniing an execution in process. eg. a file: local_debug.py containing the following code, then run this file in debug mode in pycharm, works great. It does mean you need a code snippet for every asset you want to debug though, -> maybe there is a more efficient way to do this, but for now we're just building up a long debug file with all the code snippets and just toggle code comment for the bit you want to debug.
Copy code
os.environ["DAGSTER_DEPLOYMENT_NAME"] = <...>
deployment_name = os.getenv("DAGSTER_DEPLOYMENT_NAME", "local")
resource_defs = configured_resources_by_deployment(deployment_name)

if __name__ == "__main__":
    sources = [sourceA.to_source_asset(), *source_multi_asset.to_source_assets()]
    materialize(assets=[asset_to_debug] + sources,
                partition_key=MultiPartitionKey({"rep_date": "2022-12-01", "scenario_key": "scenA"}),
                resources=resource_defs)
a

Abhishek Agrawal

03/09/2023, 7:00 AM
Okay so yeah this is something I also ended up with. A giant local_code.py file. Haha..
🌈 1
331 Views