Good afternoon, Is there any way to control the synchronies? I run several functions within an op and I have the problem that it does not wait for one to finish before starting the other. Any solution? Thanks! Example: @op def do_some_task(): function1() function2() * Dagster starts to perform function2 before function1 has finished.

Hey @Sara - looking at your code example, I would expect that

function2

would not start until

function1

completes. Is it possible that you're experiencing this with a

graph

instead of an

op

?

I guess you are right with your recommendation here @sandy However, I am facing a similar problem when interacting with duckDB from a sensor ( where the sensor might fire multiple executions in parallel. The parquet file writing works, however, dagster fails to obtain a lock on duckdb - as another thread/process is busy registering results. How can I have a lock on duckdb (for the sole purpose of registering a parquet file with it (which is a pretty instant operation)? I do not want to de-parallelize the other processing (in process executor). And I also for now do not want to 1) restructure the graph or 2) use K8s with celery.

@geoHeil unfortunately, I don't think there's a super easy answer here - duckdb has this single-threaded limitation, and there's no straightforward way to tell Dagster not to execute ops and sensors at the same time. The way I would go about this would be to create some sort of file-based lock to ensure that multiple processes aren't trying to access duckdb at the same time: https://stackoverflow.com/questions/489861/locking-a-file-in-python

Hi @sandy, it does indeed happen with graph. Any way to control it? Greetings.