Hello everyone. I'm using dagster on localhost and try to run 2 jobs: first one is for scanning a directory and storing the directory tree in Postgres db. The second job is for extraction files metadata and store the result in elasticsearch db. when I execute dagster repository from my console, everything works good but when I try to launch the jobs from dagit the following errors appears and they are linked to database connection:

Hi! do you mind sharing some code snippets? the error message is saying the return value of your op isn’t pickable, while the default io manager using trying to pickle and persist the outputs on a temp dir for passing data between ops - did you

return db_connection

in an op?

No. I try to return a db_connection while executing the job and I try to reuse in other ops within the same job.

If you’d like other ops to reuse the db_connection, you can set it as a resource: https://docs.dagster.io/concepts/resources#using-resources-with-ops returnning db_connection as an output will cause this error because dagster would try to persist the db connection on disk, while using resources will keep that db connection cross ops.

Thank you @yuhan! wonderful tool and wonderful support!