Hi there, I'm new to Dagster, what is the best way to load external Python scripts and set the working directory for the loaded files? I tried setting the working_directory key of the import in the workspace.yaml (like so https://docs.dagster.io/concepts/repositories-workspaces/workspaces#loading-relative-imports) but it doesn't seem to work, logging the current working directory still shows the path where dagit is run. Thanks for your help!

Hi Nicolas - setting working_directory changes the directories in sys.path that are used for imports, but doesn't actually change the value of os.getcwd() (so in that sense the name is a little misleading). Could you share more about what you're hoping to change the working directory for? There may be another way to accomplish what you're hoping t odo

Hi Daniel, thank you for your answer! Basically let's say I have a data extraction pipeline and a data transformation pipeline (with dbt), that both live in their own directory. Till now I use cron to orchestrate the different tasks but I want to migrate to Dagster without changing too much of my architecture. What is the best practices in term of project structure for dagster ? What is the best way to load external repositories in that case ? Thank you I'll look into running my own grpoc server, even though it seems a little bit overkill for what I want to do. Also a workaround would be to use the dagster-shell api, but there is something i can't find in the documentation : how to start the execution of a command after the execution of multiple other commands is done. Thank you for your help!

Do you have an example of code that depends on the working directory being set to a particular value? Agree that running your own grpc server is overkill for just this use case. Re: project structure, it's totally fine to have jobs that live in different folders. You just need to have a single python module or file that imports all of the jobs and use that as an entry point. You can see an example here of a dagster repository that imports jobs from several different folders: https://github.com/dagster-io/dagster/blob/master/examples/hacker_news/hacker_news/repo.py

Some jobs I want to import need to load config files from the directory of the source file. Thank you for this link with all the example. In my case I think I just need to use dagster.utils.file_relative_path. Many thanks for your help daniel