Running dbt-deps in Dagster Cloud 🧵

You’re definitely thinking along the right track! Skimming those docs, you’d want the post install hook.

No luck with post_install / pre_install shell scripts. I’m running it similarly to how you would within Docker without the same success. I just have a dagster_cloud_post_install.sh file with the following command:

RUN (cd ./dbt_project/; dbt deps --profiles-dir ./dbt_project/config/profiles.yml)

those are just shell scripts, so you'd want to follow normal shell/bash semantics - what you have inside your RUN () block would probably work.

cd ./dbt_project/; dbt deps --profiles-dir ./dbt_project/config/profiles.yml

Still no luck, unfortunately. Even if I just try and echo using a hook, it doesn’t echo it in the cloud log

I'm not using dagster cloud - using dagster and dbt in a k8s deployment

Okay, thanks for that. I’ll wait to see if Dagster support get back to me and in the meantime look into an alternative approach

only other suggestion is to run

chmod +x dagster_cloud_post_install.sh

I’d echo Adam’s suggestions. This command is run in the GitHub build so you’d see the echo or other command output there (in the actions tab) not in dagster cloud. A couple other things jump out: 1. Be sure you have disabled fast deploys by modifying the setting within the .github/workflows/deploy.yaml file 2. You don’t need the RUN bit, the lifecycle hook is just a shell script that we execute and add the RUN command too. So you can just have the command itself 3. The —profiles-dir flag should point to the config directory not the profiles file within it definitely do the chmod command Adam listed before committing that script Let us know if this doesn’t resolve the issue, and it’d be great to see the logs from the github actions side

I disabled the fast deploy which has resolve the first error in github actions

Did you run

chmod +x

on that file? You’ll need to commit the change afterwards

@Adam Bloom That worked in a local set-up 👍 Thanks @Sean Lopp Any guidance on how I could mirror this in the cloud set-up and run

chmod +x

before the post install hook?

Did you commit and push? You shouldn’t need to run that in actions - it’s part of the file metadata stored in git

If took me a hot minute the first time too. But as Adam says, if you do

git status

After running the

chmod

command you'll see that the shell script has been updated (even though the contents are the same). You'll want to add that file, commit, and push the result. Then the action should be happy. Unfortunately some git clients don't show this change as a file modification, but the manual git commands will work

@Adam Bloom @Sean Lopp That’s worked perfectly, thank you. The time you have spent on this is very much appreciated, thanks again