hey all! i'm currently using dagster to run dbt transformations, and we want to generate dbt docs after transformations are run. we currently have a github actions workflow to build an image for generating and deploying the dbt docs and push it to ECR. we are looking for the best way to call this workflow from dagster. we were thinking of calling the github actions API to create a workflow dispatch event but weren't sure the best way around authenticating because you need a JWT so we'd have to write/maintain code for it, something we'd like to avoid. we were also looking at the dagster-github integration but it looks like it only supports the github graphql API. any thoughts about the best way to approach this? we are also open to not using the github actions workflow if there is another way to build and push our docker image for the dbt docs from dagster.

Hi Salina, I’m by no means a dagster pro, but the way I set it up in my team (in airflow, pending migration to dagster) was to have a small python wrapper script around the generate docs call that will execute it and copy everything into an S3 bucket, which then gets exposed as a static website. Wouldn’t something like that work for you?

@Vinnie thanks for the input! we are building and pushing a docker image that generates the docs and uses nginx to serve our docs, but yeah we'll probably need to have some wrapper code to handle the JWT regeneration if there's no other out of the box way. i'm assuming that's what was included in your script?

@Salina Wu In our case it’s not necessary since the website can be public inside our VPN, so we’re just generating and pushing the docs to s3. I guess if you’re serving it with nginx you might need that, yeah. I’m definitely not well suited to help in this case though haha

sorry for the long response time… it sounds like one thing you’re looking for is additional methods in the dagster-github integration to support REST-endpoints in addition to the graphql endpoints (because the workflow dispatch flow is only supported on REST)?

Created issue at: https://github.com/dagster-io/dagster/issues/7739

yes @prha thanks for creating the issue!