https://dagster.io/ logo
#ask-community
Title
# ask-community
b

Brian Pohl

06/14/2023, 4:28 PM
hey Dagster team, I am using Dagster Hybrid and Github Actions to deploy. the
DAGSTER_CLOUD_GIT_SHA
environment variable is perfect for my needs, except i need the temporary merge commit that is made against my base branch, rather than the commit i made and pushed. all my other deployment processes use
GITHUB_SHA
- this page talks about how
GITHUB_SHA
is this merge commit. i have tried everything i can think of, including peeking inside the deploy code called by the CLI. but the
DAGSTER_CLOUD_GIT_SHA
is always the one I pushed - the one you can see on the PR page. i don’t necessarily need to change
DAGSTER_CLOUD_GIT_SHA
- instead if you could help me pass
GITHUB_SHA
into my Dagster deployment, that’d work too! But so far I have not had luck passing build arguments into my Docker image and setting them as env variables. (side question: should i expect that env variables set in my Docker image would be accessible in my
@repository
description?) for reference, my deployment Github workflow looks very much like this one. any ideas how i can access that sha and set it as an env variable? thanks!
🤖 1
z

Zach

06/14/2023, 5:47 PM
Have you consider just not using the github action and just using the dagster cli to deploy your code? You'd have more control over the github sha
b

Brian Pohl

06/14/2023, 6:00 PM
I’ve got a big enough team of data scientists who need to test their code on branch deployments that i need to automate these deployments. I personally wouldn’t mind using the CLI, but that won’t scale well 😕
z

Zach

06/14/2023, 6:07 PM
I mean using the CLI from github actions
just adding a step that does
dagster-cloud workspace update-location ...
b

Brian Pohl

06/14/2023, 6:09 PM
oh gotcha. i have a command that says:
Copy code
ci set-build-output --location-name=${{ matrix.location.name }} --image-tag=$IMAGE_TAG-${{ matrix.location.name }}
i don’t know this command well - is there another argument i can pass that overrides the sha?
that command ^ is passed to this action
z

Zach

06/14/2023, 6:12 PM
re-read your question - it's your docker image that you're trying to insert a particular env var into?
b

Brian Pohl

06/14/2023, 6:13 PM
yeah, ultimately i want this env var accessible within my
@repository
and
@job
Python scripts. i’m passing it into the Docker image and i think that should do the trick
z

Zach

06/14/2023, 6:14 PM
I think that command you pointing to is just telling the dagster cloud action what image to use and what location to update - you probably have a separate step where you're actually building your docker image right? Something like
Copy code
- name: Build and push fini Docker image
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          builder: ${{ steps.buildx.outputs.name }}
          tags: "${{ env.IMAGE_REGISTRY }}:${{ github.sha }}"
          labels: |
            branch=${{ github.ref_name }}
           ...
I don't think the Dagster action actually builds your image for you
b

Brian Pohl

06/14/2023, 6:17 PM
oh yeah, i’ve got an image build step. i have been fighting the build arguments to try to get an env variable set in there
Copy code
- name: Build and upload Docker image for ${{ matrix.location.name }}
  if: steps.prerun.outputs.result != 'skip'
  uses: docker/build-push-action@v4
  with:
    context: ./process
    push: true
    tags: ${{ env.IMAGE_REGISTRY }}:${{ env.IMAGE_TAG }}-${{ matrix.location.name }}
    build-args: |
      DYNAMO_TABLE_MAPPING={{ env.DYNAMO_TABLE_MAPPING }}
(Dynamo table mapping is an unrelated thing, but same concept - it requires passing an env var in) my question on that note is, am i going down the right road? if i keep fighting the build arguments and get them to work, would my jobs be able to access it?
z

Zach

06/14/2023, 6:17 PM
I see but you're trying to rely on what's injected by Dagster as environment variables
yeah I'd just do it that way, I do the same thing to pass in some environment variables via build-args
your jobs should be able to access the environment variables if you assign the build-arg to an env var in the image
👍 1
b

Brian Pohl

06/14/2023, 6:20 PM
perfect, thanks! i am just fighting some github workflow/Docker syntax cause it seems my argument is not being passed in correcctly. but once i get it working, i can use that as a template to also pass in the git sha. thanks @Zach!
🎉 1
d

daniel

06/14/2023, 7:35 PM
I suspect GITHUB_SHA is reserved by Github in some fashion which is why you're having trouble overriding it - setting some other env var on the Docker image as per Zach's suggestion seems reasonable to me
b

Brian Pohl

06/14/2023, 7:37 PM
I just got the env var working, so that's the way I'll go. Thanks guys!
2 Views