https://dagster.io/ logo
Join Slack
Powered by
I'm trying to get github actions configured to clo...
# ask-community
m
I'm trying to get github actions configured to clone our production snowflake DB as per the branch deployments guide. The latest launch_job action has a different input format to the deprecated action referenced in the guide. In effect, I am trying to get the same behaviour of the old method but using the newer action, but I can't work out how to pass the branch deployment location in to the "location_name" field, as it is getting data from 
parse_workspace
and I'm not entirely sure what data is coming from where...any help would be appreciated!
p
Those docs are a little out of date. I think you will need to update the image being referenced there:
Copy code
- name: Clone Snowflake schema upon launch
          if: github.event.action != 'closed'
          uses: dagster-io/dagster-cloud-action/tree/main/actions/launch_job@v0.1
          with:
            organization: my_org_id
            location: ${{ toJson(matrix.location) }}
            dagster_cloud_api_token: ${{ secrets.DAGSTER_CLOUD_API_TOKEN }}
            deployment: ${{ steps.deploy.outputs.deployment }}
            job: clone_prod
m
The launch_job action in the new version has the 
deployment
argument hardcoded to 
prod
, and a few variables have changed. Location doesn't exist anymore for example
Copy code
clone_prod_snowflake_db:
    runs-on: ubuntu-latest
    needs: [parse_workspace, dagster_cloud_build_push]
    name: Create branch deployment
    if: github.event.action != 'closed'
    steps:
      - name: Clone production snowflake DB
        if: github.event.action != 'closed'
        uses: dagster-io/dagster-cloud-action/actions/utils/run@v0.1
        with:
          organization_id: ${{ secrets.ORGANIZATION_ID }}
          deployment: ${{ github.ref_name }}
          location_name: ${{ toJson(needs.parse_workspace.outputs.build_info.location) }}
          job_name: clone_prod
        env:
          DAGSTER_CLOUD_API_TOKEN: ${{ secrets.DAGSTER_CLOUD_API_TOKEN }}
I'm trying to do it with the underlying util instead with the above, but I'm running in to a graphql issue
p
I’ll make some edits to the 
launch_job
action. Can you share the trace with the graphql error?
m
I think the issue was on my end rather than graphql!
Latest attempt at the action format is this:
Copy code
clone_prod_snowflake_db:
    runs-on: ubuntu-latest
    needs: [parse_workspace, dagster_cloud_build_push]
    name: Modify branched snowflake table
    steps:
      - name: Get branch name
        id: branch-name
        uses: tj-actions/branch-names@v6
 
      - name: Clone production Snowflake DB
        if: github.event.action != 'closed'
        uses: dagster-io/dagster-cloud-action/actions/utils/run@v0.1
        with:
          organization_id: ${{ secrets.ORGANIZATION_ID }}
          deployment: ${{ steps.branch-name.outputs.current_branch }}
          location_name: ${{ fromJSON(needs.parse_workspace.outputs.build_info)[0].name }}
          job_name: clone_prod
        env:
          DAGSTER_CLOUD_API_TOKEN: ${{ secrets.DAGSTER_CLOUD_API_TOKEN }}
I get
Copy code
DagsterCloudHTTPError: 401 Client Error: Unauthorized for url: 
305
<https://dagster.cloud/***/dummy-branch-trigger/graphql>: Unauthorized
What should I be passing in for the 
repository
option? I'm seeing this at the top of the logs
Copy code
Run dagster-io/dagster-cloud-action/actions/utils/run@v0.1
2
  with:
3
    organization_id: ***
4
    deployment: dummy-branch-trigger
5
    location_name: test-location
6
    job_name: clone_prod
7
    repository_name: __repository__
8
    tags_json: {}
9
    config_json: {}
10
  env:
11
    DAGSTER_CLOUD_URL: 
12
    DAGSTER_CLOUD_API_TOKEN: ***
Ah, just seen it is optional, never mind
p
If you are using a code location with a defined repository (the old 
@repository
definition), then you use that name… if you are using 
Definitions
, you should use 
___repository___
m
Yeah, we're using 
Definitions
, so isn't that
Full error
graphql.txt
p
Sorry, I’m still looking into what’s going on here. Let me investigate more and get back to you.
d
Hey Matt - I think the problem here is the deployment parameter. You'll want that to be the output of a previous 
dagster-io/dagster-cloud-action/actions/utils/deploy
step (not the branch name), something like this:
Copy code
deployment: ${{ steps.deploy.outputs.deployment }}
m
Ahh, interesting. I'll try that out
I guess an extension to this point is that the reason I decoupled it in the first place is I didn't want to have to do the full build/deploy process upstream of the database operations when I was just closing the branch
As you then spend ~5 minutes building/deploying an image you never intend to use
d
That makes sense - this is a bit involved, but I believe this CLI command will also output the branch deployment ID to use within the given branch... we should probably package that up as an action that you can call:
Copy code
dagster-cloud ci branch-deployment <folder with your dagster_cloud.yaml in it>
m
Probably a bit involved for the bandwidth we have at the moment, but will definitely revisit if/when this is made It would be good to add in a "close branch deployment" action which is also decoupled from the image build, as once we close the PR we don't want the branch deployment on dagster cloud anymore. The ideal flow would be • parse workspace • if PR is not closed ◦ build image ◦ clone production DB • else ◦ drop cloned DB ◦ close branch deployment
p
Thanks for this bit of feedback. I think we can get something like that up in the action repo. I’ll ping you when we have something for you to try out.
❤️ 1
6 Views
Open in Slack
PreviousNext
Powered by