https://dagster.io/ logo
Powered by
Title
j

Joel Olazagasti

04/26/2023, 3:29 PM
I'm trying to have our serverless branch deploy (old docker version) run some jobs after it builds as part of our CICD, following the template here. I'm struggling to correctly get the 
deployment
output needed for the run step. From what I can tell it looks like the serverless deploy composite action doesn't actually surface the output from it's 
deploy
step. Is that correct? Is there a work-around for this, or is running jobs in the serverless branch deploy just not possible?
s

Shalabh Chaturvedi

04/26/2023, 3:32 PM
Hi Joel - you are correct this is a bug. We are currently working on a fix and hope to have it released by end of week. In the meantime you can use the 
get_branch_deployment
action to get the current deployment name.
j

Joel Olazagasti

04/26/2023, 4:43 PM
Hi Shalabh! Thanks, that worked to get me past the blocker with the deployment. I'm still receiving the same error as I was before 
Exception: Unable to launch run: {'data': {'launchRun': {'__typename':206
'PipelineNotFoundError'}}}
At this point I'm not sure where it's failing. I can post the full traceback in the action if helpful, or are there are any other common pitfalls I should be aware of?
s

Shalabh Chaturvedi

04/26/2023, 5:06 PM
Can you please share your yaml that launches the jobs? Also are you able to log the deployment name so we know that works?
j

Joel Olazagasti

04/26/2023, 5:08 PM
name: Serverless Branch Deployments
on:
  pull_request:
    types: [opened, synchronize, reopened, closed]
concurrency:
  # Cancel in-progress runs on same branch
  group: ${{ github.ref }}
  cancel-in-progress: true
env:
  DAGSTER_CLOUD_URL: "<http://sealed.dagster.cloud>"
  DAGSTER_CLOUD_API_TOKEN: ${{ secrets.DAGSTER_CLOUD_API_TOKEN }}
  # ENABLE_FAST_DEPLOYS: 'true'

jobs:
  dagster_cloud_default_deploy:
    name: Dagster Serverless Deploy
    runs-on: ubuntu-latest
    outputs:
      build_info: ${{ steps.parse-workspace.outputs.build_info }}
    
    steps:
      - name: Parse cloud workspace
        if: env.ENABLE_FAST_DEPLOYS != 'true'
        id: parse-workspace
        uses: dagster-io/dagster-cloud-action/actions/utils/parse_workspace@v0.1
        with:
          dagster_cloud_file: dagster_cloud.yaml

      - name: Checkout
        if: env.ENABLE_FAST_DEPLOYS == 'true'
        uses: actions/checkout@v3
        with:
          ref: ${{ github.head_ref }}
          path: project-repo
          
      - name: Build and deploy Python executable
        if: env.ENABLE_FAST_DEPLOYS == 'true'
        uses: dagster-io/dagster-cloud-action/actions/build_deploy_python_executable@pex-v0.1
        with:
          dagster_cloud_file: "$GITHUB_WORKSPACE/project-repo/dagster_cloud.yaml"
          build_output_dir: "$GITHUB_WORKSPACE/build"
          python_version: "3.10"
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  dagster_cloud_docker_deploy:
    name: Dagster Serverless Docker Deploy
    runs-on: ubuntu-latest
    if: needs.dagster_cloud_default_deploy.outputs.build_info
    needs: dagster_cloud_default_deploy
    strategy:
      fail-fast: false
      matrix:
        location: ${{ fromJSON(needs.dagster_cloud_default_deploy.outputs.build_info) }}
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          ref: ${{ github.head_ref }}
      - name: Build and deploy to Dagster Cloud serverless
        id: build
        uses: dagster-io/dagster-cloud-action/actions/serverless_branch_deploy@v0.1
        with:
          dagster_cloud_api_token: ${{ secrets.DAGSTER_CLOUD_API_TOKEN }}
          location: ${{ toJson(matrix.location) }}
          # Uncomment to pass through Github Action secrets as a JSON string of key-value pairs
          # env_vars: ${{ toJson(secrets) }}
          organization_id: ${{ secrets.ORGANIZATION_ID }}
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      - id: get-branch
        name: Get Branch Name
        uses: dagster-io/dagster-cloud-action/actions/utils/get_branch_deployment@v0.1
        with:
          organization_id: ${{ secrets.ORGANIZATION_ID }}
      
      - name: Print get-branch outputs
        shell: bash
        run: echo "${{ toJson(steps.get-branch.outputs)}}"

      - name: Run All DBT Models
        if: github.event.action != 'closed'
        uses: dagster-io/dagster-cloud-action/actions/utils/run@v0.1
        with:
          location_name: ${{ toJson(matrix.location) }}
          deployment: ${{ steps.get-branch.outputs.deployment }}
          job_name: run_all_dbt_job
        env:
          DAGSTER_CLOUD_URL: ${{ secrets.DAGSTER_CLOUD_URL }}
          DAGSTER_CLOUD_API_TOKEN: ${{ secrets.DAGSTER_CLOUD_API_TOKEN}}
I did confirm I was getting the branch name correctly. I tested out running the run job with my 
organization_id
instead of the cloud_url, and saw the branch correctly built into the request URL
s

Shalabh Chaturvedi

04/26/2023, 5:58 PM
Ah ok. So using 
organization_id
resolved the issue, IIUC?
j

Joel Olazagasti

04/26/2023, 6:04 PM
No, sorry, it just gave me visibility to verify that the deployment input was being passed in correctly. So my issue is now somewhere else
s

Shalabh Chaturvedi

04/26/2023, 7:10 PM
Can you try adding 
organization_id
to the with: section here: 
uses: dagster-io/dagster-cloud-action/actions/utils/run@v0.1
        with:
And also removing the DAGSTER_CLOUD_URL env.
j

Joel Olazagasti

04/26/2023, 7:58 PM
Yep! That didn't work either. It does seem to be building the URL correctly in both cases (url or organization_id), if that helps narrow down the source of the issue.
The fact that Github Actions masks all of the secrets inputs in the logs certainly makes it hard to troubleshoot
s

Shalabh Chaturvedi

04/26/2023, 9:07 PM
It might be useful to try using the 
dagster-cloud
cli locally to invoke the job in the branch deployment to make sure the job exists and can be remotely launched. Example of using the cli 
dagster-cloud job launch -j run_all_dbt_job -l <location-name> -d <deployment-name-copied-from-url>
Only the api token is really sensitive and needs to be a secret. You can also try replacing the organization_id with a literal instead so it logs the clear text. If you retry and are able to share the debug logs I can look into them. Feel free to DM the logs if you want.
ah OK another suggsestion. I believe we should use 
location_name: ${{ matrix.location.name }}
instead of 
matrix.location
. Can you please try that?
j

Joel Olazagasti

04/26/2023, 9:19 PM
I'll push 
location.name
now and see! If not, I'll pick up with the CLI tomorrow
👍 1
s

Shalabh Chaturvedi

04/26/2023, 9:29 PM
please lmk how it goes. I tested with this and I am able to launch a job from the workflow.
j

Joel Olazagasti

04/26/2023, 9:50 PM
Still a no-go, although I do think that would have been a blocking issue as well. I see in past runs the full 
location
json object was being passed to 
location_name
. I'll tinker with the cloud CLI tomorrow and try to narrow down where the issue is and post back.
So I could successfully run the job locally, and I've found the difference from my local stacktrace and the github action stacktrace. You can see the URL kwarg is being populated with the organization and deployment name, but the explicit 
organization
and 
deployment
fields are not, whereas they are populated when running from a local CLI
Interestingly, I hardcoded 
organization_id
and 
location_name
, and the run succeeded. Pulling them from GH action variables does work in other steps, so I'm not quite sure why that might be
s

Shalabh Chaturvedi

04/27/2023, 5:54 PM
Here is my test code that works. Can you try just copy pasting and just replacing the job_name? 
- name: Run job
        uses: dagster-io/dagster-cloud-action/actions/utils/run@v0.1
        with:
          location_name: ${{ matrix.location.name }}
          deployment: ${{ steps.deploy.outputs.deployment }}
          job_name: all_assets_job
The only additional parameter you might need is repository_name.
Also you might need 
steps.get-branch
instead of 
steps.deploy
.
Also did hardcoding completely resolve your issue? It is weird that it didn't work when referencing the action variables. We now also deployed a new version of the deploy action so the original bug is now fixed. However you can continue to use the 
get_branch_deployment
action if you want.
2 Views
#dagster-cloudJoin Slack
Powered by