Hi Team It seems like we re mostly locked out of our Dagster dagster #dagster-plus

Hi Team, It seems like we're mostly locked out of...

Greg Bernard

07/19/2023, 3:21 PM

Hi Team, It seems like we're mostly locked out of our Dagster Serverless deployment for unknown reasons. Most-recent deploy attempt error:

Copy code

dagster_cloud.workspace.ecs.client.EcsServiceError: ECS service failed because task arn:aws:ecs:us-west-2:657821118200:task/serverless-agent-e0b8b667-f896-3f99-aa80-c73bf3fe20d9-Cluster/7f6c06e1e5944ceb96c724fe09c72329 failed: Essential container in task exited

For some reasons both prod runs and branch runs are now failing for most tasks we try to run. We haven't made changes to our prod branch since yesterday when runs seemed to be running without issue. A bit frustrated because not a single error we've received has been helpful in diagnosing what the heck is going wrong.

Greg Bernard

07/19/2023, 3:22 PM

Our DBT runs in our main and branch deployments fail with

Copy code

dagster_dbt.errors.DagsterDbtCliRuntimeError: The dbt CLI process failed with exit code 1. Check the compute logs for the full information about the error.

I don't think we have access to 'compute' logs on serverless?

Greg Bernard

07/19/2023, 3:22 PM

Task logs from most recent failed deploy

Untitled

Greg Bernard

07/19/2023, 3:25 PM

Everything actually seemed to be deploying earlier this morning without issue as well. It's only more recent deploys that just completely failed.

Shalabh Chaturvedi

07/19/2023, 4:03 PM

Hi Greg, the compute logs are in a tab right next to the event logs. You can get them with this icon: I saw some errors that might be relevant that I'll DM you.

Open in Slack

Previous Next