Hello our main deployment is encountering errors `FileNotFou

Question

Hello our main deployment is encountering errors `FileNotFoundError Errno 2 No such file or directory dagster assets dbt python repository py ` we observed this error first in several jobs starting between 11 02 and 11 25am pacific time We then attempted a re deploy and now this is happening for the entire deployment The deployment didn t change we last deployed this morning around 9 30am pacific time

Leo Qin · Answer

We recently did a branch deployment and i notice that the id associated to the agent is the same as the one in our main deployment

daniel · Answer

Hey Leo i m wondering if there might be a problem in your CI CD causing the image tag to be applied incorrectly I see that the tag being associated with your jobs and code locations is something like ```dagster image <http 657821118200 dkr ecr us west 2 amazonaws com serverless agent lt REDACTED gt None|657821118200 dkr ecr us west 2 amazonaws com serverless agent lt REDACTED gt None>``` the tag being None is definitely unexpected it should be something that uniquely identifies a particular commit

daniel · Answer

Could you possibly share your github action yaml Wondering if something is misconfigured there

daniel · Answer

The reason I think that could be related is that if the tag for everything is None it would be very easy for say a branch deployment to mess up the prod deployment since they both have the same incorrect tag

Leo Qin · Answer

we re using `dagster cloud serverless deploy` to build the image

daniel · Answer

Got it do you have a sample CLI command that you re running and do you know what version of the `dagster cloud` package you have installed in the environment where its happening

Leo Qin · Answer

` dagster cloud==1 3 3 `

Leo Qin · Answer

```dagster cloud serverless deploy location name dagster working directory dagster agent timeout 600 python file dagster assets dbt python repository py deployment $BRANCH DEPLOYMENT NAME base image $DOCKER IMAGE $GIT SHA ```

daniel · Answer

How about for that non branch deployment case

Leo Qin · Answer

same command for both that s the version of the main deployment

daniel · Answer

ah ok I saw BRANCH DEPLOYMENT NAME

daniel · Answer

ok will do a bit of digging and report back i would think that redeploying in an environment where that file exists should fix it

daniel · Answer

and that the event that caused this might have been a bad push to the None tag until we sort that out

Leo Qin · Answer

the branch deploy was also on 1 3 3

daniel · Answer

I have a possible lead I d be curious if the tag goes back to what it was before if you take dagster cloud back to 1 3 2

daniel · Answer

which is something I could check on my side once you do a deploy

Leo Qin · Answer

I m re deploying right now just to get back up and running but I see that as of 6 days ago the tag was `prod dagster lt tag gt `

Leo Qin · Answer

and we were on `1 2 2`

daniel · Answer

Or actually here s something that won t require a version change try adding ``` image $BRANCH DEPLOYMENT NAME dagster $GIT SHA``` don t forget the ` ` after the ` base image` line

daniel · Answer

if that fixes it we ll get a real fix out shortly

Leo Qin · Answer

i m doing a re deploy to get back and running but will look into this later today

Leo Qin · Answer

the ` image` argument is distinct from and compatible with the ` base image` argument right

daniel · Answer

that s right image tag would be a better name

daniel · Answer

We just merged in a real fix for this that should go out in 1 3 5 this week thanks for reporting

daniel · Answer

and sorry for the trouble