Hi team we have a dagster job stuck in starting phase We see

Question

Hi team we have a dagster job stuck in starting phase We see ` K8sRunLauncher Kubernetes run worker job created` but then nothing else after a day I couldn t find the job pod created in our k8s cluster though Wondering if you know what might have caused this and how to debug please

Hebo Yang · Answer

We are on 0 14 5 Full log

daniel · Answer

Hi Hebo if the pod isn t there what about the k8s job Does kubectl get jobs | grep lt job name gt show anything

Hebo Yang · Answer

Thanks Daniel Yes the job exists with Pods Statuses 0 Running 0 Succeeded 1 Failed

daniel · Answer

If you describe the job are there any clues

Hebo Yang · Answer

other than that `Pods Statuses 0 Running 0 Succeeded 1 Failed` it does t have much info

Hebo Yang · Answer

let me also check with our compute team

assaf · Answer

I m having the same issue Recently upgraded to 0 14 19 running on EKS This problem appeared only after the upgrade but I m not sure it s related

assaf · Answer

Here s the output when describing the K8s job

assaf · Answer

I launched a new instance of the same run from the Launchpad and now I see the pod actually gets scheduled but fails immediately on launch I don t know what to make of that stack trace

assaf · Answer

now I see this bad import was removed in

assaf · Answer

~I ll upgrade the dagster dependency in my user code image and report back ~ I see that my image is actually running dagster 0 13 18 explaining why that happened

daniel · Answer

Hey assaf that looks like a mismatch between your dagster postgres package and your dagster package yeah There should be a pin that keeps them in lockstep but it looks like that might not be getting respected