https://dagster.io/ logo
#dagster-kubernetes
Title
# dagster-kubernetes
s

Stephen Bailey

04/06/2022, 5:09 PM
what happens if i run a
helm upgrade
while jobs are running? im assuming configuration gets applied in a safe way?
j

johann

04/06/2022, 5:09 PM
Running jobs should be unaffected- they don’t pick up configmap/secret changes
👍 1
b

ba

04/06/2022, 7:37 PM
This doesn't necessarily apply to dagster jobs since we set
restartPolicy: Never
, but useful to know in general for troubleshooting: • if the configmap/secret is mounted as a volume, the file on the volume is updated, but if the job only reads it once at startup, it shouldn't affect the running job. • if the configmap/secret is mounted as an environment variable, the environment variable will not change • caveat for both: if the job restarts(job spec
restartPolicy
is not
Never
) or gets rescheduled onto another node it will probably pick up the change. It is very unlikely that a job with
restartPolicy: Never
is rescheduled, but kubernetes cannot guarantee this:
Note that even if you specify
.spec.parallelism = 1
and
.spec.completions = 1
and
.spec.template.spec.restartPolicy = "Never"
, the same program may sometimes be started twice.
s

Stephen Bailey

04/06/2022, 8:00 PM
awesome, thanks @ba
3 Views