Jumping into a k8s based deployment going to drop some initi

Question

Jumping into a k8s based deployment going to drop some initial thoughts into a thread here

Travis Cline · Answer

1 Initially expected to be able to control the run launcher configuration via the execution key in the pipeline config object

Travis Cline · Answer

2 not having more dynamic configuration AFAICT so far prevents some desirable patterns such as per pipeline service account specification

Travis Cline · Answer

3 Is it true that the dagit UI can t launch pipeline runs that use run launcher configuration

alex · Answer

gt dagit UI can t launch pipeline runs there should be a drop down to select between `start` and `launch`

alex · Answer

< nate> for other notes

Travis Cline · Answer

Ah yes thanks < alex>

alex · Answer

we are working to remove that distinction in `0 8 0`

alex · Answer

it is confusing and goofy

nate · Answer

hey Travis on 1 definitely agree the split config between instance YAML and pipeline execution config is pretty rough right now so agree here for 2 are there other fields you d like to be able to configure per pipeline run or just service account

nate · Answer

cc < cat>

Travis Cline · Answer

definitely interested in per run resource requests limits this would allow auto scaling up when a pipeline run would use a GPU for example

Travis Cline · Answer

Supplying the image on a per pipeline is a likely requirement for many folks would enable distinct dependency management between pipelines

nate · Answer

hey Travis just following up here I ve updated our Celery K8s configs <https dagster phacility com D3192> so you should be able to configure these parameters on a per pipeline basis now

Travis Cline · Answer

badass

nate · Answer

let us know if you have other feedback we re actively working on improving this integration