`k8s_job_executor` throws error `__new__() got an ...
# deployment-kubernetes
l
k8s_job_executor
throws error
__new__() got an unexpected keyword argument 'value'
after hitting Launch Run Dagster version
0.14.3
----------------- context: I have a single-op job that works perfectly with K8s run launcher. I define requirements in
@job()
However, Iā€™m moving my resource requirements into
@op()
and changing to
@job(executor_def=k8s_job_executor)
so that I can launch dynamic ops on separate pods with those requirements.
āœ… 1
Updated my dagster version to latest and now I get a different error (only thing I changed was to pass the
0.14.12
into the DAGSTER_VERSION arg in my Dockerfile)
docker build --platform linux/amd64 -f Dockerfile -t $IMAGE_URI ./ --build-arg DAGSTER_VERSION=0.14.12
my imports:
got the same error as
0.14.12
when I tried with
0.14.9
d
Hi Liezl - try tagging your base image with the dagster version you want to use instead of latest: https://hub.docker.com/r/dagster/user-code-example/tags I think by using latest you are ending up with a different dagster-cloud version than the other dagster libraries and they're supposed to stay in lockstep
šŸ‘ 1
l
k8s executor works now but Iā€™m getting an input not found error when using k8s executor + dynamic op with 1 extra arg (
dynamic_args: Dict[str, Any]
) dagster version
0.14.12
āœ… 1
d
This looks to me like you want to configure a different IO manager that persists the output of ops - the default filesystem io manager isn't going to work when each op is running in its own pod. In GCP there's an example that sets up a GCS IO manager here: https://docs.dagster.io/deployment/guides/gcp#using-gcs-for-io-management And more about IO managers in general here: https://docs.dagster.io/concepts/io-management/io-managers
šŸ‘ 1
āœ… 1