19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Channels
#announcements
Title
k
Ken
03/20/2020, 12:59 AMHello Guys,
I am trying to run my pipelines in a Kubernetes Clusters, from my local machine, using the run_launcher "dagster_k8s.launcher"
I am seeing this error:
Copy code
dagster.core.errors.DagsterInvalidConfigError: Errors whilst loading configuration for {'service_account_name': <class 'str'>, 'job_image': <class 'str'>, 'in
stance_config_map': <class 'str'>, 'dagster_home': <class 'str'>, 'image_pull_secrets': Field(<dagster.config.config_type.Noneable object at 0x7f1646c88250>,
default=@, is_required=False), 'image_pull_policy': Field(<dagster.config.config_type.String object at 0x7f164f18ce90>, default=Always, is_required=False), 'j
ob_namespace': <class 'str'>, 'env_config_maps': Field(<dagster.config.config_type.Noneable object at 0x7f162684a910>, default=@, is_required=False), 'env_sec
rets': Field(<dagster.config.config_type.Noneable object at 0x7f162684ab10>, default=@, is_required=False)}.
Error 1: Fields "['kubeconfig_file', 'load_kubeconfig']" are not defined at document config root. Available fields: "['dagster_home', 'env_config_maps', '
env_secrets', 'image_pull_policy', 'image_pull_secrets', 'instance_config_map', 'job_image', 'job_namespace', 'service_account_name']"kubeconfig_fileload_kubeconfigMy Dagster.yaml file is as follow:
Copy code
run_launcher:
module: dagster_k8s.launcher
class: K8sRunLauncher
config:
service_account_name: "environments/tester-imagery-service-account.json"
job_image: "<http://gcr.io/tester-gcr-registry/dagster_demo_pipeline_image:v1|gcr.io/tester-gcr-registry/dagster_demo_pipeline_image:v1>"
instance_config_map: "tester-dagster-demo"
job_namespace: dagster_demo_pipeline_namespace
dagster_home: "/tmp"
load_kubeconfig: True
kubeconfig_file: "environments/kube.config"dagster pipeline execute -f ./pipelines/main_pipeline.py -n process_image_pipeline -d cloud -p cloudReference for where I get the configuration parameters for k8s launcher:
...github/dagster/python_modules/libraries/dagster-k8s/dagster_k8s/launcher.py ( line 57 )
a
alex
03/20/2020, 1:07 AMthe actual config schema is down on line 124
i think if you remove ‘kubeconfig_file’ and ‘load_kubeconfig’ from your yaml it should work
as per the error message piece
Error 1: Fields “[‘kubeconfig_file’, ‘load_kubeconfig’]” are not defined at document config root. Available fields: “[‘dagster_home’, ‘env_config_maps’, 'env_secrets’, ‘image_pull_policy’, ‘image_pull_secrets’, ‘instance_config_map’, ‘job_image’, ‘job_namespace’, ‘service_account_name’]”
it is indeed odd that the config schema is out of sync with the constructor you are looking at
cc @nate / @max
k
Ken
03/20/2020, 1:13 AMThanks Alex!
I did start out with not having those 2 lines.
But, I currently want to test run my local pipeline in Kubernetes clusters.
And without those 2 lines, the run_launcher seems to think it is being run in a Kubernetes environment,
and it tries to seek for Service Host/Port env variables ( which my local environment does not have )
Error message when kube_config is not used:
Copy code
File ".../lib/python3.7/site-packages/kubernetes/config/incluster_config.py", line 51, in _load_config
raise ConfigException("Service host/port is not set.")
kubernetes.config.config_exception.ConfigException: Service host/port is not set.a
alex
03/20/2020, 1:17 AMmakes sense, let me throw a diff up to add those two fields to the config
seems like it was just an mistake on our end, apologies for the thrash
k
Ken
03/20/2020, 1:19 AMThat will be awesome
No sweat! It happens. I look forward to the update!
a
alex
03/20/2020, 1:22 AMif you have the repo checked out locally you could
pip install -e python_modules/libraries/dagster-k8s👍 1
n
nate
03/20/2020, 1:24 AMyes, a miss on our part - and thanks Alex for the quick diff
k
Ken
03/20/2020, 1:25 AMCool, I will test it out now!
I also appreciate the quick diff, thanks Alex!
Update: It's working!!
🎉 1
Hello @alex,
Thanks again for your code-diff yesterday.
After applying that patch, unfortunately, I am seeing another issue:
The pipeline will run error-free in the local machine, rather than deploying a job in the Kubernetes Clusters.
Context of My Project:
I am running a small pipeline. Locally, it works, with an empty dagster.yaml file.
The pipeline runs in my local machine.
I now want to:
• Local Machine: perform
dagster pipeline execute -f ./pipelines/main_pipeline.py -n process_image_pipeline -d cloud -p cloud Copy code
run_launcher:
module: dagster_k8s.launcher
class: K8sRunLauncher
config:
service_account_name: "environments/tester-imagery-service-account.json"
job_image: "<http://gcr.io/tester-gcr-registry/dagster_demo_pipeline_image:v1|gcr.io/tester-gcr-registry/dagster_demo_pipeline_image:v1>"
instance_config_map: "tester-dagster-demo"
job_namespace: dagster_demo_pipeline_namespace
dagster_home: "/tmp"
load_kubeconfig: True
kubeconfig_file: "environments/kube.config"
telemetry:
enabled: true
instance_id: 8a67c01a-34c5-4161-ab60-cc4c1de95a86m
max
03/21/2020, 1:33 AMHi @Ken,
dagster pipeline executek
Ken
03/21/2020, 1:36 AMThanks for the quick response, Max!
I look forward to seeing what you guys come up with.
m
max
03/21/2020, 2:35 AMok, sketch is up at https://dagster.phacility.com/D2303
k
Ken
03/21/2020, 6:38 AMThanks Max, It looks awesome!
I do not know how to leave comments in that Web Service, so I hope I could leave it here:
1. >>>
File: python_modules/dagster/dagster/cli/pipeline.py
Line: 350
New Change: ( added )
Copy code
instructions=get_pipeline_instructions('execute') Copy code
instructions=get_pipeline_instructions('launch')dagster pipeline launch Copy code
group.add_command(pipeline_launch_command)dagster pipeline launchError: No such command 'launch'.dagster pipeline launchGKE Workload has initiated, using the new launch command!
n
nate
03/21/2020, 3:41 PMgreat - thanks for the feedback @Ken I left your notes in the diff for Max—glad this is working!