https://dagster.io/ logo
#ask-community
Title
# ask-community
a

Anoop Sharma

06/11/2022, 4:56 PM
I am getting an error while deploying the dagster setup using k8s. It has 4 services - dagster-dagit, dagster-daemon, dagster-grpc and postgres. All the services/pods are running fine and I am able to see the dagit ui properly along with the repositories and jobs/ops. However, when I execute a job using launchpad, I am getting the following error
Copy code
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))
  File "/usr/local/lib/python3.7/site-packages/dagster/core/instance/__init__.py", line 1784, in launch_run
    self._run_launcher.launch_run(LaunchRunContext(pipeline_run=run, workspace=workspace))
  File "/usr/local/lib/python3.7/site-packages/dagster_docker/docker_run_launcher.py", line 152, in launch_run
    self._launch_container_with_command(run, docker_image, command)
  File "/usr/local/lib/python3.7/site-packages/dagster_docker/docker_run_launcher.py", line 97, in _launch_container_with_command
    client = self._get_client(container_context)
  File "/usr/local/lib/python3.7/site-packages/dagster_docker/docker_run_launcher.py", line 72, in _get_client
    client = docker.client.from_env()
  File "/usr/local/lib/python3.7/site-packages/docker/client.py", line 101, in from_env
    **kwargs_from_env(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/client.py", line 45, in __init__
    self.api = APIClient(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 222, in _retrieve_server_version
    f'Error while fetching server API version: {e}'
I have already added the volume mapping for docker socket inside the dagster-daemon k8s config yaml file:
Copy code
spec:
      containers:
          volumeMounts:
          - name: dockersock
            mountPath: /var/run/docker.sock
      volumes:
      - name: dockersock
        hostPath:
          path: /var/run/docker.sock
Have also added the same inside dagster.yml run_launcher config:
Copy code
run_launcher:
  module: dagster_docker
  class: DockerRunLauncher
  config:
    env_vars:
      - DAGSTER_POSTGRES_USER
      - DAGSTER_POSTGRES_PASSWORD
      - DAGSTER_POSTGRES_DB
    network: docker_network
  container_kwargs:
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock
Still unable to get rid of the issue. Any suggestions? FYI...I am using kubernetes
Deployment
and
kubectl apply
to run the services.
As a workaround for now, I have used DinD image to host the docker.sock for the
dagster-daemon
container. Is there a better way to achieve this though?
d

daniel

06/12/2022, 12:40 AM
Hi anoop - the docker run launcher is not going to work well in Kubernetes - you likely want to use the k8srunlauncher. The recommended way to deploy dagster in Kubernetes is to use our helm chart which is described here: https://docs.dagster.io/deployment/guides/kubernetes/deploying-with-helm
14 Views