https://dagster.io/ logo
#ask-community
Title
# ask-community
d

Danny Steffy

02/08/2023, 12:34 AM
trying to use sensors and schedules to start kicking off materialization of certain assets and running certain jobs. Currently getting this error on every sensor/schedule:
Copy code
Exception: 'network' cannot be used in 'container_kwargs'. Use the 'networks' config key instead.
  File "/usr/local/lib/python3.10/site-packages/dagster/_daemon/run_coordinator/queued_run_coordinator_daemon.py", line 335, in _dequeue_run
    instance.run_launcher.launch_run(
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/instance/__init__.py", line 660, in run_launcher
    launcher = cast(InstanceRef, self._ref).run_launcher
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/instance/ref.py", line 491, in run_launcher
    return self.run_launcher_data.rehydrate() if self.run_launcher_data else None
  File "/usr/local/lib/python3.10/site-packages/dagster/_serdes/config_class.py", line 101, in rehydrate
    return klass.from_config_value(self, check.not_none(result.value))
  File "/usr/local/lib/python3.10/site-packages/dagster_docker/docker_run_launcher.py", line 66, in from_config_value
    return DockerRunLauncher(inst_data=inst_data, **config_value)
  File "/usr/local/lib/python3.10/site-packages/dagster_docker/docker_run_launcher.py", line 41, in __init__
    validate_docker_config(network, networks, container_kwargs)
  File "/usr/local/lib/python3.10/site-packages/dagster_docker/utils.py", line 47, in validate_docker_config
    raise Exception(
materializing the assets manually through dagit works fine, it's only trying to schedule these jobs using sensors and schedules. We're using the DefaultRunCoordinator and DefaultRunLauncher. Am I missing some other sort of configuration here?
🤖 1
actually I just discovered we have a different configuration in the daemon, so that's likely the problem. Do we need the daemon run_launcher and run_coordinator to match what's in the dagit container?
d

daniel

02/08/2023, 12:50 AM
They don’t necessarily have to be in sync, but we generally recommend it
In part to avoid confusing situations like this where you’re not sure what the configuration actually is
d

Danny Steffy

02/08/2023, 12:51 AM
👍 I'll definitely get those in sync. I think we wants to use the DockerRunLauncher moving forward
d

daniel

02/08/2023, 12:53 AM
Schedules and sensors are always launched from the daemon but launching runs from dagit are launched from the dagit process (unless you’re using the run queue, in which case they will also be launched from the daemon) - which I think matches what you’re seeing
d

Danny Steffy

02/08/2023, 12:55 AM
so if you specify the QueuedRunCoordinator in the dagit config, it'll hand over all jobs from the dagit process to the daemon?
d

daniel

02/08/2023, 12:56 AM
That’s right
d

Danny Steffy

02/08/2023, 2:28 AM
Thank you!
2 Views