The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

can sensors run in parallel? If a sensor has a timeout for connecting to kafka for instance it blocks all the other sensors executions

i saw there is something called sensors threads pool, but i can't find documentation nor examples. Any clue?

i found this <https://github.com/dagster-io/dagster/issues/4333|feature-request>. Is therea a workaround for running multiple sensors in kubernetes at the same time (in parallel)??

this seems to work:

```sensors:
  num_workers: 16
  use_threads: true```

50 kafka request messages (with a dummy op execution) + slack succeeded notification in 6 minutes vs 11 minutes without setting the `use_threads`

Very brief docs here! <https://docs.dagster.io/deployment/dagster-instance#sensor-evaluation>