https://dagster.io/ logo
#ask-community
Title
# ask-community
c

Carlos Pega

01/20/2023, 2:14 PM
can sensors run in parallel? If a sensor has a timeout for connecting to kafka for instance it blocks all the other sensors executions
i saw there is something called sensors threads pool, but i can't find documentation nor examples. Any clue?
i found this feature-request. Is therea a workaround for running multiple sensors in kubernetes at the same time (in parallel)??
this seems to work:
Copy code
sensors:
  num_workers: 16
  use_threads: true
50 kafka request messages (with a dummy op execution) + slack succeeded notification in 6 minutes vs 11 minutes without setting the
use_threads
d

daniel

01/20/2023, 3:23 PM
c

Carlos Pega

01/20/2023, 3:23 PM
thanks! @daniel
2 Views