Hi, how would I go about adding config/resources to a sensor? thanks!

HI Oliver - unfortunately this isn't something that we currently support, but it's a common (and very reasonable) request that we'd like to support in the future. For now any configuration for a sensor needs to be in the sensor code (while supplying config for the pipeline that is launched by the sensor).

Ah ok, is there an issue tracking this

Here is one, yeah https://github.com/dagster-io/dagster/issues/3794

thanks. sorry, gonna hijack this thread with another query; I had a look into the sensor code and noticed that the consuming function for the sensor just lists the output of the sensor and returns this to the calling func here. This is a little unexpected based on my understanding of sensors (i.e I thought each yield would immediately launch a run). For my use case I was basically intending to use a sensor as a batching function for data from a streaming data source but I see that's probably not gonna work. The other way I can think of is to have another service running that consumes the stream and launches jobs with the graphql python client. Are there any other approaches I could consider?

You’re correct that sensors don’t themselves launch runs - like schedules, they produce instructions (config, tags, etc) that can be used by the dagster daemon process to launch runs. Sensors aren’t really designed to be invoked outside of the dagster daemon process - if you’re not interested in using that you may be better off setting up your own service that launches jobs in the way you described.

You’re correct that sensors don’t themselves launch runs

I understood this bit, my assumption was the instructions were sent to the daemon in real time at time of yielding rather than after the sensor has completely finished generating job instructions

Oh I see. They don’t stream in that way currently, that’s correct - the generator syntax is just for programming convenience, not to enable streaming or long-running sensors. @prha may have some thoughts here as well (possibly not until Monday though)

Yeah, just echoing daniel here. I think sensors were originally conceived as pretty quickly run functions checking external state. I think consuming a streaming data source is an interesting use case, but not one sensors were designed for.

IMO it is a little misleading to be using generators in this situation then. At least when I see a generator I'm assuming the consumer is not going to just list the output. Just my 2c anyway, keep up the good work 🙂