Michel Rouly
08/18/2022, 4:46 PMingest_redash
) and a few Sensors (e.g. redash_sensor
). When we create our Repository, we apply Graphs to Resources at the last minute to return Jobs.
We can have Sensors track Jobs if we use the Sensor factory pattern, but this makes our Repository definition pretty awkward -- you can no longer just loop over all the Graphs and apply them to Resources. You need to pull out specific Graphs, convert them to Jobs, and pass them into specific Sensors.
We'd like our Sensors to track the Graphs (@sensor
takes a GraphDefinition
for its job=
parameter) but when we do this, we run into the following error:
UserWarning: Error loading repository location repo.py:datahub:dagster._core.errors.DagsterInvalidDefinitionError: sensor 'redash_sensor' targets graph 'ingest_redash', but a different job with the same name was provided. Disambiguate between these by providing a separate name to one of them.
sandy
08/18/2022, 4:47 PMMichel Rouly
08/18/2022, 4:50 PMpipeline_name
, is that the same thing? Or maybe I'm on an old version of Dagster?pipeline_name
works great for us, but it's marked as deprecated. If job_name
works similarly, that sounds like a way forwards.