The Dagster AssetReconciliationSensor does this nifty thing where it yields a RunRequest but doesn't create a global job definition (AFAIK). Is it is simple as returning the following rather than a pre-defined job?

return RunRequest(asset_selection=list(asset_keys))

yep that'll work! as long as those assets are possible to run within the same job (so for example, a list containing two assets, one hourly partitioned and the other daily partitioned would result in an error)

do i need to specify the mega job in the

@multi_asset_sensor(job=...)

syntax?

I believe you should be able to just leave the job out entirely

I believe you should be able to just leave the job out entirely

it's worth trying, but I suspect this does not work the asset reconciliation sensor uses the

asset_selection

argument on

@sensor

to target a bunch of assets, and then the RunRequest subselects within that set.

@multi_asset_sensor

does not yet support that argument (confusingly, the

asset_selection

argument on

@multi_asset_sensor

refers to the assets to monitor). I am going to try to remedy this. as a workaround in the meantime, you could create a job that targets all your assets and point at it in the multi-asset sensor.

ah that's right, good catch

been playing with this this week. two gotches to add -- 1.

RunRequest(asset_selection=..)

does not accept

AssetSelection

as an accepted type:

dagster._check.ParameterCheckError: Param "asset_selection" is not one of ['Sequence'].

2. The "mega job", which is best most easily defined as

define_asset_job(selection=AssetSeleciton.all())

, fails if there are different partitions. I can see the tehnical complexity here and I finially understand why there were those ASSET_JOB_0 - 5 showing up in the reconicilation sensor. But, a pain nonetheless