I'm in a tricky position of trying to bridge the divide between assets and ops in a pipeline that I'm building. I've got an externally materialized asset (a periodic file drop in GCS) that I wanted to treat as a partitioned asset via a sensor. The downstream operation needs to unpack the downloaded file and generate a downstream set of assets that are dynamically partitioned based on what is included in the archive. Unfortunately there isn't yet a way to treat the downstream outputs as assets, so I have to write it as an op with in-line asset materializations. The tricky piece is that I'm trying to use an IO Manager to load the external asset into the op, but I can't get a handle of the asset key in the IO Manager. All of the relevant code is in my PR here https://github.com/mitodl/ol-data-platform/pull/991/files

Hey Tobias - one thing that might be helpful here is the ability to have an op-based job include an input that comes from an asset: https://docs.dagster.io/guides/dagster/how-assets-relate-to-ops-and-graphs#op-graphs-that-read-from-an-asset Another option would be to pass config to your op-based job that specifies which asset it depends on

Would that help with populating the asset key in the

InputContext

for the

InputManager

?

Yes, it should