Chaitya
05/25/2023, 11:42 PMload_asset_value
with a different IO manager than when it was written out?owen
05/26/2023, 5:22 PMdagster_type
of the input it's loading the asset into. In this case, you could pass in a dagster_type
argument to load_asset_value
for example, you might have an asset stored as a parquet file somewhere, but you can either load that into a spark dataframe or a pandas dataframe.Lucas Santiago Cardoso
05/26/2023, 5:52 PMowen
05/26/2023, 6:06 PMChaitya
05/26/2023, 8:29 PMrun_id
prefix. Downstream, i'm loading my asset in a separate jupyter notebook that has no context about the dagster run ID... as a result, the IO manager has no idea how to load the asset (because there's no run context within load_asset_value
. I would like to avoid making the IO manager configurable (and hardcoding the run ID in the run config).
The reason i want to specify a different IO manager is that I'd like to make a specific IO manager that I can use in this case so that I don't have to make my original IO manager configurable by run id.