https://dagster.io/ logo
#ask-community
Title
# ask-community
c

Chaitya

05/25/2023, 11:42 PM
hey folks, is it possible to load an asset via
load_asset_value
with a different IO manager than when it was written out?
👀 1
o

owen

05/26/2023, 5:22 PM
hi @Chaitya! this is not currently possible -- mind sharing a bit more about your use case here? one pattern we've recommended in the past is to have a single IOManager whose load_input function branches based on the
dagster_type
of the input it's loading the asset into. In this case, you could pass in a
dagster_type
argument to
load_asset_value
for example, you might have an asset stored as a parquet file somewhere, but you can either load that into a spark dataframe or a pandas dataframe.
l

Lucas Santiago Cardoso

05/26/2023, 5:52 PM
hey @owen, can you illustrate more about this example: one asset read and writes with pandas, but the downstream asset works with Spark?
c

Chaitya

05/26/2023, 8:29 PM
@owen Specifically in my upstream IO manager, I output stuff using a directory using a dagster
run_id
prefix. Downstream, i'm loading my asset in a separate jupyter notebook that has no context about the dagster run ID... as a result, the IO manager has no idea how to load the asset (because there's no run context within
load_asset_value
. I would like to avoid making the IO manager configurable (and hardcoding the run ID in the run config). The reason i want to specify a different IO manager is that I'd like to make a specific IO manager that I can use in this case so that I don't have to make my original IO manager configurable by run id.
3 Views