I’m using software defined assets and S3 pickle IO manager for storage. If I materialize assets together top to bottom in one go the run succeeds. But if I say materialize the first asset in a run, then materialize the second asset after that has finished, I get a failure

boto3 key not found

It looks like the storage of the assets for S3 each run makes a subdirectory and stores them in there, but when a new run kicks off it doesn’t look in the directory of the original run that produced the upstream asset.

hi @Eric Larson! I believe you should be using the

s3_pickle_asset_io_manager

instead of the

s3_pickle_io_manager

. The asset io manager stores the pickled files in locations that are not dependent on the run id

lol, yes I see mistake on my part, thanks for the quick reply