I’m having trouble passing some information between solids.

solid_A

fetches some data from an API, and uses an IO Manager that persists the data to a JSON file in GCS. The path at which to store the file is specified as config for IO Manager as described in the docs here.

solid_B

then loads that file from GCS to BigQuery, but I’m not sure how to pass the GCS URI between the two solids. I can’t have

solid_B

use the same IO Manager and have the IO Manager’s

load_input

handle locating the file because I don’t want to “load” it to perform computation. Is there a way for

solid_A

to access that IO manager config to return it as on output? Also, is there a better way I could model this series of tasks?

Hi @Prratek Ramchandani, one way to handle this would be to define your

load_input

function to simply return the URI (

context.upstream_output.config["<whatever you call the uri>"]

), instead of actually loading the data from GCS. So the protocol for this IOManager would be: on output, store data into the location defined by the config, and on input, simply retrieve the path that this data was stored in.

that’s a good idea, thanks!