Mykola Palamarchuk
03/29/2022, 2:50 PMmy_resource = context.resources.my_resource
my_bucket = my_resource['s3_bucket']
Is there a way to pass the value to s3_pickle_io_manager
?prha
03/29/2022, 4:28 PMs3_pickle_io_manager
already has access to the s3 bucket… (it specifies the bucket in config, and specifies the s3
required resource key):
https://docs.dagster.io/_apidocs/libraries/dagster-aws#dagster_aws.s3.s3_pickle_io_manager
Is there some workflow you’re trying to enable where you won’t have access to that?@io_manager(required_resource_keys={'s3', 'my_s3_bucket_providing_resource'})
def my_custom_s3_io_manager(context):
bucket_name = context.resources.my_s3_bucket_providing_resource['s3_bucket']
return PickledObjectS3IOManager(bucket_name, context.resources.s3)
Mykola Palamarchuk
03/29/2022, 4:41 PMprha
03/29/2022, 4:45 PMPickledObjectS3IOManager
into the public API of dagster_aws
though.PickledObjectS3IOManager
from dagster_aws
moduleDagster Bot
03/29/2022, 4:46 PM