https://dagster.io/ logo
d

Deveshi

03/23/2021, 12:16 PM
Hi, we have multiple aws environments and I am hosting my Dagster on a centralised EC2 instance. Does the s3_resource support 'Profiles'? Say I have two pipelines and they need to upload files to S3 buckets in separate env, using different credentials. As per the documentation, it says s3_resource is based on boto3.resource, that picks up default profile https://docs.dagster.io/_apidocs/libraries/dagster-aws#dagster_aws.s3.s3_resource
a

alex

03/23/2021, 3:27 PM
I dont think so the
s3_resource
is a basically just a wrapper over
boto
- I think I would recommend making your own to support the features you need, and ignore the ones you don’t https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-aws/dagster_aws/s3/utils.py#L31-L50
alternatively you could send a PR if supporting it would be inline with the other config options that are there
d

Deveshi

03/23/2021, 3:28 PM
okay, cool
2 Views