Adrian01/30/2021, 1:33 AM
config: s3_bucket: STORAGE_BUCKET endpoint: MINIO-SERVICE-NAME:PORT
mrdavidlaing01/31/2021, 10:16 AM
settings to work. (I couldn't; but I suspect that is related more to my companies blob storage device that dagster)
Adrian01/31/2021, 6:47 PM
mrdavidlaing02/02/2021, 8:55 PM
installed; Boto will use its certs rather than your system certs (at least on a Mac); potentially giving different behaviour when you curl an endpoint vs access it via Boto.
Simon Späti02/08/2021, 7:42 AM
in dagster_aws/s3/utils.py. I guess would be nice to have it build in as config. By the way, are you using spark? I’m struggling with io manger and spark and dataframes (pickles are not possible). Need to write my own io-manager.