Hey, I’m having an issue using the s3_fake_resource, with the following error

def put_object(self, Bucket, Key, Body, *args, **kwargs):
        self.mock_extras.put_object(*args, **kwargs)
>       self.buckets[Bucket][Key] = Body.read()
E       AttributeError: 'bytes' object has no attribute 'read'

I’m simulating mocking just uploading a zipped file so just some bytes, but it seems the fake resource expect to be able to

.read()

the Body? Why is this? Or am I using the resource wrong?

It looks like the S3 PutObjects API expects:

bytes or seekable file-like object

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.put_object But the

s3_fake_resource

incorrectly implements it to only take a seekable file-like object. We can get a fix out for this - sorry about that!

Hey, thanks for the quick reply @jordan! No worries, hopefully it’s a quick easy fix 🙂

The deeper I dig on this the less polished it gets - it looks like perhaps when it was first introduced, it was intended to be used with `ResourceDefinition.hardcoded_resource()`: https://github.com/dagster-io/dagster/blob/6ee348c9b5645688c1f3de3521158a8ebf3c1b09/python_modules/libraries/dagster-aws/dagster_aws_tests/s3_tests/test_s3_file_manager.py#L141-L152 And over time, it’s been both deprecated and then re-added - but never brought under test or up to date with current Dagster idioms. It’s actually not used anywhere in Dagster’s own tests anymore - instead, we use a real S3 resource but mock out the S3 service with moto: https://github.com/spulec/moto You might find this to be a less brittle approach since the moto implementations (particularly around popular services like S3) are quite well supported.

ok makes sense. Would something like the following be a good way to create a mock s3 resource in that case?:

import boto3
from moto import mock_s3
from dagster import resource, Array


@mock_s3
@resource(config_schema={"buckets": Array(str)})
def s3_resource_mock(init_context):
    conn = boto3.resource("s3", region_name="eu-west-2")

    for b in init_context.resource_config["buckets"]:
        conn.create_bucket(Bucket=b)

    return conn

with

buckets

being a list of buckets I expect to find in s3