It surprises me that it's not possible to specify ...
# dagster-feedback
j
It surprises me that it's not possible to specify the name of the file the
FileManager
resources write to, forcing the use of uuids. A couple of use-cases for this: an asset that produces a data-set to be written to disk; a sensor that triggers a job based on a new file that produces an output file to be wrapped in a light weight api. I guess my first question is, am I missing something; is there some reason this is a really bad idea? If not, would it be possible to add that functionality with an optional argument that defaults to the existing uuid implementation? I'd be happy to open an issue and PR for this.
d
Just took a look at the FileManager. Another question I have: will it support working with directories? Some assets are directories rather than files. Also, why reinvent the wheel when fsspec exists?
s
FileManager
is a concept from Dagster's infancy that hasn't been updated in a while. We're interesting in revisiting it at some point. I'm honestly a little bit hazy on it. It's meant to abstract over the physical location of the files - if you know where the file should be stored, why not just write there directly?
d
Oh thanks Sandy
j
Thanks Sandy, I think abstracting away the physical location is still useful - local fs for dev then s3 for prod - it's just a folder structure and naming within that location that I'd like to define. How are other people doing fileio with dagster - especially with blob storage? Just setting up dev services locally that are equivalent to prod services and then hard-coding access?