https://dagster.io/ logo
Title
c

Chris Nogradi

04/26/2023, 9:20 PM
I am using the documented sensor paradigm which monitors an s3 bucket for new files and invokes a job that materializes an asset. The problem I have is that the output produced is the same for each different source file and I can't figure how to make each invocation of the asset yield a different asset key per source file given to it. Ideas?
c

chris

04/26/2023, 9:38 PM
when you define an asset in software, the same asset key is intentionally given to every artifact materialized by that software defined asset. This is probably better modeled as a partition of the asset per file: https://docs.dagster.io/concepts/partitions-schedules-sensors/partitions#dynamically-partitioned-assets
c

Chris Nogradi

04/26/2023, 10:01 PM
@chris So do the emitted outputs have different path keys when partitioning is used so that the iomanager can save them to different places?
c

chris

04/27/2023, 4:45 PM
that’s right
c

Chris Nogradi

04/27/2023, 11:07 PM
thanks @chris