Philippe Laflamme
01/18/2023, 10:33 PMtimezone
argument is used when declaring a partition (i.e.: here)? There’s this about schedules, but it doesn’t mention anything about materializing an asset. Specifically, I’m wondering how it interacts with the Dagster materialization UI and its own Timezone setting. I’m also wondering about what it may mean for an IO manager that needs to read/write partition string representation in a filename or path.chris
01/19/2023, 7:47 PMPhilippe Laflamme
01/19/2023, 11:07 PMUTC
. So ideally, we’d be able to specify partitions in any of the following 3 timezones and dagster would just “get it right”:
• the asset’s local timezone (e.g.: America/Los_Angeles
)
• the browser’s (user’s) timezone (e.g.: America/New_York
)
• UTC
It’s generally a lot easier to think in the asset’s “local timezone”, but making this choice explicit makes it a lot easier to understand what’s going to happen. e.g.: if I ask for partition 2023-01-19T10:00
which one of those 3 possible timestamps am I referring to?
Lastly, when writing the partition out to disk, I’d like to use UTC
timestamps so we don’t have to deal with daylight savings shenanigans.
Also, I haven’t thought this through entirely, but I think we can ignore timezones for any partitioning that’s coarser than hourly