https://dagster.io/ logo
Title
g

geoHeil

04/26/2023, 11:20 AM
How can partition dependencies https://dagster.slack.com/archives/C02LJ7G0LAZ/p1671645522498899?thread_ts=1671232829.834029&cid=C02LJ7G0LAZ be pushed to the IO manager or referenced there? It is pretty clear to me how to get access to these handles in the Asset - but as mentioned there if the SCD MERGE into is handled by the IO manager how can I push the reference of the previous day over to it? Is a custom data type required (dict) which pushes both current_partition + previous_partition over to the IO manager? Or is there a more straight forward approach available?
s

sandy

04/26/2023, 9:11 PM
Hey @geoHeil - you can use
context.step_context.asset_partition_key_for_input
- would that work for you?
g

geoHeil

04/26/2023, 9:14 PM
indeed - this should work so if the current partition key would be 3 and the asset_partition_key_for_input lagged by 1 should be 2 and 2 is returned then it should work
do I understand this correctly?
s

sandy

04/26/2023, 9:15 PM
that's correct
:rainbow-daggy: 1
g

geoHeil

04/27/2023, 8:28 PM
I cannot see any references to the
partition_mapping=TimeWindowPartitionMapping(start_offset=-1, end_offset=-1),
in dagit. Is this intentional? Or do I still have an issue somewhere?
s

sandy

04/27/2023, 9:27 PM
Those are not currently represented in Dagit
g

geoHeil

04/28/2023, 3:47 AM
do you have any plans to do so?
s

sandy

04/28/2023, 3:24 PM
We would like to, but it's not at the top of our current priority list. Would you be up for filing a Github issue? We can use it to track demand.