Daniel Gafni
07/12/2022, 3:35 PM.upstream_output.has_asset_partitions
of the InputContext
in the IO Manager is for some reason set to False
. I can't access the upstream partitions. This happens even if I try to materialize the non-partitioned asset as a part of a partitioned job including the both assets.
Is this intended?yuhan
07/12/2022, 7:48 PMsandy
07/12/2022, 10:32 PMcontext.has_asset_partitions
work? (i.e. no upstream_output
)Daniel Gafni
07/13/2022, 3:24 PMFalse
toosandy
07/14/2022, 6:48 PMDaniel Gafni
07/18/2022, 10:37 AMcontext.asset_partitions_def
is only accessible from the IO Manager context, not from the OPExecutionContext
(in the @asset
decorator)
2. I still can't get the exact partitions (keys) set. I can access the context.asset_partitions_def.get_partition_keys()
method, but it requires passing current_time
. I can get it from context.asset_partition_key
, but I'm not sure if this is a good general solution. What would be your advise here?sandy
07/18/2022, 4:17 PMcurrent_time
is Optional on get_partition_keys
- in general, it doesn't need to be passedDaniel Gafni
07/18/2022, 6:19 PMsandy
07/19/2022, 12:29 AM