Andras Somi
08/08/2023, 5:15 PM@op
def my_op(my_input):
...
@graph_asset(
name="my_graph_asset",
ins={
"my_input": AssetIn(
AssetKey(["my_upstream_asset"]),
partition_mapping=LastPartitionMapping(),
)
},
)
def valuation_components(my_input):
return my_op(my_input)
but it still tries to load all the partitions from the upstream in my_op
. How to do this properly? (I guess I need to do something with my op, but I cannot figure out what.)Andras Somi
08/08/2023, 6:52 PMPickledObjectS3IOManager
from dagster_aws.s3
, which is a subclass of UPathIOManager
. Is it possible that this won’t support partition mapping?yuhan
08/09/2023, 10:46 PMAndras Somi
08/10/2023, 3:22 PM