https://dagster.io/ logo
Title
d

Daniel Mosesson

03/13/2023, 9:11 AM
What is the best way to have a partitioned graph backed asset depend on the latest materialization of some other (non-partitioned) asset?
:dagster-bot-responded-by-community: 1
j

Jakub Zgrzebnicki

03/13/2023, 9:30 AM
I have defined something like this:
@asset(
    key_prefix=KEY_PREFIX,
    ins={
        'types_mapping': AssetIn(
            key=[KEY_PREFIX, 'types_mapping'],
            partition_mapping=LastPartitionMapping()
        )
    }
)
And type_mapping is daily partitioned asset 😉
But from What you describe you need to have it the other way: In that case it should be simple asset dependency
d

Daniel Mosesson

03/13/2023, 9:47 AM
Even if the upstream asset has no partitions?
j

Jakub Zgrzebnicki

03/13/2023, 9:49 AM
yes, if the input is not partitioned there shouldn't be a problem
d

Daniel Mosesson

03/13/2023, 9:55 AM
That looks like it should work, but when I try and materialize a downstream partition, I get an error saying it cannot be materialized because upstream partitions are missing
j

Jakub Zgrzebnicki

03/13/2023, 9:56 AM
is upstream partition materialized too?
d

Daniel Mosesson

03/13/2023, 9:56 AM
Yes
but I put the asset
in
for the graph backed part, and then passed it as a param to the op
but that might not be right
j

Jakub Zgrzebnicki

03/13/2023, 9:57 AM
could you show the error or definition of your assets?
d

Daniel Mosesson

03/13/2023, 10:00 AM
its basically like:
@asset
def asset1():
 ...

@op
def op1(asset1):
 ....

@graph_asset(
  partitions_def=HourlyPartition(...),
  ins = {
     'asset1': AssetIn('asset1', partition_mapping = LastPartitionMapping())
  }
)
def g1(asset1):
  op1(asset1)
Sorry for the retype, I can't copy and paste from the environment
j

Jakub Zgrzebnicki

03/13/2023, 10:07 AM
in that case you don't need partition_mapping as your upstream is not partitioned.
d

Daniel Mosesson

03/13/2023, 10:15 AM
and passing it plain just works... :headbang: I just made it way too complicated
thank you!
👍 1