https://dagster.io/ logo
#dagster-feedback
Title
# dagster-feedback
g

Gabe Schine

12/24/2022, 2:56 AM
Would it be possible to allow
AssetsDefinition.from_graph()
to accept an
output_required
boolean flag like
@asset
does?
Context: I have a graph that materializes an external asset - I'd like to
yield AssetMaterialization
with the associated metadata but seem to be forced to return a value from the graph.
s

sandy

12/27/2022, 3:56 PM
Hey Gabe - you can accomplish this by setting
is_required=False
on the
Out
of the relevant op inside the graph
g

Gabe Schine

12/27/2022, 5:57 PM
Ok I'll give this a try and report back. Thanks
No luck. I set the following on the last
@op
in the graph: and this is the error I get when loading the definitions:
Copy code
dagster._check.CheckError: Invariant failed. Description: All leaf nodes within graph 'historical_perimeters_tiles' must generate outputs which are mapped to outputs of the graph, and produce assets. The following leaf node(s) are non-asset producing ops: {'generate_and_upload_tiles'}. This behavior is not currently supported because these ops are not required for the creation of the associated asset(s).
generate_and_upload_tiles
yields an
AssetMaterialization
conditionally on some configuration, but otherwise does not return a value or yield anything else.
s

sandy

12/29/2022, 5:52 PM
Does the asset key of the
AssetMaterialization
yielded by
generate_and_upload_tiles
depend on stuff that happens at runtime? Or is it known ahead of time? If the latter, I'd recommend not explicitly yielding an
AssetMaterialization
, and instead conditionally yielding an
Output
.
g

Gabe Schine

12/29/2022, 6:26 PM
I was hoping to avoid an output because the actual asset is remote -- I'd like just the metadata to be recorded. Is that possible?
s

sandy

12/29/2022, 6:26 PM
you can do
Output(None, metadata=...)
g

Gabe Schine

12/29/2022, 6:26 PM
But no the key is static with configuration.
Ok let me give that a try
I'm having the last op
return Output(None, metadata={...})
, and then the
@graph
returns the result of that op directly.
s

sandy

01/03/2023, 5:09 PM
if you use
Output[None]
, as the type annotation on your decorated function, I believe that should stop it from writing a pickled None to disk
g

Gabe Schine

01/12/2023, 9:15 PM
Thx, btw, I had family in town and wasn't working -- I'll give this a shot and report back.
That appears to have worked. Thanks!
11 Views