Would it be possible to allow `AssetsDefinition.fr...
# dagster-feedback
g
Would it be possible to allow
AssetsDefinition.from_graph()
to accept an
output_required
boolean flag like
@asset
does?
Context: I have a graph that materializes an external asset - I'd like to
yield AssetMaterialization
with the associated metadata but seem to be forced to return a value from the graph.
s
Hey Gabe - you can accomplish this by setting
is_required=False
on the
Out
of the relevant op inside the graph
g
Ok I'll give this a try and report back. Thanks
No luck. I set the following on the last
@op
in the graph: and this is the error I get when loading the definitions:
Copy code
dagster._check.CheckError: Invariant failed. Description: All leaf nodes within graph 'historical_perimeters_tiles' must generate outputs which are mapped to outputs of the graph, and produce assets. The following leaf node(s) are non-asset producing ops: {'generate_and_upload_tiles'}. This behavior is not currently supported because these ops are not required for the creation of the associated asset(s).
generate_and_upload_tiles
yields an
AssetMaterialization
conditionally on some configuration, but otherwise does not return a value or yield anything else.
s
Does the asset key of the
AssetMaterialization
yielded by
generate_and_upload_tiles
depend on stuff that happens at runtime? Or is it known ahead of time? If the latter, I'd recommend not explicitly yielding an
AssetMaterialization
, and instead conditionally yielding an
Output
.
g
I was hoping to avoid an output because the actual asset is remote -- I'd like just the metadata to be recorded. Is that possible?
s
you can do
Output(None, metadata=...)
g
But no the key is static with configuration.
Ok let me give that a try
I'm having the last op
return Output(None, metadata={...})
, and then the
@graph
returns the result of that op directly.
s
if you use
Output[None]
, as the type annotation on your decorated function, I believe that should stop it from writing a pickled None to disk
g
Thx, btw, I had family in town and wasn't working -- I'll give this a shot and report back.
That appears to have worked. Thanks!