I m using the new define assets from graph feature in 0 15 0 dagster #ask-community

I'm using the new 'define assets from_graph' featu...

John Boyle

06/15/2022, 3:53 AM

I'm using the new 'define assets from_graph' feature in 0.15.0. I can get it to work but I can't figure out how to add the created graph->asset to an asset group. It appears to get added do an asset group named 'default'.

Copy code

my_graph_asset = AssetsDefinition.from_graph(my_graph)

graph doesn't have a group_name property, nor does from_graph(). In dagit I see my original asset group (my_asset_group) and another asset group named 'default'. my graph asset has upstream dependencies and I see the connections between my original group and the new one, so it works, but ideally I'd like to add the graph asset to the other asset group. is there a way to do this? thanks!

claire

06/15/2022, 3:30 PM

Hi there John. This is something we'll add within the next two releases, it was something we didn't quite have time to finish within 0.15.0. For now, one workaround you can do:

Copy code

AssetsDefinition.from_graph(my_graph).with_prefix_or_group(group_names_by_key={AssetKey("foo"): "group_name"})

👍 1

claire

06/15/2022, 3:30 PM

I'll make an issue so we can track this as well:

claire

06/15/2022, 3:30 PM

@Dagster Bot issue add group_name to AssetsDefinition.from_graph

Dagster Bot

06/15/2022, 3:30 PM

Created issue at: https://github.com/dagster-io/dagster/issues/8407

John Boyle

06/15/2022, 3:32 PM

Thanks Claire! I'll try the workaround.

John Boyle

06/15/2022, 5:25 PM

hm, I tried it and I think I'm missing a step. First I replaced my original with the workaround. I removed:

Copy code

my_graph_asset = AssetsDefinition.from_graph(my_graph)

and replaced with:

Copy code

AssetsDefinition.from_graph(my_graph).with_prefix_or_group(group_names_by_key={AssetKey("my_graph_asset"): "group_name"})

results in: Import Error: cannot import name "my_graph_asset" so I added my original line back (i.e. maybe I need to create the asset first like before)

Copy code

my_graph_asset = AssetsDefinition.from_graph(my_graph)
AssetsDefinition.from_graph(my_graph).with_prefix_or_group(group_names_by_key={AssetKey("my_graph_asset"): "group_name"})

and this doesn't trigger an error, but the result is the same as when I wasn't using the workaround. i.e. Expected: I would see only the asset_group "group_name" in dagit with 'my_graph' connected at the correct spot. Observed: I see two asset groups. "group name" and "default" and 'my_graph' is connected with lines, not as a box (i.e. I have to click on the link to jump to the 'default' group'). I feel like I'm missing a conceptual step here.

claire

06/15/2022, 6:09 PM

AssetsDefinition.from_graph infers the output asset keys to be the output names of the graph defined via the ‘out’ param in the graph decorator, or if the ‘out’ doesn't exist, the output asset key is the name of the decorated graph. I think if “my_graph” is the name of your decorated graph function, you will need to specify AssetKey(“my_graph”) instead of AssetKey(“my_graph_asset”)

👍 1

claire

06/15/2022, 6:10 PM

It's confusing for sure, and we'll have ergonomic improvements to in the next couple of weeks.

John Boyle

06/15/2022, 6:15 PM

That worked! thanks for the extra explanation. Also, no worries on the temporary confusion. I'm really impressed with how responsive you all have been and appreciate it.

blob cheer 1

3 Views

Open in Slack

Previous Next