The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

<@U01AV4K3W14> <https://github.com/dagster-io/dagster/discussions/14477#discussioncomment-6124333> I think it almost works even in the dynamic way sufficiently well. The only remaining issue is the layout of the SQL code. Can the formatting be changed to look as nice as in the old API?

Did you try `MetadataValue.md`? What did you get?

this is not 100% exactly the same - but I think good enough to get the job done

Execution Started At
Execution Completed At
Execution Duration
also are missing - but it should be fairly simple to re-add them. <https://github.com/dagster-io/dagster/discussions/14477#discussioncomment-6124650> however: 1) How is `stream` different from `stream_raw_events` in the example linked from the dagster git? 2) Is the
```event_node_info```
 the same type of event or should this go outside the loop?

just to be clear, these are different APIs you are talking about.
• When you are talking about formatting in the old API, you are describing the description of the Software-defined asset. This occur at definition time.
• The API you are using is the metadata for the asset materialization. This occurs in a computation at run time.


&gt; Execution Started At
&gt; Execution Completed At
&gt; Execution Duration
Sure, these could be added back as default metadata entries. Curious why you find them valuable?

The original thought process for leaving them out was that we wanted the user to make a conscious decision about what metadata they want to add as part of their asset materializations.

As the Dagster framework, we didn’t want to make this assumption on behalf of the user.

Runtime is the most interesting to be honest

But when trying to retrofit these (delbierately) AttributeError: 'Output' object has no attribute 'event' is failing

As seen: <https://github.com/dagster-io/dagster/discussions/14477#discussioncomment-6124650>

where can I get hold of the `
```event```
in my function with the list of streamed events?

The full example is here: <https://github.com/dagster-io/dagster/blob/master/examples/experimental/tutorial_dbt_dagster_v2/tutorial_dbt_dagster_v2/assets/runtime_metadata.py>

• You need to stream the raw events using `.stream_raw_events`. These are the raw logs that are outputted by dbt. Their API is as follows: <https://docs.getdbt.com/reference/events-logging#structured-logging>
• Then for each event, you convert them to their corresponding Dagster events (`Output`, `AssetObservation`)
• For the `Output`’s, then you look at the original raw log and extract the relevant information from it.

Is it guaranteed that all observation events occur before the out? And that all assets occur in order : observation 1 out 1, observation 2 out 2?

They occur in the order that is outputted by the dbt logs.