Weekly Release Highlights 1 0 11 sparkles new Assets can be

Question

Weekly Release Highlights 1 0 11 sparkles new Assets can be loaded outside of a run e g in notebooks handshake Snowflake IO Manager now works with dbt assets linked paperclips Python dependency latest version of Alembic is supported

Stephen Bailey · Answer

i like this highlights in slack thing

geoHeil · Answer

< yuhan> can you explain `Assets can be loaded outside of a run e g in notebooks` in more detail or link to a description

sandy · Answer

here s the documentation <https docs dagster io master concepts assets software defined assets loading asset values outside of dagster runs>

geoHeil · Answer

Interesting but where would Jupyter get the repository definition from

sandy · Answer

you would need your Dagster python code to be installed in the same python environment is that difficult in your setup

geoHeil · Answer

I think this should be doable I will have to experiment with this

Nicolas Galland · Answer

great feature thanks

geoHeil · Answer

< sandy> how to handle the types if the associated IO manager can work with i e pandas and spark dataframes as the output How can the desired type of the loaded asset be specified in ```repo load asset value AssetKey asset1 ```

geoHeil · Answer

In any case I get `Attempting to access asset key but it was not provided when constructing the OutputContext` when trying to execute `repo load asset value AssetKey prefix key `

geoHeil · Answer

<https dagster slack com archives C01U5LFUZJS p1665073045992489> might be related

sandy · Answer

here s a PR that should fix this issue <https github com dagster io dagster pull 9914>

sandy · Answer

gt How can the desired type of the loaded asset be specified `load asset value` accepts a `python type` argument that allows you to specify that type that you want to load into details here <https docs dagster io apidocs repositories dagster RepositoryDefinition load asset value>

geoHeil · Answer

When I pass a spark dataframe as the type `DagsterInvariantViolationError Attempting to access op def but it was not provided when constructing the InputContext` is the error message I am greeted with

geoHeil · Answer

This is due to `meta config = context op def output defs 0 metadata` trying to access the metadata which is part of the assets definition

geoHeil · Answer

how can an `op def` be provided to this input context

sandy · Answer

hey < geoHeil> we can include the `op def` on there in a future change but why not just use `context metadata`

geoHeil · Answer

probably because in an old version of dagster this was mandatory it is certainly much nicer to use context metadata I somehow overlooked this change Let me try this

geoHeil · Answer

seems to work fine Great

sandy · Answer

awesome