https://dagster.io/ logo
#dagster-releases
Title
# dagster-releases
y

yuhan

09/29/2022, 10:06 PM
Weekly Release Highlights: 1.0.11 🆕 Assets can be loaded outside of a run, e.g. in notebooks 🤝 Snowflake IO Manager now works with dbt assets 🖇️ Python dependency: latest version of Alembic is supported
🎉 11
D 9
s

Stephen Bailey

09/30/2022, 2:07 AM
i like this highlights-in-slack thing!
☝️ 3
daggy love 1
g

geoHeil

09/30/2022, 1:03 PM
@yuhan can you explain
Assets can be loaded outside of a run, e.g. in notebooks
in more detail or link to a description?
g

geoHeil

09/30/2022, 2:53 PM
Interesting - but where would Jupyter get the repository definition from?
s

sandy

09/30/2022, 2:54 PM
you would need your Dagster python code to be installed in the same python environment. is that difficult in your setup?
g

geoHeil

10/01/2022, 6:55 AM
I think this should be doable - I will have to experiment with this.
n

Nicolas Galland

10/05/2022, 12:00 PM
great feature, thanks
g

geoHeil

10/06/2022, 2:28 PM
@sandy how to handle the types if the associated IO manager can work with i.e. pandas and spark dataframes as the output? How can the desired type of the loaded asset be specified in:
Copy code
repo.load_asset_value(AssetKey("asset1"))
In any case I get:
Attempting to access asset_key, but it was not provided when constructing the OutputContext
when trying to execute
repo.load_asset_value(AssetKey(["prefix", "key"]))
s

sandy

10/06/2022, 6:26 PM
here's a PR that should fix this issue: https://github.com/dagster-io/dagster/pull/9914
How can the desired type of the loaded asset be specified
load_asset_value
accepts a
python_type
argument that allows you to specify that type that you want to load into. details here: https://docs.dagster.io/_apidocs/repositories#dagster.RepositoryDefinition.load_asset_value
g

geoHeil

10/07/2022, 7:16 PM
When I pass a spark dataframe as the type:
DagsterInvariantViolationError: Attempting to access op_def, but it was not provided when constructing the InputContext
is the error message I am greeted with.
This is due to:
meta_config = context.op_def.output_defs[0].metadata
trying to access the metadata which is part of the assets definition.
how can an
op_def
be provided to this input context?
s

sandy

10/07/2022, 11:26 PM
hey @geoHeil - we can include the
op_def
on there in a future change, but why not just use
context.metadata
?
g

geoHeil

10/08/2022, 7:30 AM
probably because in an old version of dagster this was mandatory - it is certainly much nicer to use context.metadata. I somehow overlooked this change. Let me try this
seems to work fine! Great
s

sandy

10/10/2022, 3:05 PM
awesome