Hi team, I want to ask about Jupyter notebook. Can we paste the input of the notebook on run config (image below) instead of

ins

parameters in

define_dagstermill_asset

jupyter_notebook = define_dagstermill_asset(
            name=f"jupyter_notebook_{source}",
            notebook_path=file_relative_path(__file__, "./notebooks/jupyter-sql-snowflake.ipynb"),
            ins={
                "config": AssetIn(f"config_key_{source}"),            <----------- this input
            },
            partitions_def=partitions_dict[f"fivetran_{source}"],
            group_name="jupyter_notebook",
        )

I'm not super familiar with dagstermill, but I don't think it'd be possible to set that input at runtime because that's what defines your upstream asset dependency, and Dagster needs to know what your asset dependencies at compile time are in order to build your asset graph

@Zach so are there any chance that we can use the partition as the input of the dagstermill_asset?

I'd have to defer to someone who knows that integration better than I do, it sure seems like it should work with partitions but I don't know how you would do that

Like the partition_key_to_vars_fn parameters in load_assets_from_dbt_project that translate a given partition key (e.g. ‘2022-01-01’) to a dictionary of vars to be passed into the dbt invocation

What you are suggesting could work but is not best practice - I think the situation is just a bit rough until we add support for

partition_key

from dagstermill.

@asset(
   group_name="jupyter_notebook",
   name="config_key",
   partitions_def=partitions_dict[f"fivetran_{source}"],
        )
   def config_key(context):
        partition_key = context.asset_partition_key_for_output()     <---------using this as the output
        return partition_key

   # Asset backed by a Jupyter notebook
jupyter_notebook = define_dagstermill_asset(
    name="jupyter_notebook",
    notebook_path=file_relative_path(__file__, "./notebooks/jupyter-sql-snowflake.ipynb"),
    ins={
        "partition_key": AssetIn("config_key"),
        },
    partitions_def=partitions_dict[f"fivetran_{source}"],
    group_name="jupyter_notebook",
    )

Hi guy, I figure out a way how to pass the partition-key to jupyter notebook asset, that is using

asset_partition_key_for_output

and returning the partition-key as the output of the upstream.