Hi all, need some help 🙂 I try to parametrize all similar assets based on Yaml file. How to catch asset metadata (name, keys, ...) during an asset execution ? Example of what I try to :

@asset()
def test_asset(context):
    print(context.asset_name)
    >>> test_asset

Thanks !

This should be possible. You can check everything that’s passed to the

context

arg by inspecting the

OpExecutionContext

Class: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster/dagster/_core/execution/context/compute.py#L91 From then it’s just a matter of figuring out what information you need exactly

I will look at it, thanks 🙂

I would maybe look into what you’re trying to do exactly though, in many cases, using partitioned assets might be the way to go.

Today, I create one extraction asset for each source table. The format looks always the same, only the parameters changes. Example :

@asset(key_prefix=["ETL"],required_resource_keys={"my_sap_co"},retry_policy=sap_retry_policy)
def SAP_TABLE1(context):
    
    my_connection = context.resources.my_sap_co
    SAP_TABLE1 = my_connection.get_table(
      table="SAP_TABLE1",
      fields="F1,F2,F3",
      filters=["Filter1","Filter2"])
  

    return SAP_TABLE1

I would like to create a Yaml file containing all the parameters for each asset instead. Example :

asset:
  SAP_TABLE1:
      fields: F1,F2,F3
      filters: 
        - ["Filter1","Filter2"]
  SAP_TABLE2:
      fields: ....
      filters: 
        - [....]

The ultimate goal is to be able to create assets on the fly by reading this Yaml file :)

What I'm trying do is very similar to this, if you find an example or have one you can share that'd be much appreciated!

Right, in case you wanna track these assets separately (read: not as partitions of a single asset) I’d probably use an asset factory type of logic, something like:

def make_sap_table(conf):
    my_assets = []
    for item in conf:
        @asset(name=item.name, key_prefix=["ETL"],required_resource_keys={"my_sap_co"},retry_policy=sap_retry_policy)
        def table_asset(context):
            
            my_connection = context.resources.my_sap_co
            SAP_TABLE = my_connection.get_table(
            table=item.table,
            fields=item.fields,
            filters=item.filters)
        

            return SAP_TABLE
        my_assets.append(table_asset)

    return my_assets

very cool ! I will try it 🙂 thanks