https://dagster.io/ logo
Title
m

martin o leary

05/18/2023, 1:30 PM
I asked this before but can't seem to find the answer (and don't have access to my old code 😞 ) How can I get the compiled SQL from within the
runtime_metadata_fn
? I definitely did have access to it (back a few months ago maybe) and added to the asset metdata before but it's not showing in the
node_info
dictionary:
{
    'database': 'db', 
    'schema': 'schema', 
    'name': 'name', 
    'resource_type': 'model', 
    'package_name': 'core', 
    'path': 'cpath', 
    'original_file_path': '', 
    'unique_id': '.', 
    'fqn': [], 
    'alias': '', 
    'checksum': {
        'name': 'sha256', 
        'checksum': ''
    }, 
    'config': {
        'enabled': True, 
        'alias': None, 
        'schema': None, 
        'database': None, 
        'tags': [], 
        'meta': {}, 
        'materialized': 'table', 
        'incremental_strategy': None, 
        'persist_docs': {}, 
        'quoting': {}, 
        'column_types': {}, 
        'full_refresh': None, 
        'unique_key': None, 
        'on_schema_change': 'ignore', 
        'grants': {}, 
        'packages': [], 
        'docs': {
            'show': True, 
            'node_color': None
        }, 
        'post-hook': [], 
        'pre-hook': []
    }, 
    'tags': [], 
    'description': '', 
    'columns': {}, 
    'meta': {}, 
    'docs': {
        'show': True, 
        'node_color': None
    }, 
    'patch_path': None, 
    'build_path': None, 
    'deferred': False, 
    'unrendered_config': {
        'materialized': 'table'
    }, 
    'created_at': time, 
    'relation_name': '', 
    'raw_code': 'raw uncompiled code, 
    'language': 'sql', 
    'refs': [], 
    'sources': [], 
    'metrics': [], 
    'depends_on': {
        'macros': [], 'nodes': []
    }, 
    'compiled_path': None
}
r

rex

05/18/2023, 2:41 PM
This information is coming from your manifest. Have you tried compiling your dbt manifest locally and inspecting to see where the field lives? From the json schema, it looks like it should live under `raw_code`: https://schemas.getdbt.com/dbt/manifest/v9/index.html#nodes_additionalProperties_oneOf_i3_raw_code
m

martin o leary

05/18/2023, 3:00 PM
Hey @rex -
raw_code
contains un-compiled code. I want to show the compiled code so that my users who are in around dagit can easily copy + paste the SQL into a query editor and reproduce the same results. This was definitely possible before but I'm not entirely sure how I did it!
r

rex

05/18/2023, 3:06 PM
So I was able to get this field (it’s optional, meaning that I don’t think it shows up in all manifests that are created by dbt) to show up by running
dbt compile
yeah, I tried finding the same field after running
dbt parse
(which also produces a manifest) and it doesn’t show up. so use
dbt compile
(which makes sense, as the name suggests!)
🙏 1
m

martin o leary

05/18/2023, 3:11 PM
OK cool - that shows up in manifest now - getting there 🙂 If I get it to show up - does that mean I should load my project into dagster from the manifest?
r

rex

05/18/2023, 3:11 PM
yeah, we recommend that you load from the manifest for performance reasons (since
dbt compile
can potentially take a long time)
🙏 1
m

martin o leary

05/18/2023, 3:21 PM
Boom! There it is 🙂 - thank you very much @rex!