https://dagster.io/ logo
Join Slack
Powered by
I am loading 2 different dbt projects with "load_a...
# ask-community
s
I am loading 2 different dbt projects with "load_assets_from_dbt_project" and each asset is grouped using "node_info_to_group_fn". This is the result(see image). As can you see the sources of my models are outside of the group. I want sources inside the same group, Is this possible? I want that sources inside the same group.
m
You should double check the AssetKey associated with the sources imported. You will find they are probably not identical. We also have 2 dbt projects and in order for things to be wired up correctly, we needed to ensure the AssetKey generated by the 
load_assets_…
function map correctly to wire up things as we expect
s
Hi @Manish Khatri I am pretty sure that the function "node_info_to_group_fn" does not give you access to the "source" only give acess to the model.. see my images . I am printing the type and names and only the models are included
The object for each node is this
Copy code
{
  'resource_type': 'model', 
	'depends_on': {
		'macros': ['macro.dbt.is_incremental'],
		'nodes': ['source.netflix.airbnb.listings']}, 
		'config': {
			'enabled': True, 
			'alias': None, 
			'schema': None, 
			'database': None, 
			'tags': [], 
			'meta': {}, 
			'materialized': 'incremental', 
			'incremental_strategy': None, 
			'persist_docs': {}, 
			'quoting': {}, 
			'column_types': {}, 
			'full_refresh': None, 
			'unique_key': None, 
			'on_schema_change': 'fail',
			'grants': {}, 
			'packages': [], 
			'docs': {
				'show': True, 
				'node_color': None}, 
				'netflix': None, 
				'post-hook': [], 
				'pre-hook': []
			}, 
			'database': 'Temporal', 
			'schema': 'PRACTICES', 
			'fqn': ['netflix', 'src', 'src_listings'], 
			'unique_id': 'model.netflix.src_listings', 
			'raw_code': "WITH raw_listing", 
			'language': 'sql', 
			'package_name': 'netflix', 
			'root_path': '/opt/dagster/app/dbt/netflix/seek_connect', 
			'path': 'src/src_listings.sql', 
			'original_file_path': 'models/src/src_listings.sql', 
			'name': 'src_listings', 
			'alias': 'src_listings', 
			'checksum': {'name': 'sha256', 'checksum': '370f8f95def342266932d323f6a2be1df6c62c47607d66aa7422e9a206a0b5f1'}, 
			'tags': [], 
			'refs': [], 
			'sources': [['airbnb', 'listings']], 
			'metrics': [], 
			'description': 'Internal staging model from source directly', 
			'columns': {}, 
			'meta': {}, 
			'docs': {
				'show': True, 
				'node_color': None}, 
				'patch_path': '<netflix://models/src/src_listings.yml>', 
				'compiled_path': None, 
				'build_path': None, 
				'deferred': False, 
				'unrendered_config': {
					'netflix': None, 
					'materialized': 'incremental', 
					'on_schema_change': 'fail'
				}, 
			'created_at': 1677254905.8756075
	}
You only have a property called "sources" that is a List with string nothing else
I forgot add the output of the logger. You can see that only print 3 things that are the models direct children
m
Ah yes. Ok, can you click on both 
reviews
and take a look at the AssetKey assigned to it on the right hand window pane that will show up. It will be the very 1st line of text. Just below it will be “View in Asset Catalog” link. I believe the 2 asset keys are different, that is why they appear as 2 different entities
If both models 
src_reviews
access it via the same source arguments, e.g. 
{{ source("something", "reviews") }}
you should find that the DAG will only show 1 reviews box with an arrow going to each of the dbt models
s
Yes , is like you said. The question was like: Can I broke this rules? I can force the grouping of sources and models but Sandy Alrady answer me that I need to create a sourceAsset in order to do it
m
A SourceAsset would be helpful as it does document the metadata of the tables. But I wouldn’t know much more beyond that
Open in Slack
PreviousNext
Powered by