https://dagster.io/ logo
#ask-community
Title
# ask-community
c

Chris Evans

05/25/2022, 11:07 PM
Hey @owen, just wanted to flag something I noticed moving back to the dbt cli resource from the rpc resource for one of my user repos. There seems to be some logging changes going on w/ DBT at least on version 1.1.0. Everything looks great w/ the default dbt logging when running commands via dagster-dbt on a Mac. However, when I run the same command on Linux (containerized) I get a whole host of extra logging in addition to the correctly formatted messages (screen below). When I enable this flag
DBT_ENABLE_LEGACY_LOGGER=True
, the excess verbose messages disappear. However, the messages coming back aren't parsed correctly because it seems like the message structure has changed to something more nested:
Copy code
{'timestamp': '2022-05-25T22:32:28.957628Z', 'message': '{"code": "Z015", "data": {}, "invocation_id": "f8f78495-2ab1-4aa5-aa95-6b466097fd67", "level": "info", "log_version": 2, "msg": "Finished cleaning all paths.", "pid": 6183, "thread_name": "MainThread", "ts": "2022-05-25T22:32:28.957440Z", "type": "log_line"}', 'channel': 'dbt', 'level': 11, 'levelname': 'INFO', 'thread_name': 'MainThread', 'process': 6183, 'extra': {'run_state': 'internal'}}
🤖 1
o

owen

05/25/2022, 11:09 PM
ah I see, thanks for flagging! I'll see if there's a quick fix that we can get in for next week's release (I think you're right that this is probably because of a change in dbt 1.1.0)
c

Chris Evans

05/25/2022, 11:18 PM
Of course, looking forward to it!
s

Steven Litvack-Winkler

10/10/2022, 6:05 PM
My team is encountering a similar issue, what is the status of this? @owen
o

owen

10/10/2022, 6:13 PM
hi @Steven Litvack-Winkler -- what version are you running of dagster-dbt / dbt-core? this is working as of the most recent version of dagster-dbt, but a bug could have resurfaced
s

Steven Litvack-Winkler

10/10/2022, 6:18 PM
I’m running dagster 0.15.10 and dbt-core 1.2.1.
o

owen

10/10/2022, 6:34 PM
got it -- I believe this issue got fixed in dagster 1.0 (which corresponds to dagster-dbt 16.0). I think dagster-dbt 16.x should be compatible with dagster 0.15.10 (even if pip complains about it), so if you're not in a position to upgrade your dagster version, you may be able to upgrade just that version
s

Steven Litvack-Winkler

10/10/2022, 6:39 PM
Okay, I’ll try this. Thanks for the help.
Just want to give a small update. I have dagster deployed via helm and kubernetes. I was able to upgrade my project to dagster 1.0 easily but this isn’t fixing the issue and I’m guessing the issue is that the deployment is through an older version. Looking to upgrade the deployment as well this or next week.
2 Views