hi, I am trying to get my dbt users to adopt dagster as a part of their development flow. Unfortunately the dagster-dbt errors are unusable due to cryptic errors messages:

Encountered an error while running operation: Runtime Error
  Pickling client objects is explicitly not supported.
  Clients have non-trivial state that is local and unpickleable.

This forces them to fallback to running the dbt CLI instead. Is there way to disable this hardcoded

--log-format json

and see the full error stack?

hi @Stanley Yang -- do you have a screenshot of what this error looks like in dagit (vs what it looks like in the dbt CLI)? Ideally the json format shouldn't be relevant, as the log lines in dagit will have the message parsed out of them. However, for some dbt versions I believe this parsing logic can get confused, so if you have a specific version you're running on that could be helpful for debugging

thanks for looking into this @owen We are running on dagster 0.14.3 and dbt 1.0.0 Here is a screenshot of the error and the raw JSON:

{
  "code": "Q001",
  "data": {
    "exc": "Runtime Error\n Pickling client objects is explicitly not supported.\n Clients have non-trivial state that is local and unpickleable."
  },
  "invocation_id": "c0d51ae7-a863-4283-84ec-a3bbc8a39804",
  "level": "error",
  "log_version": 1,
  "msg": "Encountered an error while running operation: Runtime Error\n Pickling client objects is explicitly not supported.\n Clients have non-trivial state that is local and unpickleable.",
  "node_info": {},
  "pid": 381,
  "thread_name": "MainThread",
  "ts": "2022-05-25T14:54:21.302904Z",
  "type": "log_line"
}

hm that's surprising to me -- that error does seem to be coming from bigquery, but I'm not sure why the log format would have any impact on the execution. If you copy paste the same command that dagster is running (from the logs), and just remove the --log-format json, it works?

hi @owen, sorry for the late reply. Yes that is correct. If I copy paste the dbt command outputted in dagster, if I omit the

--log-format json

, or replace with

--log-format default

, it returns the correct error.

ah no problem -- what is the correct error in this case?

@owen one example is a missing table. In dagster it returns:

Clients have non-trivial state that is local and unpickleable.

When the log-json bit is removed, the output is:

19:48:43  1 of 1 ERROR creating table model mydataset.my_table................. [ERROR in 0.29s]
19:48:43  
19:48:43  Finished running 1 table model, 2 hooks in 1.79s.
19:48:43  
19:48:43  Completed with 1 error and 0 warnings:
19:48:43  
19:48:43  Runtime Error in model my_table (models/my_table.sql)
19:48:43    404 Not found: Table myproject:mydataset.some_required_table was not found in location US

oh wow that is really strange

and yes, the desired behavior is to have the default output. I see a potential challenge as the dbt binary would need to return logs in both JSON and default

gotcha -- it definitely feels like a bug in dbt-land that the log format could impact execution (and I'd encourage you to file a bug report over there). however, I believe that we can make the --log-format json configurable without too much issue. This can probably get into next week's release.

yeah agreed. It is definitely more on the dbt and BQ side. That said, making the log-format configurable would be great!