HI All, Is Dagster Serverless down ? Could not reach user code server. gRPC Error code: UNKNOWN

Hi, do you mind sharing your org name? I can take a look…

@prha Oath

I see the serverless agent and code locations up for your project… may have been some temporary error that we’ve recovered from, though I can see the same error in the logs

Yes, I can deploy succesfully

So, the latest theory is that running the job is causing OOM issues on the code location, which causes it to restart.

Thanks. Trying that

For the dbt error in the run logs, are you by any chance changing the target path in your

dbt_project.yml

file? This is something that we can’t handle very well (reference: https://github.com/dagster-io/dagster/issues/10918).

I read that; i did not change anything. Job just stopped working suddenly. It's working on my laptop.

@jitesh soni what version of dbt-core do you have installed on your laptop? A new version of that just came out today

Hi @prha , I am unable to find this

Isolate Run Environment

option . Screenshot attached

You’ll probably have to hold down shift and click the materialize button in order to see those options.

Do not see it 😞

Ah shift-click on the “materialize all” text not the drop down

Server not found 😞

I disabled non-isolated runs for your deployment, so you might be able to try again. For the dbt error, I wonder if you could upgrade your

dbt-core

package locally and repro the error? If it does repro locally with that version (which released today), you might want to pin the package to an earlier version.

Hi @prha , Host are still not reachable!

The above exception was caused by the following exception:
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNKNOWN
details = "Exception calling application: No server created with the given handle"
debug_error_string = "{"created":"@1674794708.234532664","description":"Error received from peer ipv4:10.0.94.163:4000","file":"src/core/lib/surface/call.cc","file_line":966,"grpc_message":"Exception calling application: No server created with the given handle","grpc_status":2}"
>

I’m not sure how the browser would have any effect on your grpc server, but I am glad that you are unblocked now.

Folks, I checked the same issue occurred again on multiple days. Any suggestions ?

FileNotFoundError: [Errno 2] No such file or directory: '/root/.pex/user_code/b64440d5603ca80da76721e1a6eaa442b73252b7/working_directory/root/assets_dbt_python/../dbt_project/target/run_results.json'
  File "/root/.pex/installed_wheels/77b42134a058a50fb83cd3926911571c40f8a31dfcf3a9f3584f5e74ebb969c7/dagster_dbt-0.17.13-py3-none-any.whl/dagster_dbt/cli/utils.py", line 168, in parse_run_results
    with open(run_results_path, encoding="utf8") as file:
The above exception occurred during handling of the following exception:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x99 in position 16: invalid start byte

Do you possibly have dbt-core 1.4.0 installed? The latest version of dagster added a temporary pin to dbt-core 1.3.0 until we can fix some breaking changes in the dbt-core release

@daniel I have not upgraded local to 1.4.0 . What's confusing to me is that I do not supply dbt when I deploy to Dagster. Where do I specify which version of DBT is deployed to Dagster ?

dagster-dbt depends on dbt-core - i think the easiest way to resolve this would be to upgrade to the latest version of dagster (1.1.14) where we have a pin applied to fix a breaking change introduced in dbt-core 1.4.0

Hello - I looked into it briefly as well. It appears the project is using dbt-core 1.4.1 and dagster 1.1.13. This just depends on when the project was first deployed from github. As daniel suggested, setting

dagster==1.1.14

in your setup.py should force a rebuild of your dependencies and pull in the pinned dbt-core.