After updating our dagster instance to the latest ...
# ask-community
h
After updating our dagster instance to the latest version my databricks jobs are suddenly failing - I can’t really figure out what has changed that is causing this, but the job fails on trying to get files from the stdout directory in my job folder on dbfs, which does not exist.
Copy code
{ 'error_code': 'RESOURCE_DOES_NOT_EXIST', 'message': 'No file or directory exists on path ' '/tmp/dagster_staging/XXX/XXX/stdout.'}
note: removed job hash and job name. The path itself exists, just stdout does not
Ah, it’s just that the job fails quickly. It took a while for the error to show up in the databricks error logs
j
Getting stuck on this error at the moment trying to get a PoC working on databricks. Sure is confusing, at least with respect to how it is on EMR