19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Hi Dagster - I'm getting errors in an op - and the...
# ask-communitym
Matthew Karas
06/23/2022, 11:41 AMHi Dagster - I'm getting errors in an op - and the stack trace is not very helpful. Is there any way to get more information from Dagster as to why something is failing?
Copy code
Stack Trace:
File "/usr/local/lib/python3.9/site-packages/dagster/core/execution/plan/utils.py", line 47, in solid_execution_error_boundary
yield
, File "/usr/local/lib/python3.9/site-packages/dagster/utils/__init__.py", line 406, in iterate_with_context
next_output = next(iterator)🤖 1
d
daniel
06/23/2022, 1:50 PMHi Matthew - this is in Dagit where you're seeing this? Typically the stack trace would continue and there would be more information below (there should also be an error message above the stack trace)
daniel
06/23/2022, 1:52 PMHere's what I see in Dagit when an exception is raised within an op - is that different from what you're seeing?
m
Matthew Karas
06/23/2022, 2:46 PMYeah - this is all I get
Copy code
dagster.core.errors.DagsterExecutionStepExecutionError: Error occurred while executing op "<OP NAME HERE>":
The above exception was caused by the following exception:
RuntimeError: generator raised StopIteration
Stack Trace:
File "/usr/local/lib/python3.9/site-packages/dagster/core/execution/plan/utils.py", line 47, in solid_execution_error_boundary
yield
, File "/usr/local/lib/python3.9/site-packages/dagster/utils/__init__.py", line 406, in iterate_with_context
next_output = next(iterator)Matthew Karas
06/23/2022, 2:46 PMOP NAME HERE is our internal code
d
daniel
06/23/2022, 2:46 PMAh OK - that error message "RuntimeError: generator raised StopIteration" is helpful, but also confusing
daniel
06/23/2022, 2:47 PM(helpful for me, that is)
m
Matthew Karas
06/23/2022, 2:47 PMIt would have been easier to know which line caused the exception.
Matthew Karas
06/23/2022, 2:47 PMI had to dig for that.
d
daniel
06/23/2022, 2:47 PMWere you eventually able to figure out what the problem was?
daniel
06/23/2022, 2:48 PMThere's something specifically unique about the error here that seems to be circumventing our usual error reporting
m
Matthew Karas
06/23/2022, 2:48 PMYes - a query didn't return any rows
d
daniel
06/23/2022, 2:49 PMAny chance there's a minimal way to reproduce it that I could run myself without needing full access to your code?
daniel
06/23/2022, 2:49 PMIn general, exceptions should be bubbled up and reported - something about the way your op handled the error case seems to be not handled correctly on our side
daniel
06/23/2022, 2:49 PMIs the op a generator?
daniel
06/23/2022, 2:50 PMMaybe we don't handle ops that are generators but don't yield anything well?
m
Matthew Karas
06/23/2022, 2:50 PMIts a function that is called by an op - it wasn't actually a query that returned no rows... it was a next() call on an iterator statement
Matthew Karas
06/23/2022, 2:51 PMfoo = next(f for f in foos if foo["bar"] == "baz")
d
daniel
06/23/2022, 2:51 PMAh, maybe we're not handling RuntimeErrors well
m
Matthew Karas
06/23/2022, 2:51 PMDoes everything need to be in an op?
Matthew Karas
06/23/2022, 2:51 PMBecause I like to break down my code into small reusable functions.
d
daniel
06/23/2022, 2:52 PMOps can certainly call out to other functions - I might not totally understand the question
m
Matthew Karas
06/23/2022, 2:52 PMYou had it right.
Matthew Karas
06/23/2022, 2:52 PMok
d
daniel
06/23/2022, 3:01 PMOK, I can reproduce this if the op raises a StopIteration specifically (Other exceptions seem to include a stack trace). Gross! I'll file an issue, thanks for the report
daniel
06/23/2022, 3:08 PMoh weird, the inner stack trace with the useful information even appears in the dagit command-line output! This should be easy to fix then
m
Matthew Karas
06/23/2022, 3:48 PMThanks @daniel!
5 Views