https://dagster.io/ logo
#ask-community
Title
# ask-community
m

Matthew Karas

06/23/2022, 11:41 AM
Hi Dagster - I'm getting errors in an op - and the stack trace is not very helpful. Is there any way to get more information from Dagster as to why something is failing?
Copy code
Stack Trace:
  File "/usr/local/lib/python3.9/site-packages/dagster/core/execution/plan/utils.py", line 47, in solid_execution_error_boundary
    yield
,  File "/usr/local/lib/python3.9/site-packages/dagster/utils/__init__.py", line 406, in iterate_with_context
    next_output = next(iterator)
I don't get information about what line of code is failing - what iterator is being iterated over - or anything really. Is there anything I can do to get this to provide more useful information?
🤖 1
d

daniel

06/23/2022, 1:50 PM
Hi Matthew - this is in Dagit where you're seeing this? Typically the stack trace would continue and there would be more information below (there should also be an error message above the stack trace)
Here's what I see in Dagit when an exception is raised within an op - is that different from what you're seeing?
m

Matthew Karas

06/23/2022, 2:46 PM
Yeah - this is all I get
Copy code
dagster.core.errors.DagsterExecutionStepExecutionError: Error occurred while executing op "<OP NAME HERE>":
The above exception was caused by the following exception:
RuntimeError: generator raised StopIteration

Stack Trace:
  File "/usr/local/lib/python3.9/site-packages/dagster/core/execution/plan/utils.py", line 47, in solid_execution_error_boundary
    yield
,  File "/usr/local/lib/python3.9/site-packages/dagster/utils/__init__.py", line 406, in iterate_with_context
    next_output = next(iterator)
OP NAME HERE is our internal code
d

daniel

06/23/2022, 2:46 PM
Ah OK - that error message "RuntimeError: generator raised StopIteration" is helpful, but also confusing
(helpful for me, that is)
m

Matthew Karas

06/23/2022, 2:47 PM
It would have been easier to know which line caused the exception.
I had to dig for that.
d

daniel

06/23/2022, 2:47 PM
Were you eventually able to figure out what the problem was?
There's something specifically unique about the error here that seems to be circumventing our usual error reporting
m

Matthew Karas

06/23/2022, 2:48 PM
Yes - a query didn't return any rows
d

daniel

06/23/2022, 2:49 PM
Any chance there's a minimal way to reproduce it that I could run myself without needing full access to your code?
In general, exceptions should be bubbled up and reported - something about the way your op handled the error case seems to be not handled correctly on our side
Is the op a generator?
Maybe we don't handle ops that are generators but don't yield anything well?
m

Matthew Karas

06/23/2022, 2:50 PM
Its a function that is called by an op - it wasn't actually a query that returned no rows... it was a next() call on an iterator statement
foo = next(f for f in foos if foo["bar"] == "baz")
d

daniel

06/23/2022, 2:51 PM
Ah, maybe we're not handling RuntimeErrors well
m

Matthew Karas

06/23/2022, 2:51 PM
Does everything need to be in an op?
Because I like to break down my code into small reusable functions.
d

daniel

06/23/2022, 2:52 PM
Ops can certainly call out to other functions - I might not totally understand the question
m

Matthew Karas

06/23/2022, 2:52 PM
You had it right.
ok
d

daniel

06/23/2022, 3:01 PM
OK, I can reproduce this if the op raises a StopIteration specifically (Other exceptions seem to include a stack trace). Gross! I'll file an issue, thanks for the report
oh weird, the inner stack trace with the useful information even appears in the dagit command-line output! This should be easy to fix then
m

Matthew Karas

06/23/2022, 3:48 PM
Thanks @daniel!
4 Views