https://dagster.io/ logo
#ask-community
Title
# ask-community
a

Andrei Duralei

12/19/2022, 2:12 PM
Hi everyone! Is it possible to dagster not to skip op if some inputs are optional and not provided?
🤖 1
For example:
Copy code
@op(out={"output1": Out(is_required=True), "output2": Out(is_required=False)})
def op1():
  if something:
    yield Output()
  else:
    yield Output()
    yield Output()

@op
def op2(output1: str, output2: Optional[str] = None):
  #do something

@graph
def my_graph():
  output1, output2 = op1()
  op2(output1, output2)
op2 will not trigger even if output2 is optional parameter for this op and not being returned from op1
a

Adam Ward

12/19/2022, 2:49 PM
Have you tried setting the default value for
output2
to an empty string instead (
''
)? This is only a guess but I suspect that op2 does not trigger as it does not have a value for all dependencies.
a

Andrei Duralei

12/19/2022, 2:56 PM
It doesn’t work too (I mean empty string). It has default value None for output2 and output1 is returned by op1.
a

Adam Ward

12/19/2022, 3:01 PM
I personally deal more in assets than ops. You may want to consider using an asset for
op2
and defining a non-argument dependency. If I'm understanding what you are looking for, that should allow you to execute
op2
without waiting for input from
output2
.
a

Andrei Duralei

12/19/2022, 3:12 PM
Actually, I need output1 or output2 from op1 to start processing op2. I think it not my case 😞
a

Adam Ward

12/19/2022, 3:16 PM
Oh ok. What if you passed the return value from op1 as a tuple with the second value defaulting to an empty string? This would ensure that op2 always receives the dependent value, triggering a run.
Copy code
@op(out={"output": Out(is_required=True)})
def op1():
  if something:
    yield (Output(), '')
  else:
    yield (Output(), Output())

@op
def op2(output: tuple):
  #do something
Think that might work?
Full disclosure: I cannot test this right now so I apologise if these suggestions miss the mark. Just brainstorming out loud with you here. 🙂
a

Andrei Duralei

12/19/2022, 3:24 PM
I think it should work. I was thinking about it as last solution, because it will require for me to check and skip ops between op1 and op2 by my self. This example is just to simplify my request.
a

Adam Ward

12/19/2022, 3:29 PM
Ah - It's never quite as simple as we hope, is it? 😉
😅 1
a

Andrei Duralei

12/19/2022, 3:29 PM
For sure!
a

Adam Ward

12/19/2022, 3:29 PM
Best of luck! ☘️
🙏 1