Solaris Wang
07/08/2022, 9:50 PM@op
def return_b(arg):
return arg + "b"
def return_a(arg):
return arg + "a"
def c(init_arg='hello', fn=(return_a, return_b)):
next_step = None
for next_step in fn:
prev_step=next_step or init_arg
next_step(prev_step)
# expected_output = return_a(return_b('hello'))
and here’s my sad attempt at creating an op_factory that’ll give me an iterable with the functions, although how i’ll string it together in a job is beyond me atm
def op_factory(steps) -> OpDefinition:
for step in steps:
if step[:6] == 'import':
@op(name=step, **kw)
def importer():
print("import stuff")
yield importer
elif step == 'parquet':
@op(name=step, **kw)
def to_parquet():
print("transform stuff")
yield to_parquet
prha
07/08/2022, 9:55 PMSolaris Wang
07/08/2022, 9:58 PMprha
07/08/2022, 10:00 PMSolaris Wang
07/08/2022, 10:01 PMprha
07/08/2022, 10:01 PMSolaris Wang
07/08/2022, 10:05 PMprha
07/08/2022, 10:06 PMSolaris Wang
07/08/2022, 10:07 PMprha
07/08/2022, 10:08 PMSolaris Wang
07/08/2022, 10:08 PMprha
07/11/2022, 3:18 PMSolaris Wang
07/11/2022, 3:35 PMprha
07/11/2022, 3:59 PMSolaris Wang
07/11/2022, 7:59 PMprha
07/11/2022, 8:05 PMSolaris Wang
07/11/2022, 10:32 PMprha
07/11/2022, 11:28 PM