Hello Dagster experts: DynamicOuts and execution ...
# ask-community
j
Hello Dagster experts: DynamicOuts and execution order: I have a job that looks like
Copy code
@graph()
def data_table_graph(table_name, overlapping_periods: List[str] = []):
    echo_data(table_name)
    echo_data(overlapping_periods)
    table_df = read_table_file(table_file_name=table_name)
    echo_data_frame(table_df)

@graph
def first_graph():
    table_file_list = build_table_list()
    table_file_list.map(lambda table_file: data_table_graph(table_file, overlapping_periods))
Today when I execute it, it executes build_table_list() -> echo_data(table1) -> echo_data(table2).... -> read_table_file(table1)... -> echo_data_frame(table1).... Is there a way to get it to run each data_table_graph in series rather than each subop? So something like build_table_list() -> echo_data(table1) -> read_table_file(table1) -> echo_data_frame(table1) ->echo_data(table2) -> read_table_file(table2) -> echo_data_frame(table2) Basically more like a for loop, we're moving from Luigi and confused about how to do this properly, we tried limiting concurrency already with a
Copy code
execution_config = {
        "config": {
            "multiprocess": {
                "max_concurrent": 1,
            },
        },
    }
    return RunConfig(ops=config_map, execution=execution_config)
o
hi @Jason Myers! I think this discussion might be along the lines of what you're looking for: https://github.com/dagster-io/dagster/discussions/8871
j
thank you...