Hi, I have several queries defined as assets, all ...
# ask-community
p
Hi, I have several queries defined as assets, all under the same group_name and I use define_asset_job() and AssetSelection to define a job for them. I would like to sequence a specific group of slow query assets, so that they run one-by-one instead of in parallel. E.g. Query2 runs after Query1 finishes, but Query2 doesn't depend on Query1 and Query2 should still run if Query1 fails. What's the best way to accomplish this?
🤖 1
Seems like I needed to configure an executor like: default_executor_def=multiprocess_executor.configured({"max_concurrent": 1}) . This seemed to work when defined for the repository, but not when defining individual executors for each job.
s
That's surprising to me that it didn't work for the job. How did you launch the job?
p
This worked:
Copy code
@repository(default_executor_def=multiprocess_executor.configured({"max_concurrent": 1}))
def a_repo():
    return [
        *with_resources(
            asset_list,
            resource_defs={
                "db_resource": db_resource,
            },
        ),
        define_asset_job(
            name="a_job",
            selection=AssetSelection.groups("query_group"),
        ),
    ]
This didn't work, and seemed to have the default 4 concurrent:
Copy code
@repository
def a_repo():
    return [
        *with_resources(
            asset_list,
            resource_defs={
                "db_resource": db_resource,
            },
        ),
        define_asset_job(
            name="a_job",
            selection=AssetSelection.groups("query_group"),
            executor_def=multiprocess_executor.configured({"max_concurrent": 1})
        ),
    ]
And I launched in UI by going to the job and clicking the "materialize" button.
s
ah - I believe this is due to a bug with that button. if you go to the job and go to the Launchpad and click "Launch run", I would expect this to work
p
@sandy Thanks for the help! It worked from Launchpad and scheduled.