Hello! Testing Q: Is there a way to call

job.execute_in_process()

in an async context? I'm trying to write a test for a sensor that terminates running jobs after some period of time (based off of this github answer). To test this, I was thinking of having a job that runs forever, while having the sensor periodically run in the background using threads. But when I try to do this, I get the following sqlite error:

(sqlite3.ProgrammingError) SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140180967843648 and this is thread id 140180440418048

The code roughly looks like:

_run_forever_job = {...}

def test_sensor():
    instance = DagsterInstance.ephemeral()
    context = build_sensor_context(instance=instance)
    def periodically_run_sensor(context):
            while True:
                job_timeout_terminator_sensor(context)
                time.sleep(0.25)

    thread = threading.Thread(target=periodically_run_sensor, args=[context])
    thread.start()
    _run_forever_job.execute_in_process(instance=instance)

I also tried running

execute_in_process

in a thread, with the same result. Is there a better way to write a test for this?

Hi Jeremy - what version of Dagster are you using?

I'm using

dagster = "1.1.14"

right now.

Can you try upgrading? I think a fix for this may have gone out in the last couple of weeks

Will do and report back. Thanks for the tip.

Ah, that's true, but as per the comment here that launcher is only intended for testing: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster/dagster/_core/launcher/sync_in_memory_run_launcher.py#L9