Hello :wave: It looks like the SQLite connection ...
# ask-community
f
Hello đź‘‹ It looks like the SQLite connection used by Dagster to log events is not thread-safe. The connection is created in one thread and is being used in another thread, causing the following error message :
Copy code
sqlalchemy.exc.ProgrammingError: (sqlite3.ProgrammingError) SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140024476800832 and this is thread id 140024381171456.
(Background on this error at: <https://sqlalche.me/e/14/f405>)
Here's how to reproduce the error:
Copy code
import concurrent.futures

from dagster import get_dagster_logger, graph, op


def my_function(x):
    logger = get_dagster_logger()
    <http://logger.info|logger.info>(f"Got {x=}")
    return x + 1


@op
def my_op():
    with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
        results = list(executor.map(my_function, range(3)))
    print(results)
    return results


@graph
def my_graph() -> None:
    my_op()


def main() -> None:
    my_graph.execute_in_process()


if __name__ == "__main__":
    main()
I was not able to fix the issue, unless I remove logging from
my_function
(but I need logging), or use a ProcessPoolExecutor (which in my case is significantly less performant than ThreadPoolExecutor)
I am on the latest version of dagster (1.1.18)
o
hi @FĂ©lix Tremblay! As of the version of dagster released yesterday (1.1.19), you shouldn't see this error anymore
❤️ 1
big dag eyes 1
f
Thank you @owen!
Hello @owen, after upgrading to 1.1.19, I still get the same error
a
Looks like the fix landed just after the branch cut so will actually be out in this weeks release on Thursday.
🎉 1
f
thank you! it works