hello dagsters, how can i do the “python logging capturing” described HERE with an in-process execution? i try executing a job using

.execute_in_process()

, but i get no captured python logs. when i start the job using

dagster job execute

it works.

Hi @Axel Bock do you have some code snippets you can share? any output you're getting that shows the log capture working with

dagster job execute

and not working with

execute_in_process

would also be helpful!

hi @Axel Bock, I believe what's happening is that by default, when you use .execute_in_process(), an in-memory dagster instance will be created to store the logs/events produced by that run. This dagster instance will not have any of the configurations set on your main dagster instance, and will disappear after the run completes. If you want the logs captured and stored in your default dags\ter instance (the one that's configured with that dagster.yaml file), you can do this with

.execute_in_process(instance=DagsterInstance.get())

. If you want the logs captured, but stored to a different, temporary dagster instance, you can do this with

.execute_in_process(instance=DagsterInstance.local_temp(overrides={"python_logs": <whatever config>})