Hi guys, I am trying to use AIrbyte integration as code functionality and I understand that you need to do:

dagster-airbyte check --module my_python_module.my_submodule:reconciler
dagster-airbyte apply --module my_python_module.my_submodule:reconciler

In order to check and apply changes. However, I would like to do this from within the code. Which is fine I can do

if not reconciler.check().is_empty(): reconciler.apply()

This works ok if you just run

dagit

But it breaks if you run

dagster dev

. Because the code is being executed both by

dagster

dev at the start time and

dagit

which causes memory leak. Is there a recommended way of doing this?

Hi Arman, I would not recommend running this code as part of loading your Dagster project since it will run the (potentially computationally expensive) reconciliation every time you start a run or reload your code location. I would instead recommend either running the CLI or a Python script to run

reconciler.apply()

as part of CI or as a standalone step.