Hey all so i have a dagster kubernetes instance and i was wo

Question

Hey all so i have a dagster kubernetes instance and i was wondering is it possible to let my instance execute on my local vm The use case is to run tests on code that haven t been deployed as images yet I wondered if this was possible through a combination of celery worker and api grpc sweat smile LMK if more clarification is needed

johann · Answer

Hi Charles you re looking to test your jobs locally

Charles Leung · Answer

yes slightly smiling face For a regression test for example the code is in git but i don t want to produce an artifact until the test passes

Charles Leung · Answer

so i have a docker image built with the git checked in code but i don t want to push to repository and deploy just want it to run in process in that image

johann · Answer

Ideally you shouldn t need to mess with Celery at all the executor and run launcher abstractions exist to help you run jobs locally or in production without changing business logic

johann · Answer

Of course if your job requires some credentials etc it can be some work to provide those or mock out those components

Charles Leung · Answer

the complexity comes from the fact that we are trying to avoid giving our build agent permissions to execute locally in CI disappointed

Charles Leung · Answer

hence i have to execute it remotely on a separate dagster instance

Charles Leung · Answer

maybe i m overcomplicated it i should just have a job to do a git checkout before executing on the image x

johann · Answer

Not sure I fully follow but some thoughts The best case would be if you could test your job in process possibly by mocking out the resource etc that you don t want available in your CI <https docs dagster io tutorial intro tutorial testable testing ops and jobs> If you do have to commit to launching jobs remotely doing so over the graphql api is likely the best option