Good morning! Does anyone know how to force a gRPC repository to reload? I add new jobs dynamically following this example, but the new jobs don't show up unless I restart the code server (reloading via Dagit does nothing).

Hi Anthony - is it possible to share your code? Do you mean that you made your own subclass of RepositoryData?

Maybe you can make a request to the same endpoint that dagit UI use

I subclassed

RepositoryData

yes. Please ignore the mess in

_construct_job_def_from_filename

class DynamicRepo(RepositoryData):
    def __init__(self, yaml_filename: str):
        yaml_filepath = Path(__file__).parent / yaml_filename
        assert yaml_filepath.is_file()
        self._yaml_filepath = yaml_filepath
        self._filenames = self._update_filenames_from_yaml()

    def get_all_pipelines(self):
        return [self._construct_job_def_from_filename(fn) for fn in self._filenames]

    def _construct_job_def_from_filename(self, filename: str):
        module_name = filename
        sys.path.insert(0, str(Path(__file__).parent))  # Modify path to find local imports
        module = importlib.import_module(module_name, 'jobs')
        job_name = filename.split('.')[1] + '_job'
        return module.__dict__[job_name]

    def _update_filenames_from_yaml(self) -> list[str]:
        with open(self._yaml_filepath, "r") as f:
            yml = yaml.safe_load(f)
            if "generated_jobs" in yml.keys():
                return yml["generated_jobs"]

where are you using your DynamicRepo?

@Jean Gonzalez `DynamicRepo`is located inside

repo.py

on my local machine, then hosted on a grpc server with the following args:

api grpc --python-file /parent/path/repo.py --working-directory /parent/path --host 0.0.0.0 --port 4266

Anthony just to confirm, you also have an @repository that returns a DynamicRepo i assume

OH you're right! I cache the job-list in the constructor smh

Hi @Anthony Reksoatmodjo Were you able to load new dynamically created jobs while using reload? I had similar issue that based on some condition new jobs should be created but they only shows when I restart the repository server. In logs I can see that the jobs are created, but somehow their definition is not visible.

@Jakub Zgrzebnicki Once i fixed the bug, yes, i was able to use the ReloadRepository graphql endpoint to reload the code server and see the new jobs. I didn't have to restart the entire GRPC server anymore