Good morning! Does anyone know how to force a gRPC...
# ask-community
a
Good morning! Does anyone know how to force a gRPC repository to reload? I add new jobs dynamically following this example, but the new jobs don't show up unless I restart the code server (reloading via Dagit does nothing).
d
Hi Anthony - is it possible to share your code? Do you mean that you made your own subclass of RepositoryData?
s
Maybe you can make a request to the same endpoint that dagit UI use
a
I subclassed
RepositoryData
yes. Please ignore the mess in
_construct_job_def_from_filename
Copy code
class DynamicRepo(RepositoryData):
    def __init__(self, yaml_filename: str):
        yaml_filepath = Path(__file__).parent / yaml_filename
        assert yaml_filepath.is_file()
        self._yaml_filepath = yaml_filepath
        self._filenames = self._update_filenames_from_yaml()

    def get_all_pipelines(self):
        return [self._construct_job_def_from_filename(fn) for fn in self._filenames]

    def _construct_job_def_from_filename(self, filename: str):
        module_name = filename
        sys.path.insert(0, str(Path(__file__).parent))  # Modify path to find local imports
        module = importlib.import_module(module_name, 'jobs')
        job_name = filename.split('.')[1] + '_job'
        return module.__dict__[job_name]

    def _update_filenames_from_yaml(self) -> list[str]:
        with open(self._yaml_filepath, "r") as f:
            yml = yaml.safe_load(f)
            if "generated_jobs" in yml.keys():
                return yml["generated_jobs"]
@Saul Burgos The problem isn't making the request; the problem is (i'm speculating) that calling
ReloadWorkspaceMutation
only asks the gRPC server if it's there, but doesn't trigger an internal reload.
j
where are you using your DynamicRepo?
a
@Jean Gonzalez `DynamicRepo`is located inside
repo.py
on my local machine, then hosted on a grpc server with the following args:
Copy code
api grpc --python-file /parent/path/repo.py --working-directory /parent/path --host 0.0.0.0 --port 4266
d
Anthony just to confirm, you also have an @repository that returns a DynamicRepo i assume
It won't re-create your DynamicRepo when you press reload in dagit, but it should call get_all_pipelines again
so you may need to move some logic out of the constructor
❤️ 1
a
OH you're right! I cache the job-list in the constructor smh
j
Hi @Anthony Reksoatmodjo Were you able to load new dynamically created jobs while using reload? I had similar issue that based on some condition new jobs should be created but they only shows when I restart the repository server. In logs I can see that the jobs are created, but somehow their definition is not visible.
a
@Jakub Zgrzebnicki Once i fixed the bug, yes, i was able to use the ReloadRepository graphql endpoint to reload the code server and see the new jobs. I didn't have to restart the entire GRPC server anymore