I'm trying to upgrade from 1.13.13 to 1.4.10, but ...
# ask-community
z
I'm trying to upgrade from 1.13.13 to 1.4.10, but I'm getting an error on loading my code into Dagster. It seems to think an op is being invoked when used in a graph and is giving me the no context error.
Copy code
Stack Trace:
  File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_grpc/server.py", line 290, in __init__
    self._loaded_repositories: Optional[LoadedRepositories] = LoadedRepositories(
  File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_grpc/server.py", line 134, in __init__
    loadable_targets = get_loadable_targets(
  File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_grpc/utils.py", line 37, in get_loadable_targets
    else loadable_targets_from_python_file(python_file, working_directory)
  File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/workspace/autodiscovery.py", line 26, in loadable_targets_from_python_file
    loaded_module = load_python_file(python_file, working_directory)
  File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/code_pointer.py", line 83, in load_python_file
    return import_module_from_path(module_name, python_file)
  File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_seven/__init__.py", line 49, in import_module_from_path
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "etxlib/etxdagster/repo.py", line 58, in <module>
    from etxlib.etxdagster.pipelines.genetic_variant_processing.concrete import (
  File "/Users/zach/Documents/empirico/projects/etxlib/etxlib/etxdagster/pipelines/genetic_variant_processing/concrete.py", line 5, in <module>
    from etxlib.etxdagster.pipelines.genetic_variant_processing.quality_control import (
  File "/Users/zach/Documents/empirico/projects/etxlib/etxlib/etxdagster/pipelines/genetic_variant_processing/quality_control.py", line 262, in <module>
    def run_quality_control_on_genetic_data():
  File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/definitions/decorators/graph_decorator.py", line 79, in __call__
    ) = do_composition(
  File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/definitions/composition.py", line 1035, in do_composition
    output = fn(**kwargs)
  File "/Users/zach/Documents/empirico/projects/etxlib/etxlib/etxdagster/pipelines/genetic_variant_processing/quality_control.py", line 263, in run_quality_control_on_genetic_data
    summary = generate_file_summary_table(obtain_all_file_metadata())
  File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/definitions/assets.py", line 310, in __call__
    return direct_invocation_result(self, *args, **kwargs)
  File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/definitions/op_invocation.py", line 151, in direct_invocation_result
    raise DagsterInvalidInvocationError(
Here's the usage of
generate_file_summary
, which it the function it seems to be choking on:
Copy code
@graph(name="conduct_genetic_data_quality_control")
def run_quality_control_on_genetic_data():
    summary = generate_file_summary_table(obtain_all_file_metadata())
    compare_samples_across_chromosomes(obtain_filenames(summary))
obtain_all_file_metadata
is just a normal op:
Copy code
@op
def obtain_all_file_metadata(context: OpExecutionContext) -> list:
  ...
What changed in between 1.3.13 and 1.4.10 that would make a graph definition like this fail?
a
is
generate_file_summary_table
an
@asset
?
z
Ah it does appear to be an asset. Good catch. This loads with 1.3.x - Dagster must just treat it as an op in that and previous versions?
a
ya the PR linked above links the blame rev that broke it that went out in 1.4.5
z
Got it, thanks!