Zach
08/30/2023, 12:38 AMStack Trace:
File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_grpc/server.py", line 290, in __init__
self._loaded_repositories: Optional[LoadedRepositories] = LoadedRepositories(
File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_grpc/server.py", line 134, in __init__
loadable_targets = get_loadable_targets(
File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_grpc/utils.py", line 37, in get_loadable_targets
else loadable_targets_from_python_file(python_file, working_directory)
File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/workspace/autodiscovery.py", line 26, in loadable_targets_from_python_file
loaded_module = load_python_file(python_file, working_directory)
File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/code_pointer.py", line 83, in load_python_file
return import_module_from_path(module_name, python_file)
File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_seven/__init__.py", line 49, in import_module_from_path
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "etxlib/etxdagster/repo.py", line 58, in <module>
from etxlib.etxdagster.pipelines.genetic_variant_processing.concrete import (
File "/Users/zach/Documents/empirico/projects/etxlib/etxlib/etxdagster/pipelines/genetic_variant_processing/concrete.py", line 5, in <module>
from etxlib.etxdagster.pipelines.genetic_variant_processing.quality_control import (
File "/Users/zach/Documents/empirico/projects/etxlib/etxlib/etxdagster/pipelines/genetic_variant_processing/quality_control.py", line 262, in <module>
def run_quality_control_on_genetic_data():
File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/definitions/decorators/graph_decorator.py", line 79, in __call__
) = do_composition(
File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/definitions/composition.py", line 1035, in do_composition
output = fn(**kwargs)
File "/Users/zach/Documents/empirico/projects/etxlib/etxlib/etxdagster/pipelines/genetic_variant_processing/quality_control.py", line 263, in run_quality_control_on_genetic_data
summary = generate_file_summary_table(obtain_all_file_metadata())
File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/definitions/assets.py", line 310, in __call__
return direct_invocation_result(self, *args, **kwargs)
File "/Users/zach/Documents/empirico/projects/etxlib/.venvs/hailenv109/lib/python3.9/site-packages/dagster/_core/definitions/op_invocation.py", line 151, in direct_invocation_result
raise DagsterInvalidInvocationError(
Here's the usage of generate_file_summary
, which it the function it seems to be choking on:
@graph(name="conduct_genetic_data_quality_control")
def run_quality_control_on_genetic_data():
summary = generate_file_summary_table(obtain_all_file_metadata())
compare_samples_across_chromosomes(obtain_filenames(summary))
obtain_all_file_metadata
is just a normal op:
@op
def obtain_all_file_metadata(context: OpExecutionContext) -> list:
...
What changed in between 1.3.13 and 1.4.10 that would make a graph definition like this fail?alex
08/30/2023, 2:45 PMgenerate_file_summary_table
an @asset
?alex
08/30/2023, 3:12 PMZach
08/30/2023, 3:22 PMalex
08/30/2023, 3:28 PMZach
08/30/2023, 3:44 PM