https://dagster.io/ logo
Join Slack
Powered by
I wanted to run something by here before submittin...
# ask-community
a
I wanted to run something by here before submitting bug report. Currently on 
1.3.11
. We are using the 
k8s_job_executor
. We have one asset that has no upstream inputs. However, it fails with 
$DAGSTER_HOME "/opt/dagster/dagster_home" is not a directory or does not exist.
Since the other assets aren't failing, I'm guessing something about not having an input is causing the dagster home directly to not be created on the 
step
container. Full traceback in 🧵
Copy code
dagster._core.errors.DagsterInvariantViolationError: $DAGSTER_HOME "/opt/dagster/dagster_home" is not a directory or does not exist. Dagster requires this environment variable to be set to an existing directory in your filesystem

  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/execution/plan/execute_plan.py", line 273, in dagster_event_sequence_for_step
    for step_event in check.generator(step_events):
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/execution/plan/execute_step.py", line 369, in core_dagster_event_sequence_for_step
    for user_event in check.generator(
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/execution/plan/execute_step.py", line 90, in _step_output_error_checked_user_event_sequence
    for user_event in user_event_sequence:
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/execution/plan/compute.py", line 192, in execute_core_compute
    for step_output in _yield_compute_results(step_context, inputs, compute_fn):
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/execution/plan/compute.py", line 161, in _yield_compute_results
    for event in iterate_with_context(
  File "/usr/local/lib/python3.11/dist-packages/dagster/_utils/__init__.py", line 445, in iterate_with_context
    next_output = next(iterator)
                  ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/execution/plan/compute_generator.py", line 124, in _coerce_op_compute_fn_to_iterator
    result = invoke_compute_fn(
             ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/execution/plan/compute_generator.py", line 118, in invoke_compute_fn
    return fn(context, **args_to_pass) if context_arg_provided else fn(**args_to_pass)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/adam_etl/dag/assets/rl_report.py", line 38, in risk_list_summary
    with defs.get_asset_value_loader() as loader:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/definitions/definitions_class.py", line 476, in get_asset_value_loader
    return self.get_repository_def().get_asset_value_loader(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/definitions/repository_definition/repository_definition.py", line 340, in get_asset_value_loader
    return AssetValueLoader(
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/storage/asset_value_loader.py", line 46, in __init__
    self._instance = self._exit_stack.enter_context(DagsterInstance.get())
                                                    ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/instance/__init__.py", line 508, in get
    raise DagsterInvariantViolationError(
s
based on that stack trace, it looks like it's happening when you're 
get_asset_value_loader
, which is failing because it expects there to be a DAGSTER_HOME that points to an existing directory
a
Yes, I'm not sure why it is calling get_asset_loader when there are no input assets, so I'm assuming it just loads that as part of the normal asset initialization. Question is why it fails on this asset and not others, and I can only assume it is the combination of 
k8s_job_executor
+ the lack of inputs in this asset is causing some directory initialization to not occur and so the container for the single 
step
inside the asset doesn't get the default dagster home directory.
s
Yes, I'm not sure why it is calling get_asset_loader when there are no input assets, so I'm assuming it just loads that as part of the normal asset initialization.
based on the stack trace, I believe it's getting called in your code?
Copy code
File "/code/adam_etl/dag/assets/rl_report.py", line 38, in risk_list_summary
    with defs.get_asset_value_loader() as loader:
a
Yes, that was it, we had to load assets somewhat dynamically in this. So apparently initialization of the dagster home directory happens if you have an asset specified as an input, but not if you don't.
5 Views
Open in Slack
PreviousNext
Powered by