Aki Iwa
08/26/2023, 5:36 AMIndexError: list index out of range error
occurred while reading configuration values in yaml.
The details of the error are described in the thread.
The error is said to be caused by loading a yaml file, but usually the yaml file is loaded and executed without any problems.
I did not make any changes to the file when the error occurred.
I am loading the yaml with load_yaml_from_path
, could there be a problem with this module?
The version is below. I have omitted some modules that I think are irrelevant.
psutil==5.6.3
PyYAML==6.0
pydantic==1.10.6
gspread==4.0.0
boto3==1.17.14
dagster==1.3.7
dagster-dbt==0.19.7
dagster-cloud==1.3.7
dagster-aws
Aki Iwa
08/26/2023, 5:37 AMdagster._core.definitions.events.Failure: remove_rds is failure.
File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/execute_plan.py", line 262, in dagster_event_sequence_for_step
for step_event in check.generator(step_events):
File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/execute_step.py", line 369, in core_dagster_event_sequence_for_step
for user_event in check.generator(
File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/execute_step.py", line 90, in _step_output_error_checked_user_event_sequence
for user_event in user_event_sequence:
File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/compute.py", line 192, in execute_core_compute
for step_output in _yield_compute_results(step_context, inputs, compute_fn):
File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/compute.py", line 161, in _yield_compute_results
for event in iterate_with_context(
File "/usr/local/lib/python3.11/site-packages/dagster/_utils/__init__.py", line 445, in iterate_with_context
next_output = next(iterator)
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/compute_generator.py", line 124, in _coerce_op_compute_fn_to_iterator
result = invoke_compute_fn(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/dagster/_core/execution/plan/compute_generator.py", line 118, in invoke_compute_fn
return fn(context, **args_to_pass) if context_arg_provided else fn(**args_to_pass)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/danballone_etl/ops/op.py", line 76, in remove_rds
raise Failure(
The above exception occurred during handling of the following exception:
IndexError: list index out of range
File "/root/danballone_etl/ops/op.py", line 74, in remove_rds
run_result = run_ecs("/root/danballone_etl/ops/bq_elt.yml", exec_time, 2)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/danballone_etl/ops/run_task.py", line 118, in run_ecs
arn = response["tasks"][0]["taskArn"]
~~~~~~~~~~~~~~~~~^^^