jamie
02/27/2023, 4:30 PMjamie
02/27/2023, 4:30 PMdagster._core.errors.DagsterExecutionHandleOutputError: Error occurred while handling output "result" of step "iris_data":
File "/opt/homebrew/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_plan.py", line 269, in dagster_event_sequence_for_step
for step_event in check.generator(step_events):
File "/opt/homebrew/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 386, in core_dagster_event_sequence_for_step
for evt in _type_check_and_store_output(step_context, user_event):
File "/opt/homebrew/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 439, in _type_check_and_store_output
for evt in _store_output(step_context, step_output_handle, output):
File "/opt/homebrew/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 640, in _store_output
for elt in iterate_with_context(
File "/opt/homebrew/lib/python3.10/site-packages/dagster/_utils/__init__.py", line 469, in iterate_with_context
with context_fn():
File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py", line 153, in __exit__
self.gen.throw(typ, value, traceback)
File "/opt/homebrew/lib/python3.10/site-packages/dagster/_core/execution/plan/utils.py", line 85, in op_execution_error_boundary
raise error_cls(
The above exception was caused by the following exception:
AttributeError: 'NoneType' object has no attribute 'write'
File "/opt/homebrew/lib/python3.10/site-packages/dagster/_core/execution/plan/utils.py", line 55, in op_execution_error_boundary
yield
File "/opt/homebrew/lib/python3.10/site-packages/dagster/_utils/__init__.py", line 471, in iterate_with_context
next_output = next(iterator)
File "/opt/homebrew/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 630, in _gen_fn
gen_output = output_manager.handle_output(output_context, output.value)
File "/opt/homebrew/lib/python3.10/site-packages/dagster/_core/storage/db_io_manager.py", line 139, in handle_output
self._handlers_by_type[obj_type].handle_output(context, table_slice, obj, conn)
File "/opt/homebrew/lib/python3.10/site-packages/dagster_gcp_pandas/bigquery/bigquery_pandas_type_handler.py", line 41, in handle_output
pandas_gbq.to_gbq(
File "/opt/homebrew/lib/python3.10/site-packages/pandas_gbq/gbq.py", line 1220, in to_gbq
connector.load_data(
File "/opt/homebrew/lib/python3.10/site-packages/pandas_gbq/gbq.py", line 614, in load_data
chunks = tqdm.tqdm(chunks)
File "/opt/homebrew/lib/python3.10/site-packages/tqdm/std.py", line 1109, in __init__
self.refresh(lock_args=self.lock_args)
File "/opt/homebrew/lib/python3.10/site-packages/tqdm/std.py", line 1361, in refresh
self.display()
File "/opt/homebrew/lib/python3.10/site-packages/tqdm/std.py", line 1509, in display
self.sp(self.__str__() if msg is None else msg)
File "/opt/homebrew/lib/python3.10/site-packages/tqdm/std.py", line 350, in print_status
fp_write('\r' + s + (' ' * max(last_len[0] - len_s, 0)))
File "/opt/homebrew/lib/python3.10/site-packages/tqdm/std.py", line 343, in fp_write
fp.write(_unicode(s))
File "/opt/homebrew/lib/python3.10/site-packages/tqdm/utils.py", line 89, in __getattr__
return getattr(self._wrapped, name)
Does the io manager create the dataset and table automatically?
Also I adjusted bellow so i can use these as the column headers in Bigquery:
names=[
"sepal_length_cm",
"sepal_width_cm",
"petal_length_cm",
"petal_width_cm",
"species",
],
jamie
02/27/2023, 4:34 PMgoogle auth …
or have GOOGLE_APPLICATION_CREDENTIALS
set
2. how are you running the code - ie materializing via the UI, running via the CLI, etc
3. can you update the asset to look like this just to confirm that the dataset is being downloaded correctly?
@asset
def iris_data() -> pd.DataFrame:
df = pd.read_csv(
"<https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data>",
names=[
"sepal_length_cm",
"sepal_width_cm",
"petal_length_cm",
"petal_width_cm",
"species",
],
)
print(df)
return df
Benedikt Buchert
02/27/2023, 5:59 PMBenedikt Buchert
02/27/2023, 6:00 PM