Hi getting an error with the fivtran syncs:

dagster._core.errors.DagsterExecutionHandleOutputError: Error occurred while handling output "dbt_source_google_ads_1234_account_history" of step "fivetran_sync_zen_naughty":
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_plan.py", line 269, in dagster_event_sequence_for_step
    for step_event in check.generator(step_events):
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_step.py", line 386, in core_dagster_event_sequence_for_step
    for evt in _type_check_and_store_output(step_context, user_event):
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_step.py", line 439, in _type_check_and_store_output
    for evt in _store_output(step_context, step_output_handle, output):
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_step.py", line 640, in _store_output
    for elt in iterate_with_context(
  File "/usr/local/lib/python3.8/site-packages/dagster/_utils/__init__.py", line 473, in iterate_with_context
    return
  File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/utils.py", line 85, in op_execution_error_boundary
    raise error_cls(
The above exception was caused by the following exception:
dagster._check.CheckError: Invariant failed. Description: Unexpected 'None' output value. If a 'None' value is intentional, set the output type to None.
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/utils.py", line 55, in op_execution_error_boundary
    yield
  File "/usr/local/lib/python3.8/site-packages/dagster/_utils/__init__.py", line 471, in iterate_with_context
    next_output = next(iterator)
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_step.py", line 630, in _gen_fn
    gen_output = output_manager.handle_output(output_context, output.value)
  File "/usr/local/lib/python3.8/site-packages/dagster/_core/storage/db_io_manager.py", line 143, in handle_output
    check.invariant(
  File "/usr/local/lib/python3.8/site-packages/dagster/_check/__init__.py", line 1684, in invariant
    raise CheckError(f"Invariant failed. Description: {desc}")

How can I make sure fivetran syncs can return none?

fivetran_assets = load_assets_from_fivetran_instance(
    fivetran=fivetran_instance,
    key_prefix=["dbt_source"],
    connector_to_asset_key_fn=connector_to_asset_key_fn,
    connector_filter=connector_filter,
    io_manager_key=None
)

Hi Benedikt - what IO manager are you providing (if any) as your default IO manager? That’s what is defaulted to in this case with

io_manager_key=None

I added the new bigquery_pandas io manager as the default one to the project. I thought I was removing that io Manager by setting it to none. Do I need to provide an io manager? Fivetran is writing to bigquery but dagster does not have any control over were fivetran writes the data to or does it?

I think what we’re doing behind the scenes here is not well telegraphed - specifying

None

is equivalent to telling Dagster to use the default IO manager, which in this case is the bigquery_pandas IO manager. You’re right that Fivetran is writing to Bigquery rather than Dagster, but Dagster will still trigger the IO manager with its output. One quick workaround is to provide an alternate IO manager:

from dagster import fs_io_manager

...

fivetran_assets = load_assets_from_fivetran_instance(
    fivetran=fivetran_instance,
    key_prefix=["dbt_source"],
    connector_to_asset_key_fn=connector_to_asset_key_fn,
    connector_filter=connector_filter,
    io_manager_key="noop_io_manager"
)

...

resources = {
    "noop_io_manager": fs_io_manager
}

Thank you. Out of interest. What does the file system io manager do with the output of fivetran?

It’s not the actual fivetran sync data (since fivetran rather than Dagster handles that output). In this case it’s just some metadata about the sync which gets serialized into a file

So what does the snowflake io manager do with that data just disregard it? https://docs.dagster.io/integrations/fivetran#step-3-adding-downstream-assets

Ahh I see. The intent here is to use the IO manager to load the data in a downstream asset, but not to use the IO manager to store the output data (since Fivetran itself is doing that). I think that the error you’re seeing here is actually an issue with compatibility between this particular I/O manager and the Fivetran integration (the I/O manager expects output and isn’t getting any). It should be a quick fix: https://github.com/dagster-io/dagster/pull/12560

Got it thank you 🙏