Hey. I have a bunch of Airbyte streams, all going ...
# integration-airbyte
c
Hey. I have a bunch of Airbyte streams, all going to S3, prefixed w/
$NAME/
(note the
/
), so the stream name winds up being something like
e2e/data
. When I use
construct_airbyte_assets
and set _`destination_tables`_ as just
data
, it'll fail with
DagsterStepOutputNotFoundError
. If set
destination_tables
to
e2e/data
, it'll fail with
"e2e/data" is not a valid name in Dagster. Names must be in regex ^[A-Za-z0-9_]+$
. The _`asset_key_prefix`_ only prefixes the asset w/in Dagster, but still doesn't find any valid output from the source. What am I missing - is there not a way to map the source Airbyte stream name into a valid Dagster asset name?
🤖 1
s
Thanks for the PR, our dagster-airbyte expert will hopefully get to it next week
c
Just wanted to follow up on that PR - any chance I could get a review there?
s
cc @ben