Trying to follow this doc

Question

Trying to follow this doc <https docs dagster io guides dagster airbyte ingestion as code> great writeup by the way however i m not sure how to configure the values under `Streams` in the Airbyte UI Destination Namespace Namespace Custom Format Destination Stream Prefix Tried looking through the docs code but wasn t very clear to me

rex · Answer

I suggest that you repost your question in < C047VPGQUTV|dagster airbyte>

ben · Answer

Hi < sar> you can configure the destination namespace as follows ```AirbyteConnection destination namespace=AirbyteDestinationNamespace SAME AS SOURCE ``` valid values are `AirbyteDestinationNamespace SAME AS SOURCE` `AirbyteDestinationNamespace DESTINATION DEFAULT` or a custom string the same format that you would use in the Airbyte UI

ben · Answer

Right now we don t have support for destination prefixes but this PR <https github com dagster io dagster pull 11057> should enable you to set them as an optional argument for a connection Hopefully this will be out with this week s release

ben · Answer

I m noticing now that the autogenerated `AirbyteConnection` API docs don t show the constructor s documentation will push a fix for that but here is a link to those docs <https dagster fqt9qd90e elementl vercel app apidocs libraries dagster airbyte dagster airbyte AirbyteConnection>

sar · Answer

Thanks Ben for all the info

sar · Answer

< ben> Am I correct in assuming when specifying an `AirbyteConnection` that a `stream config` needs to be specified as well The reason I ask because if you re setting up a db sync you d have to get a list of tables in that db and specify each one of those streams which would be quite tedious if you want to sync it all Additionally if you end up adding tables you d have to go back to your code and add those streams as well Am I missing something

ben · Answer

Yes right now we require that users explicitly provide stream config One option could be for us to discover all streams at apply time so that users could just say to sync all streams with a particular sync mode It s my understanding that Airbyte <https github com airbytehq airbyte issues 12089|doesn t support detecting enabling new tables by default> I think the best we could do here would be to try to reread the schema and enable the new stream on apply

sar · Answer

It would be great to discover all streams at apply time and sync them with a particular sync mode to start with