Hi all, currently evaluating airbyte ingestion as ...
# ask-community
Hi all, currently evaluating airbyte ingestion as code part. Why was
endpoint chosen instead of
to be used here: https://github.com/dagster-io/dagster/blob/e625673a057129437df64ca4728c087a8d8cb30[…]_modules/libraries/dagster-airbyte/dagster_airbyte/resources.py It doesn't return non-public connectors, which aren't really a thing if you host airbyte yourself and have access to all of the connectors anyways. Which is causing problems for us. Is this on purpose? Are we missing anything? I could submit a pull request that changes the endpoint, if it's ok.
cc @ben
Hi Emilja, I think it should be fine to make this change, and I’d be happy to approve it if PRed in. I’m curious what the difference in behavior you’re experiencing is - certain connectors aren’t visible on one endpoint?
Hi, Ben. Yes, that's exactly it, certain connectors aren't visible on that endpoint (for example, hubspot). There's little in airbyte documentation but looking at the code, it looks like there supposed to be a way to limit certain connectors for a workspace. If I'm not mistaken, connectors that aren't public (it's how a field is called, I guess it means that the connectors aren't officially supported by airbyte team) should be explicitly enabled. However at least for us, the open source airbyte's UI can access and use all of the connectors. 🤷‍♀️
@ben https://github.com/dagster-io/dagster/pull/12012 let me know if I need to change anything
Hi Emilja, I’ve left a few quick changes on the PR, but looks good to me! I think we can get this in for this week’s release
Hi, that's great! Re-added the