With dagster-fivetran integration, is there a way ...
# ask-community
m
With dagster-fivetran integration, is there a way to filter out connectors which are paused when using load_assets_from_fivetran_instance? There are quite a few connectors and a subset of them paused.
s
Also interested in this
m
I was able to solve it like so.. Basically got the paused connectors via an api call and used it in connector_filter of load_assets_from_fivetran_instance
Copy code
def get_paused_connectors(api_key: str, api_secret: str) -> List[dict]:
    url = '<https://api.fivetran.com/v1/connectors>'
    url_connectors = '<https://api.fivetran.com/v1/groups/mess_sieve/connectors>'
    auth = (api_key, api_secret)

    response = requests.get(url_connectors, auth=auth)
    response.raise_for_status()

    connectors = response.json()['data']['items']

    paused_connectors = [c["id"] for c in connectors if c['paused']]

    return paused_connectors

# usage
api_key = fivetran_api_key
api_secret = fivetran_api_secret
paused_connectors = get_paused_connectors(api_key, api_secret)

fivetran_instance = FivetranResource(
    api_key=fivetran_api_key,
    api_secret=fivetran_api_secret,
)
fivetran_assets = load_assets_from_fivetran_instance(fivetran_instance, key_prefix=["fivetran"],  connector_filter=lambda connector: connector.connector_id not in paused_connectors)
``````
👌 1