https://dagster.io/ logo
#ask-community
Title
# ask-community
t

Tristan Schrader

07/12/2023, 1:09 AM
Just starting to use
dagster
. I’ve tried to use a Resource in my Asset, but I’m stuck with a
dagster._core.errors.DagsterInvalidDefinitionError: Invalid type: dagster_type must be an instance of DagsterType or a Python type: got GoogleDriveClient.
error. Here’s my minimal working example:
Copy code
from dagster import asset, ConfigurableResource, Definitions
from googleapiclient.discovery import build

class GoogleDriveClient(ConfigurableResource):
    def get_folder(self, name: str) -> str:
        query = f"mimeType='application/vnd.google-apps.folder' and trashed=false and name='{name}'"
        response = build("drive", "v3").files().list(q=query).execute()
        return response.get("files", [])[0].get("id", "")

@asset
def get_folder(gdrive: GoogleDriveClient) -> str:
    return gdrive.get_folder("My Folder")

defs = Definitions(assets=[get_folder], resources={"gdrive": GoogleDriveClient()})
Here’s what I would like to work:
Copy code
from __future__ import annotations
from typing import TYPE_CHECKING
from dagster import asset, ConfigurableResource, Definitions
from googleapiclient.discovery import build

if TYPE_CHECKING:
    from googleapiclient._apis.drive.v3.schemas import File

class GoogleDriveClient(ConfigurableResource):
    def get_folder(self, name: str) -> File:
        query = f"mimeType='application/vnd.google-apps.folder' and trashed=false and name='{name}'"
        response = build("drive", "v3").files().list(q=query).execute()
        return response.get("files", [])[0]

@asset
def get_folder(gdrive: GoogleDriveClient) -> File:
    return gdrive.get_folder("My Folder")

defs = Definitions(assets=[get_folder], resources={"gdrive": GoogleDriveClient()})
This gives the error I mentioned above. I thought it might be because the
File
type is not available at
dagster
runtime because of the
TYPE_CHECKING
conditional import. The problem is that when I remove the conditional import (I think it’s standard for
mypy
these days), I get
ModuleNotFoundError: No module named 'googleapiclient._apis'
. That’s because
mypy
resolves the correct location in
site-packages
where the stubs were downloaded (
googleapiclient-stubs
), but I guess
dagster
doesn’t. Not really sure what to do… My understanding is the stubs I’ve downloaded are following all the standards. They work with
mypy
at least. Is there some way to tell
dagster
to find these correctly? Thanks in advance.
I found the following bug reports: https://github.com/dagster-io/dagster/issues/4121 https://github.com/dagster-io/dagster/issues/2467 Looks like these are pretty much what I’m facing. The problem is that they have to be deferred because they are stub files.
o

Odette Harary

07/13/2023, 2:47 PM
Thanks Tristian - if any issues aren't covered by these feel free to open one up and we'll take a look
2 Views