https://dagster.io/ logo
#integration-bigquery
Title
# integration-bigquery
m

Miguel Caballero Pinto

06/05/2023, 2:10 PM
Hello! I have been successful at using BigQueryResource on an asset, but when I use it in a sensor I get this error when trying to get the client:
Copy code
AttributeError: 'generator' object has no attribute 'get_client'
The code of the sensor is like this:
Copy code
@sensor(job=person_enrichment_job_input_job)
def person_enrichment_job_input_sensor(
    context: SensorEvaluationContext,
    bigquery: BigQueryResource,
):
    with bigquery.get_client() as client:
        .....
It fails when calling
get_client
function. Any idea of what may be going wrong? Is there a difference in the implementation of using resources on sensors vs assets?
In case this is useful, when I print the variable
bigquery
I get this:
Copy code
<generator object BigQueryResource.get_object_to_set_on_execution_context at 0x28ff5d6d0>
If I print the same parameter in an asset, I get a
BigQueryResource
instance instead
So it seems the parameter is resolved to different types on assets vs sensors
j

jamie

06/05/2023, 3:25 PM
hmmm yeah that may be the case that the resource is provided in different ways. looping in @ben .
❤️ 1
c

Charlie Bini

06/27/2023, 5:01 PM
@Miguel Caballero Pinto @jamie was this solved? just ran into it myself
I can get it working using
next(resources.db_bigquery)
but that seems kinda odd
j

jamie

06/27/2023, 5:27 PM
i don’t know if @ben updated anything about how resources are provided to sensors, but on the bigquery side, the bigquery resource is still a generator (this is because of some auth related stuff). i’ve been working on a way to make it not a generator between some other tasks, but i’ve had some higher priority work in the past few weeks so i haven’t been able to focus on it
b

ben

06/27/2023, 5:30 PM
Sorry for the late reply! I think I can see why this is happening, let me try to put a fix together
43 Views