Hi folks ! I try to implement single run backfill...
# ask-community
m
Hi folks ! I try to implement single run backfill with a custom IO manager. I read the related documentation and found that I need to replace
asset_partition_key_for_output
method by, for example,
asset_partition_key_range_for_input
. The given example don't talk to me in order to perform the transition 😕 I know I need to adpat my where condition like this (see full asset below):
WHERE CREATED_DATE >= '{partition_date_str_*start*}' AND CREATED_DATE <= '{partition_date_str_*end*}'
My asset :
Copy code
@asset(required_resource_keys={"connection"},
       metadata={"partition_expr": "CREATED_DATE"},
       io_manager_key=my_io_manager_key,
       partitions_def=DailyPartitionsDefinition(start_date='2023-06-01'))

def TABLE1(context):

    my_connection = context.resources.connection
    partition_date_str = context.asset_partition_key_for_output()

    TABLE1 = my_connection.get_table(
    f"SELECT * FROM TABLE1 WHERE CREATED_DATE = '{partition_date_str}'")

    return TABLE1
Do you have other example or can you provide any help to do it ? Thanks a lot ! 🙂
o
hi @Malo PARIS! The basic format you'd want here would be
Copy code
partition_date_start, partition_date_end = context.asset_partitions_time_window_for_output()
these two variables are going to be datetime objects, so you can format them into strings in whichever way makes sense for your use case, then plug that into:
Copy code
WHERE CREATED_DATE >= '{partition_date_str_start}' AND CREATED_DATE <= '{partition_date_str_end}'
❤️ 1
m
it's working, thanks :)