Yepher
08/26/2023, 3:00 PM# Find array of customer_ids to analyze
@asset
def find_customers_to_analyze(context):
with database.get_resource().get_client() as client:
detector = CustomerWrapper(context)
return detector.find_customrs_to_analyze() # returns an array of zero ore more customer_ids
# For each customer_id returned by find_customers_to_analyze
@asset
def analyze_customer(context, customer_id):
with database.get_resource().get_client() as client:
detector = CustomerWrapper(context)
return detector.check_for_anomalies(customer_id)
It would be nice if analyze_customer
would run as many as possible in parallel as allowed by some concurrent limit.
Is there some documentation or tutorial somewhere that describes how to achieve this. I assume this is a very common usecase but my Google-foo is not strong enough to find what I am looking for. I found similar unanswered question os SOmartin o leary
08/26/2023, 3:39 PMYepher
08/26/2023, 4:38 PMops
Can I do the same thing with assets or must I use ops?Yepher
08/26/2023, 4:48 PMmartin o leary
08/26/2023, 5:20 PM