won
07/17/2022, 8:36 AM'SolidExecutionContext' object has no attribute 'bind'
how can i use log_event
without passing context
in finction?
@op(out={"raw_list": Out(List[Dict])})
def recursive_collect_data(
# context,
token: str, page_n: int = 1):
limit = 100
response = requests.request(
method='GET',
url='<https://api.avito.ru/core/v1/items>',
headers={"Authorization": f'Bearer {token}'},
params={'per_page': limit, 'page': page_n, 'status': ['active']},
)
raw_list = response.json()
# context.log_event(
# AssetMaterialization(
# asset_key="avito items report",
# metadata={
# "page №": <http://MetadataValue.int|MetadataValue.int>(page_n),
# "length": <http://MetadataValue.int|MetadataValue.int>(len(raw_list['resources'])),
# "report meta": MetadataValue.json(raw_list['meta']),
# },
# )
# )
if len(raw_list.get('resources')) < limit:
return raw_list['resources']
else:
return raw_list['resources'] + recursive_collect_data(
# context=context,
token=token, page_n=raw_list['meta']['page'] + 1,
)
jamie
07/18/2022, 2:44 PMrequests.request
calls and yield the response json as dynamic outs that each get processed by an op that logs the asset materialization event
2. keep everything in one op, but put the requests.request
call and the event logging in a for loop that runs until raw_list.get('resources') < limit