Aki Iwa
05/24/2023, 5:34 AMdaniel
05/24/2023, 3:05 PMAki Iwa
05/24/2023, 9:28 PMdaniel
05/24/2023, 9:29 PMAki Iwa
05/24/2023, 9:30 PM@job(
resource_defs={
"io_manager": s3_pickle_io_manager.configured(
{"s3_bucket": "analytics-rd-test", "s3_prefix": "my-cool-prefix"}
),
"s3": s3_resource,
},
executor_def=in_process_executor,
tags={MAX_RUNTIME_SECONDS_TAG: 10},
)
def elt_job():
"""
ジョブパイプライン
"""
.......
daniel
05/24/2023, 9:31 PMAki Iwa
05/24/2023, 9:33 PMdaniel
05/24/2023, 10:15 PMAki Iwa
05/24/2023, 10:21 PMc4de26a5
daniel
05/24/2023, 10:26 PMAki Iwa
05/24/2023, 10:36 PMprod
daniel
05/25/2023, 12:27 AMdaniel
05/25/2023, 12:27 AMAki Iwa
05/25/2023, 12:44 AMdaniel
05/25/2023, 1:25 AMdaniel
05/25/2023, 1:26 AMAki Iwa
06/06/2023, 9:36 AMrun_monitoring:
enabled: true
start_timeout_seconds: 180
cancel_timeout_seconds: 180
daniel
06/06/2023, 2:55 PMpoll_interval_seconds
key as well (the default is 120 so checking once every couple of minutes)Aki Iwa
06/06/2023, 8:28 PM