Kyle Downey
11/16/2021, 8:18 PMload_uniswap_trades_local_job = <http://load_uniswap_trades.to|load_uniswap_trades.to>_job(
resource_defs={
'partition_bounds': partition_bounds,
'moonstream_client': live_moonstream_client.configured({'api_key': os.getenv('MOONSTREAM_API_KEY')})
},
config=hourly_download_config,
# config={
# 'resources': {
# 'partition_bounds': {
# 'config': {
# 'start': '2021-11-13 00:00:00',
# 'end': '2021-11-13 01:00:00',
# }
# },
# }
# }
)@hourly_partitioned_config(start_date=datetime(2021, 11, 16))
def hourly_download_config(start: datetime, end: datetime):
return {
'resources': {
'partition_bounds': {
'config': {
'start': start.strftime(PARTITION_BOUND_DATE_FMT),
'end': end.strftime(PARTITION_BOUND_DATE_FMT),
}
},
}
}
sandy
11/16/2021, 9:20 PMdagster job execute
. Because you're providing partitioned config to your job (via hourly_download_config
), Dagster needs to know what partition to use to generate config when launching the job
dagster job execute
doesn't allow you to provide a partition, so no partition is provided, so Dagster can't use the config from hourly_download_config
Does that make sense? In the future, we should probably add a --partition
arg to dagster job execute
Kyle Downey
11/17/2021, 10:44 PM