The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hi, is it possible to create Multi-dimensionally partitioned Jobs, or it’s something that’s only officially supported by assets?

Hey Oren. It is supported to create multidimensional partitioned jobs

Thanks Claire! Are there examples on how to use it? For static/dynamic/time-based partitions there are decorates that make is more clear

Here's a simple example if this helps:
```@op
def my_op():
    return 1


@job(
    partitions_def=MultiPartitionsDefinition(
        {
            "abc": StaticPartitionsDefinition(["a", "b", "c"]),
            "time": DailyPartitionsDefinition("2023-01-01"),
        }
    )
)
def my_job():
    my_op()```

Ummm. How do you use the partition keys inside the ops? In the other <https://docs.dagster.io/concepts/partitions-schedules-sensors/partitions#partitioned-non-asset-jobs|examples> for partitioned jobs they use PartitionConfig in order to map the partition keys to op params.

Looks like I can do something like this:
```multi_part_def = MultiPartitionsDefinition(
    {
        "abc": StaticPartitionsDefinition(["a", "b", "c"]),
        "time": DailyPartitionsDefinition("2023-01-01"),
    }
)

multi_part_config = PartitionedConfig(
    partitions_def=multi_part_def,
    run_config_for_partition_fn=lambda partition: {
        "ops": {
            "process_data_for_date": {
                "config": {
                    "time": partition.value.keys_by_dimension["time"],
                    "abc": partition.value.keys_by_dimension["abc"],
                }
            }
        }
    },
)

@job(config=multi_part_config)
def do_stuff_partitioned():
    process_data_for_date()```


You should be able to call context.partition_key to fetch the currently executing partition key:
```@op
def my_op(context):
   context.partition_key.keys_by_dimension
   ...```

Thanks Claire, I thought that might be the other option (reading it from the context as opposed to passing it via the configuration). It’s easier, but it means that the `ops` need to be aware that the job is defined as a partitioned job, which I prefer to avoid at the moment. I found how to create a PartitionedConfig though, and that should be good enough for what I currently need.