Daniel Michaelis
04/06/2021, 5:02 PMdagster-pyspark = "==0.10.9"
and dagster-celery-k8s = "==0.10.9"
). However, when the dependencies are resolved, the 0.10.9 versions of the specified packages are installed but the newest versions of dependencies such as dagster-spark, dagster-k8s and dagster-celery (currently 0.11.3) which were not explicitely specified. As I understand all versions of dagster packages should always be identical. Of course I can add dagster-spark, dagster-k8s and dagster-celery to my requirements file (pyproject.toml in poetry) but I can't foresee future changes or additions in dependencies for dagster packages, so I'd have to do that every time I add a package or update a version. Is it possible that the dagster team makes sure that these mismatches don't occur by preventing dependency resolution if versions of two or more dagster packages are unidentical/incompatible, and by locking the versions of dependencies to the specified version of the corresponding dagster package (i.e. if I specify dagster-pyspark = "==0.10.9"
, dagster-spark = "==0.10.9"
is implied? Or am I missing something?max
04/06/2021, 6:37 PMx.y.z
all the other dagster packages are pinned to that same releaseDaniel Michaelis
04/07/2021, 10:01 AM