Hi For multiple locations in Dagster Cloud When deploying wi dagster #dagster-plus

Hi, For multiple locations in Dagster Cloud, When ...

Aki Iwa

07/22/2023, 11:56 PM

Hi, For multiple locations in Dagster Cloud, When deploying with dagster-io/dagster-cloud-action/actions/hybrid_branch_deploy, sometimes it succeeds even though the conditions have not been changed, and sometimes it fails with the error

No module named xxxxxxx

.What could be the cause of this?

Aki Iwa

07/23/2023, 12:34 AM

Error Details

Aki Iwa

07/23/2023, 2:10 AM

Settings • dagster_cloud.yml

Copy code

locations:
  - location_name: A
    code_source:
      package_name: A_etl
    build:
      directory: A
      registry: <http://xxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/repository|xxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/repository>
    container_context:
      .....
  - location_name: B
    code_source:
      package_name: B_etl
    build:
      directory: B
      registry: <http://xxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/repository|xxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/repository>
    container_context:
      .....

Aki Iwa

07/23/2023, 3:12 AM

This problem could be the same one. https://dagster.slack.com/archives/C01U954MEER/p1685041217052529?thread_ts=1684879037.710179&cid=C01U954MEER

Shalabh Chaturvedi

07/23/2023, 3:48 AM

Hi Akira - looks like it cannot find a package when running. Do you have the

corp_etl

package installed in your docker image? Either you will need to copy and install that package or if it is not a package (just a directory) you should copy the directory under the build directory that you are deploying.

Aki Iwa

07/23/2023, 4:07 AM

Yes. I have the package installed. So, even though I haven’t made any changes, the deploy may or may not succeed.

Shalabh Chaturvedi

07/23/2023, 5:29 AM

That is unexpected. When you do get this error are you able to pull the latest image from your registry and run it locally to verify that the package is there? Do you use pip or another way of installing the package?

Aki Iwa

08/26/2023, 12:11 PM

Sorry for the late reply. I have confirmed that the package exists and that it can be installed. For the time being, I avoided the problem by loading it in a different way, it seems that there are other people who are experiencing the same problem. https://dagster.slack.com/archives/C01U954MEER/p1691405585980059?thread_ts=1684879037.710179&cid=C01U954MEER

Shalabh Chaturvedi

08/26/2023, 10:40 PM

This might be related to an issue where images built for multiple code locations use the same tag and can overwrite each other. Because the builds run concurrently, this might work if the build with the right build directory finishes last and overwrites the docker tag. This issue has been fixed in the new version of the github workflow: https://github.com/dagster-io/dagster-cloud-hybrid-quickstart/ which doesn't use the

hybrid_branch_deploy

action. You could upgrade to the newer version to improve the deploy pipeline. We will also work on fixing the issue with

hybrid_branch_deploy

itself.

Open in Slack

Previous Next