Hi again :wave:, Meeting some issues on the branc...
# dagster-plus
Hi again 👋, Meeting some issues on the branch-deployment CD github action. Basically, i have apython dependency looking for a github repository on my setup.py,
Copy code
        'internals @ git+<https://my-private-repo>@github.com/repo.git']
Which works in local, by executing the
pip install -e '.[dev]'
command. Therefore, it seems like the deploy fails on the CI/CD github action with the error shown on the screenshot. (It looks like github actions try to fetch for a local directory) Does anyone have an idea about how to solve this issue ?
Hi Charles, thanks for the clear bug report regarding treating the remote path as local. We have a fix for that in our release pipeline and and will be rolling it out today. However the other issue here is that this is a private repository and you will need some credentials to check that out in a github workflow. Do you have a username and token in your git+https url to this repo above or do you have some other credentials set up for in the workflow for your private repo?
Here is a solution that can help with installing the internal repo, even without the local filepath bugfix: 1. In your github workflow, add the following to the top of the steps to checkout and build your private repo.
Copy code
- name: Checkout internal repo
        uses: actions/checkout@v3
          token: ${{ secrets.GH_PAT }}
          repository: private-org/internal-repo
          path: deps/internal-repo
      - name: Build internal repo into a wheel
        run: >
          cd deps/internal-repo &&
          python setup.py bdist_wheel &&
          cp dist/*whl $GITHUB_WORKSPACE/deps
      - name: Configure dependency resolution to use locally built deps/
        run: >
          echo "[global]" > $GITHUB_WORKSPACE/deps/pip.conf &&
          echo "find-links = " >> $GITHUB_WORKSPACE/deps/pip.conf &&
          echo "    file://$GITHUB_WORKSPACE/deps/" >> $GITHUB_WORKSPACE/deps/pip.conf &&
          echo "PIP_CONFIG_FILE=$GITHUB_WORKSPACE/deps/pip.conf" > $GITHUB_ENV
          && cat $GITHUB_WORKSPACE/deps/pip.conf
          && ls $GITHUB_WORKSPACE/deps/
2. Create a github personal access token and set it as the
secret for your actions. 3. You may need to fix the "Build internal repo" to cd into the right directory within your internal repo 4. Replace
"internals @ git+https..."
in your setup.py with just
. The PIP_CONFIG_FILE points the dependency resolution to use the locally build wheel for the "internals" package. Let us know if this helps.
Hi Shalbabh, Thanks very much for your answers, I'm currently using a Personnal Access Token (PAT) from Github to authenticate, with no other processes. I will try the solution above and keep you in touch
Hey ! Thanks very much for your help 🙏, I had to work around some dependency hell issues, But you fix works like a charm ! No more error regarding my private repository🙂 ! I could not pass the CI, because of the DBT submodule integration, (i've never setup submodules , especially on github actions before), But i will try to dig by myself ! Have a great day