Gitlab credentials are not formatted properly when "gitlab" is not part of the URL

[svenrdz]

Bug summary

I have been using GitlabCredentials blocks with a git-based deployments, and that has always worked fine. Today, I ran into an error at the git_clone pull step, with a very cryptic code 128 error from git. The main difference was the gitlab instance that I tried using today, which is a self-hosted one that looks like git.server.fr, where previously, I was using one that looked like gitlab.server.fr.

The new one does not contain the full string "gitlab", so I believe the function prefect.runner.storage:_format_token_from_credentials is not prefixing my access token with oauth2: as it should for gitlab repos.

For the time being, I am simply going to prefix my gitlab tokens with oauth2: to be safe.

It would probably make sense for _format_token_from_credentials to take into account the type of the block used, as it currently only guesses what to do based on the netloc of the Gitlab URL.

Version info

Version:             3.1.12
API version:         0.8.4
Python version:      3.11.10
Git commit:          e299e5a7
Built:               Thu, Jan 9, 2025 10:09 AM
OS/Arch:             linux/x86_64
Profile:             me
Server type:         server
Pydantic version:    2.10.5
Integrations:
  prefect-shell:     0.3.1
  prefect-docker:    0.6.2
  prefect-gitlab:    0.3.1

Additional context

No response

[zzstoatzz]

hi @svenrdz - thank you for the issue!

any chance you have an MRE for this to make it easier to implement a fix?

[svenrdz]

Hi @zzstoatzz, here is a way to reproduce the issue using a simple function that mocks gitlab's behaviour of rejecting a clone without oauth2: prefixing the token.
I used docker compose as I'm clueless on how to simulate different hostnames locally, and that's what makes the clone not work currently. I found prefect only adds authentication to the URL of a repo if it is served through https, that's why I added self-signed certificates, created during the build, such that https can be used locally.

I apologize in advance as this is probably quite far from minimal, but it reproduces the issue as I found it in a real setup.

I've uploaded the files on this repo: https://github.com/svenrdz/prefect-gitlab-issue-mre

server.py

from typing import Annotated

import uvicorn
from fastapi import Depends, FastAPI, HTTPException, Response
from fastapi.security import HTTPBasic, HTTPBasicCredentials

app = FastAPI()
security = HTTPBasic()

@app.get("/{org}/{repo}/{path:path}")
async def mock_git_repo(
    credentials: Annotated[HTTPBasicCredentials, Depends(security)],
) -> Response:
    if credentials.username != "oauth2":
        raise HTTPException(status_code=401, detail="Unauthorized")
    return Response(status_code=200)


if __name__ == "__main__":
    uvicorn.run(
        app,
        host="0.0.0.0",
        port=8000,
        ssl_keyfile="key.pem",
        ssl_certfile="cert.pem",
    )

pull.py

import asyncio

from prefect.runner.storage import GitRepository
from prefect_gitlab import GitLabCredentials


async def clone_repo(url):
    repo = GitRepository(
        url=url,
        credentials=GitLabCredentials(token="token"),
    )
    await repo.pull_code()


if __name__ == "__main__":
    asyncio.run(clone_repo("https://fake-gitlab:8000/org/repo.git"))  # works
    asyncio.run(clone_repo("https://fake-git:8000/org/repo.git"))  # error

Dockerfile

FROM python:3.12-slim-bookworm

RUN apt-get update && \
	apt-get upgrade -y && \
    apt-get install -y git

WORKDIR /mre
RUN pip install fastapi[standard] prefect[gitlab]
RUN git config --global http.sslverify false
RUN openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes -batch
ADD server.py pull.py .

docker-compose.yaml

services:
  fake-git:
    build: .
    command: python server.py

  fake-gitlab:
    build: .
    command: python server.py

  pull:
    build: .
    command: python pull.py
    depends_on:
      fake-gitlab:
        condition: service_started
      fake-git:
        condition: service_started

With these 4 files in a directory, running docker compose up should start the 2 mocked git servers inside the docker network, with their hostnames containing "gitlab" and "git", and a third service that tries to pull a repo from each mock servers. The first one correctly pulls an empty repo, and the 2nd one errors, as it does not believe it to be a gitlab server, with the following message:

Traceback (most recent call last):
  File "/mre/pull.py", line 19, in <module>
    asyncio.run(clone_repo("https://fake-git:8000/org/repo2.git"))  # error
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 686, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/mre/pull.py", line 14, in clone_repo
    await repo.pull_code()
  File "/usr/local/lib/python3.12/site-packages/prefect/runner/storage.py", line 282, in pull_code
    await self._clone_repo()
  File "/usr/local/lib/python3.12/site-packages/prefect/runner/storage.py", line 314, in _clone_repo
    raise RuntimeError(
RuntimeError: Failed to clone repository 'https://fake-git:8000/org/repo2.git' with exit code 128.