Feature request: xz support for deb datasource

[Finkregh]

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

github, 39.60.0

Please tell us more about your question or problem

I use the deb datasource, which currently only supports .gz compressed files. This repo though only has .xz files: https://deb.debian.org/debian-security/dists/bookworm-security/main/binary-amd64/ in comparison to https://deb.debian.org/debian/dists/bookworm/main/binary-amd64/

Edit: same with the -updates repo.

This is especially an issue for updates that are security relevant, e.g. smarty4:
https://packages.debian.org/bookworm/smarty4

4.3.0-1+deb12u2 is not available in the normal repos.

Logs (if relevant)

Logs

2024-12-13T11:20:32.682458478Z          "options": {
2024-12-13T11:20:32.682459682Z            "headers": {
2024-12-13T11:20:32.682460855Z              "user-agent": "RenovateBot/39.60.0 (https://github.com/renovatebot/renovate)",
2024-12-13T11:20:32.682462154Z              "accept-encoding": "gzip, deflate, br"
2024-12-13T11:20:32.682463371Z            },
2024-12-13T11:20:32.682464768Z            "url": "https://internal_mirror_of_deb_debian_org_security/dists/bookworm-security/main/binary-amd64/Packages.gz",
2024-12-13T11:20:32.682466039Z            "hostType": "deb",
2024-12-13T11:20:32.682467259Z            "username": "",
2024-12-13T11:20:32.682468380Z            "password": "",
2024-12-13T11:20:32.682469527Z            "method": "GET",
2024-12-13T11:20:32.682470637Z            "http2": false
2024-12-13T11:20:32.682471832Z          },
2024-12-13T11:20:32.682473138Z          "response": {
2024-12-13T11:20:32.682474295Z            "statusCode": 404,
2024-12-13T11:20:32.682475516Z            "statusMessage": "Not Found",
2024-12-13T11:20:32.682476885Z            "body": "{\n  \"errors\" : [ {\n    \"status\" : 404,\n    \"message\" : \"Not Found\"\n  } ]\n}",
2024-12-13T11:20:32.682481011Z            "headers": {
2024-12-13T11:20:32.682482169Z              "date": "Fri, 13 Dec 2024 11:20:32 GMT",
2024-12-13T11:20:32.682483389Z              "content-type": "application/json;charset=ISO-8859-1",
2024-12-13T11:20:32.682484801Z              "transfer-encoding": "chunked",
2024-12-13T11:20:32.682486071Z              "connection": "keep-alive",
2024-12-13T11:20:32.682487264Z              "x-jfrog-version": "Artifactory/xxx",
2024-12-13T11:20:32.682488489Z              "x-artifactory-id": "xxx",
2024-12-13T11:20:32.682489701Z              "x-artifactory-node-id": "internal-artifactory-0",
2024-12-13T11:20:32.682491380Z              "strict-transport-security": "max-age=31536000; includeSubDomains"
2024-12-13T11:20:32.682492522Z            },
2024-12-13T11:20:32.682493700Z            "httpVersion": "1.1",
2024-12-13T11:20:32.682494954Z            "retryCount": 0
2024-12-13T11:20:32.682496243Z          }
2024-12-13T11:20:32.682497524Z        }

[frederic-klein]

Willing to try to provide a PR. Ideas:

1. xz as fallback, if Packages.gz does not exist
2. check InRelease for Version and use xz compression, if version >= 12
3. expose compression as configuration
4. check InRelease for matching path and resolve compression
5. ...

note Packages.xz is a valid option, compare: https://wiki.debian.org/DebianRepository/Format#Compression_of_indices

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

Please follow these steps:

1. Read our guide on creating a minimal reproduction.
2. Go to our minimal reproduction template repository.
3. Select the Use this template button to create a new repository based on the template.
4. Work on the minimal reproduction in your own repository.
5. Fill out the information in your repository's README.md.
6. Add the link to your reproduction to the first post of your Discussion. If you are not the original author, you can post a new comment with the link.

If you need help with running renovate on your minimal reproduction repository, please refer to our Running Renovate guide.
Good luck,

The Renovate team