ci: Refactor Dockerfile & entrypoint

[upbqdn]

Motivation

CI contains a bunch of nits:

• Tests for the scanner don't run.
• The suite for all unit tests runs twice unnecessarily.
• We no longer have experimental features in Zebra, but we still have a failing workflow that tries to build a Docker image with experimental features.
• Zebra runs under root inside Docker, which is a security risk.
• The runtime Docker stage for production images contains unneeded packages, making the images larger than necessary and increasing the surface for security vulnerabilities.
• The deps Docker stage for tests contains unneeded packages, which slows CI down.
• Some CI tests pass even though they shouldn't.
• The entrypoint script contains unused variables and is rather complex.
• The EXPOSE instruction contains two ports, but Zebra uses only one of them at a time, and it doesn't contain some other ports Zebra uses.
• Rust features are passed from CI to Dockerfile and entrypoint in many variables. In the Dockerfile, we currently have:
  • FEATURES (which comes from RUST_PROD_FEATURES in CI)
  • TEST_FEATURES (which comes from RUST_TEST_FEATURES in CI)
  • EXPERIMENTAL_FEATURES (which should probably come from RUST_EXPERIMENTAL_FEATURES in CI, containing "shielded-scan", but it doesn't, so the Dockerfile uses the default value "journald prometheus filter-reload")
  • ENTRYPOINT_FEATURES = "$FEATURES $TEST_FEATURES"
  • ENTRYPOINT_FEATURES_EXPERIMENTAL = $ENTRYPOINT_FEATURES $EXPERIMENTAL_FEATURES
• CI uses some vars that don't exist in the entrypoint.
• The RPC server is enabled by default in our production Docker images, even though Zebra doesn't enable it by default.

Close #9210.

Solution

• Fix the tests for the scanner.
• Don't run all unit tests twice.
• Use only one variable to pass Rust features from CI to Dockerfile and entrypoint.
• Remove the workflow for building Docker images with experimental features.
• Create a non-privileged system user in the runtime Docker stage and switch to it.
• Don't use gosu.
• Remove all packages from the runtime stage.
• Remove unneeded packages from the deps stage
• Fix some malfunctioning CI tests.
• Don't use the EXPOSE instruction in Docker.
• Bump the Rust version in Dockerfile.
• Change the location of the entrypoint in Docker images from /etc/zebrad to /usr/local/bin.
• Refactor the structure of the entrypoint; remove redundant env vars, and add docs.
• Explicitly specify the location of the conf and cache dirs in Docker according to https://specifications.freedesktop.org/basedir-spec/latest/.
• Prepare the Dockerfile and entrypoint files for deploying a Testnet mining instance.
• Rename the ZEBRA_CACHED_STATE_DIR env var to ZEBRA_CACHE_DIR since that dir no longer contains only the state but also the network cache and the cookie file.
• Don't enable the RPC server by default. This is a significant breaking change for users.
• Refactor how we configure Zebra inside Docker. This is a breaking change.

Tests

• Manually test that zebrad runs under the new zebra user:

  Running

  docker build -f docker/Dockerfile --target runtime -t zebra:local .
  docker run -d --rm --name zebra_local zebra:local
  docker exec -it -u root zebra_local bash
  apt-get update && apt-get install -y procps
  ps aux

  displays

  USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
  zebra          1 86.3  2.7 6605720 2691512 ?     Ssl  09:49  31:03 zebrad -c /etc/zebrad/zebrad.toml
  root         150  0.0  0.0   4188  3368 pts/0    Ss   10:23   0:00 bash
  root         438  0.0  0.0   8088  4044 pts/0    R+   10:25   0:00 ps aux
  

PR Checklist

• The PR name is suitable for the change log.
• The solution is tested.
• The PR has a priority label.