Homebrew formula has linkage issue on Apple silicon setup for python application with selenium as dependency

[arpitchaudhary]

brew doctor output

arpitchaudhary@Arpits-MacBook-Pro Formula % brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: No Cask quarantine support available: unknown reason.


Warning: Some installed kegs have no formulae!
This means they were either deleted or installed manually.
You should find replacements for the following formulae:
  selenium-issue

Warning: Some installed formulae are deprecated or disabled.
You should find replacements for the following formulae:
  [email protected]
  [email protected]


Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause formulae that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  kubernetes-cli

### Verification

- [X] My "`brew doctor` output" above says `Your system is ready to brew.` and am still able to reproduce my issue.
- [X] I ran `brew update` twice and am still able to reproduce my issue.
- [X] This issue's title and/or description do not reference a single formula e.g. `brew install wget`. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

### `brew config` output

```shell
- redacted as I have internal company specific setup but is reproduced on other machines too.

What were you trying to do (and why)?

I was trying to install a python package with dependency on selenium. I saw linkage issues that also shows up when trying to update the formula leading to linkage not being created and needing to reinstall or link manually through link command.

What happened (include all command output)?

Installing a formula which reproduces the issue. Formula code added to reproductions steps:

$ brew install --build-from-source selenium-issue.rb
==> Auto-updating Homebrew...
Adjust how often this is run with HOMEBREW_AUTO_UPDATE_SECS or disable with
HOMEBREW_NO_AUTO_UPDATE. Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
Error: Failed to load cask: selenium-issue.rb
Cask 'selenium-issue' is unreadable: wrong constant name #<Class:0x000000010142a458>
Warning: Treating selenium-issue.rb as a formula.
==> Fetching selenium-issue
==> Downloading https://files.pythonhosted.org/packages/5f/3f/f55eef404adae2d5429728722d6a81ad6ac50a80e9b47be046cfbe97bc44/eg-1.2.2.tar.gz
Already downloaded: /Users/arpitchaudhary/Library/Caches/Homebrew/downloads/6c68494c43b81bc063a2c83aec718fedb75d7407eeff65b1c112357c23da093a--eg-1.2.2.tar.gz

==> python3.11 -m venv --system-site-packages --without-pip /opt/homebrew/Cellar/selenium-issue/0.0.1/libexec
==> /opt/homebrew/Cellar/selenium-issue/0.0.1/libexec/bin/python3 -m pip install -v selenium==4.17.2
🍺  /opt/homebrew/Cellar/selenium-issue/0.0.1: 1,210 files, 34.4MB, built in 10 seconds
==> Running `brew cleanup selenium-issue`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).

Linking failed:

$ brew linkage selenium-issue 
System libraries:
  /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  /System/Library/Frameworks/Security.framework/Versions/A/Security
  /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
  /usr/lib/libSystem.B.dylib
  /usr/lib/libiconv.2.dylib
Missing libraries:
  /usr/local/opt/xz/lib/liblzma.5.dylib

What did you expect to happen?

Brew install to succeed with all linkage correctly. Brew tries to check link with liblzma.5.dylib at location where it is installed on intel machines but I have an apple silicon Mac.

Step-by-step reproduction instructions (by running brew commands)

Steps to reproduce:

1. Install the formula attached: brew install --build-from-source selenium-issue.rb .
2. Check if there are any broken linkages: brew brew linkage selenium-issue .
   a. We see a missing xz library: /usr/local/opt/xz/lib/liblzma.5.dylib when it is available in different location. This location mentioned is to be used in x86 platforms only while we are on arm64 (Apple Silicon) platform.
   b. Otool command shows that brew is considering wrong library:

$ otool -L /opt/homebrew/Cellar/selenium-issue/0.0.1/libexec/lib/python3.11/site-packages/selenium/webdriver/common/macos/selenium-manager
/opt/homebrew/Cellar/selenium-issue/0.0.1/libexec/lib/python3.11/site-packages/selenium/webdriver/common/macos/selenium-manager (architecture x86_64):
        /usr/local/opt/xz/lib/liblzma.5.dylib (compatibility version 10.0.0, current version 10.5.0)
        /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 1241.60.3)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1953.255.0)
        /System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60420.60.24)
        /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
/opt/homebrew/Cellar/selenium-issue/0.0.1/libexec/lib/python3.11/site-packages/selenium/webdriver/common/macos/selenium-manager (architecture arm64):
        /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 1241.60.3)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1953.255.0)
        /System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60420.60.24)
        /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)

As seen in otool output /usr/local/opt/xz/lib/liblzma.5.dylib should have needed only for x86 platform.

Demo Brew formula to reproduce:

class SeleniumIssue < Formula
  include Language::Python::Virtualenv

  desc "Test Formula to reproduce issue caused by installing any pythin package with selenium as dependency."

  # Dummy values as we will be installing a package through pip so dependencies are easily installed as well.
  homepage "https://github.com/srsudar/eg"
  url "https://files.pythonhosted.org/packages/5f/3f/f55eef404adae2d5429728722d6a81ad6ac50a80e9b47be046cfbe97bc44/eg-1.2.2.tar.gz"
  sha256 "8d3745eceb2a4c91507b1923193747b7ae88888e6257eb8aaccf7deae2a300a7"
  version "0.0.1"

  depends_on "[email protected]"

  def install
    virtualenv_create(libexec, "python3.11")
    @venv_root = Pathname.new(libexec)
    system @venv_root/"bin/python3", "-m", "pip", "install",
           "-v", "--ignore-installed", "selenium==4.17.2"
  end

end

[MikeMcQuaid]

/opt/homebrew/Cellar/selenium-issue/0.0.1/libexec/lib/python3.11/site-packages/selenium/webdriver/common/macos/selenium-manager

This is provided by Pip and not Homebrew. You'll need to figure out with them why it's got this invalid linkage.

• redacted as I have internal company specific setup but is reproduced on other machines too.

@arpitchaudhary For future reference: you're going to need to supply something here. Feel free to redact parts but not providing all of it, particularly when you also have brew doctor issues, means we can't really help here.

[carlocab]

This is provided by Pip and not Homebrew.

Yes. Your command for installing selenium-issue uses pre-built wheels hosted on PyPI:

❯ python3 -m venv venv
❯ source venv/bin/activate
❯ python3 -m pip install --ignore-installed selenium==4.17.2
[snip]
❯ otool -L venv/lib/python3.12/site-packages/selenium/webdriver/common/macos/selenium-manager
venv/lib/python3.12/site-packages/selenium/webdriver/common/macos/selenium-manager (architecture x86_64):
        /usr/local/opt/xz/lib/liblzma.5.dylib (compatibility version 10.0.0, current version 10.5.0)
        /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 1241.60.3)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1953.255.0)
        /System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60420.60.24)
        /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
venv/lib/python3.12/site-packages/selenium/webdriver/common/macos/selenium-manager (architecture arm64):
        /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 1241.60.3)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1953.255.0)
        /System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 60420.60.24)
        /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)

You probably want to read https://docs.brew.sh/Python-for-Formula-Authors.

[gibfahn]

This is provided by Pip and not Homebrew. You'll need to figure out with them why it's got this invalid linkage.

This makes sense for homebrew core, but for external taps, it would be nice to be able to avoid the formula being broken when using pypi wheels. If the binary runs on the user machine, does it matter if an optional linkage that isn't used is present?

Especially as in this case the linkage that homebrew complains about isn't present in the arm64 slice that is actually being used.

It's also surprising that brew install works, but later brew upgrade will fail due to this linkage error. If this blocks I'd expect it to block everywhere.

[carlocab]

If it's in a slice that's never intended to be used you can call deuniversalize_machos to get rid of the non-native slices.

[MikeMcQuaid]

If the binary runs on the user machine, does it matter if an optional linkage that isn't used is present?

It should still be a warning but this is why it's not an error, yeh.

but later brew upgrade will fail due to this linkage error. If this blocks I'd expect it to block everywhere.

Does it? This would be a different issue, if you can reproduce please open that.