Skip to content

Commit

Permalink
docs(Python): reapply suggestions and update outdated info
Browse files Browse the repository at this point in the history
* Reapply dropped commit in #16643
* Change bindings example as `protobuf` no longer has any
* Remove `name "foo"` in example
* Replace `Language::Python.setup_install_args` example with `pip`
* Add some hints for CMake and autotools scripts given common issue of
  incorrect Python selection due to allowing multiple Python deps
* Add note that `.pth` files should be avoided due to contamination

Co-authored-by: Eric Knibbe <[email protected]>
Signed-off-by: Michael Cho <[email protected]>
  • Loading branch information
cho-m and EricFromCanada committed Apr 14, 2024
1 parent 17d5ab3 commit 2199258
Showing 1 changed file with 16 additions and 12 deletions.
28 changes: 16 additions & 12 deletions docs/Python-for-Formula-Authors.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This document explains how to successfully use Python in a Homebrew formula.

Homebrew draws a distinction between Python **applications** and Python **libraries**. The difference is that users generally do not care that applications are written in Python; it is unusual that a user would expect to be able to `import foo` after installing an application. Examples of applications are [`ansible`](https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/a/ansible.rb) and [`jrnl`](https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/j/jrnl.rb).

Python libraries exist to be imported by other Python modules; they are often dependencies of Python applications. They are usually no more than incidentally useful in a terminal. An example of a library is the bindings that are installed by [`protobuf`](https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/p/protobuf.rb).
Python libraries exist to be imported by other Python modules; they are often dependencies of Python applications. They are usually no more than incidentally useful in a terminal. An example of a library is the bindings that are installed by [`libxml2`](https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/lib/libxml2.rb).

Bindings are a special case of libraries that allow Python code to interact with a library or application implemented in another language.

Expand Down Expand Up @@ -53,7 +53,6 @@ For most applications, all you will need to write is:
class Foo < Formula
include Language::Python::Virtualenv

name "foo"
# ...
url "..."
sha256 "abc123abc123abc123abc123abc123abc123abc123abc123abc123abc123abc1"
Expand All @@ -70,7 +69,6 @@ This is exactly the same as writing:
class Foo < Formula
include Language::Python::Virtualenv

name "foo"
# ...
url "https://example.com/foo-1.0.tar.gz"
sha256 "abc123abc123abc123abc123abc123abc123abc123abc123abc123abc123abc1"
Expand Down Expand Up @@ -154,18 +152,22 @@ Bindings should follow the same advice for Python module dependencies as librari
If the bindings are installed by invoking a `setup.py`, do something like:

```ruby
cd "source/python" do
system Formula["[email protected]"].opt_bin/"python3", *Language::Python.setup_install_args(prefix)
end
system "python3.y", "-m", "pip", "install", *std_pip_args(build_isolation: true), "./source/python"
```

If the configure script takes a `--with-python` flag, it usually will not need extra help finding Python.
If `cmake` finds a different Python, sometimes you can help it find the correct Python by setting one of the following variables with the `-D` option:

* `Python3_EXECUTABLE` for the [`FindPython3`](https://cmake.org/cmake/help/latest/module/FindPython3.html) module
* `Python_EXECUTABLE` for the [`FindPython`](https://cmake.org/cmake/help/latest/module/FindPython.html) module
* `PYTHON_EXECUTABLE` for the [`FindPythonInterp`](https://cmake.org/cmake/help/latest/module/FindPythonInterp.html) module

If the configure script takes a `--with-python` flag, it usually will not need extra help finding Python. However, if there are multiple Python formulae in the dependency tree, it may need help finding the correct one.

If the `configure` and `make` scripts do not want to install into the Cellar, sometimes you can:

1. call `./configure --without-python` (or a similar named option)
2. `cd` into the directory containing the Python bindings
3. call `setup.py` with `system` and `Language::Python.setup_install_args` (as described above)
3. call `setup.py` with `system` and `std_pip_args` (as described above)

Sometimes we have to edit a `Makefile` on-the-fly to use our prefix for the Python bindings using Homebrew's [`inreplace`](Formula-Cookbook.md#inreplace) helper method.

Expand All @@ -175,7 +177,7 @@ Remember: there are very limited cases for libraries (e.g. significant amounts o

**We do not use the `python-` prefix for these kinds of formulae!**

### Examples of allowed libaries in homebrew-core
### Examples of allowed libraries in homebrew-core

* `numpy`, `scipy`: long build time, complex build process

Expand All @@ -197,15 +199,17 @@ Most formulae presently just install to `prefix`.

Library dependencies must be installed so that they are importable. To minimise the potential for linking conflicts, dependencies should be installed to `libexec/<vendor>` and added to `sys.path` by writing a second `.pth` file (named like "homebrew-foo-dependencies.pth") to the `prefix` site-packages.

Non keg-only formulae with general Python library dependencies (e.g. `setuptools`, `six`) should not use this approach as it will contaminate the system `site-packages` with all libraries installed inside `libexec/<vendor>`.

## Further down the rabbit hole

Additional commentary that explains why Homebrew does some of the things it does.

### Setuptools vs. Distutils vs. pip

Distutils is a module in the Python standard library that provides developers a basic package management API. Setuptools is a module distributed outside the standard library that extends Distutils. It is a convention that Python packages provide a `setup.py` that calls the `setup()` function from either Distutils or Setuptools.
Distutils was a module in the Python standard library that provided developers a basic package management API until its removal in Python 3.12. Setuptools is a module distributed outside the standard library that extends and replaces Distutils. It is a convention that Python packages provide a `setup.py` that calls the `setup()` function from either Distutils or Setuptools.

Setuptools provides the `easy_install` command, which is an end-user package management tool that fetches and installs packages from PyPI, the Python Package Index. `pip` is another, newer end-user package management tool, which is also provided outside the standard library. While pip supplants `easy_install`, it does not replace the other functionality of the Setuptools module.
Setuptools used to provide the `easy_install` command, which was an end-user package management tool that fetched and installed packages from PyPI, the Python Package Index. `easy_install` was removed in Setuptools 52. `pip` is another, newer end-user package management tool, which is also provided outside the standard library. While `pip` supplants `easy_install`, it does not replace the other functionality of the Setuptools module.

Distutils and pip use a "flat" installation hierarchy that installs modules as individual files under `site-packages` while `easy_install` installs zipped eggs to `site-packages` instead.

Expand Down Expand Up @@ -245,4 +249,4 @@ It is probably safe to use `--prefix` with `--root=/`, which should work with ei

### `pip` vs. `setup.py`

[PEP 453](https://legacy.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors) makes a recommendation to downstream distributors (us) that sdist tarballs should be installed with `pip` instead of by invoking `setup.py` directly. For historical reasons we did not follow PEP 453, so some formulae still use `setup.py` installs. Nowadays, most of the formulae use `pip` as we have migrated them to this better way of installation.
[PEP 453](https://legacy.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors) makes a recommendation to downstream distributors (us) that sdist tarballs should be installed with `pip` instead of by invoking `setup.py` directly. For historical reasons we did not follow PEP 453, so some formulae still use `setup.py` installs. Nowadays, most of the formulae use `pip` as we have migrated them to this preferred method of installation.

0 comments on commit 2199258

Please sign in to comment.