Python third-party tools configuration: pyproject.toml vs setup.cfg
A couple of years ago two PEPs have been proposed to fix some package building interface problems. Basically, in order to build a Python package we need to checkout source of the project, then install a build system and finally run it. PEP 517 defines the last step, including how to get the build system dynamically specify more dependencies that the build system requires to perform its job. The PEP 518 defines the second step, including a new configuration file pyproject.toml to be used by the building system.
As setuptools have been dominating in Python packaging field for a long time, many third-party tools adopted its configuration file setup.cfg too. Libraries for testing, static code analysis tools like linters or type checkers along with their custom configuration files often allow to use setup.cfg. After all, it’s INI-style configuration format, easy for humans to read and edit it, with no standard behind it though.
Moving tool settings to the setup.cfg as opposed to supporing a bunch of custom config files seems to be common practice. But now that we have the new PEPs, should we abandon setup.cfg in favour of the brand new pyproject.toml? Let’s see.
6 popular Python tools and their configs
Here are the 6 Python tools that I use daily both at my job and for the personal projects: pytest for testing, pytest-cov as a pytest plugin for coverage tool that calculates code coverage, mypy as a type checker, flake8 as a code style checker, black for code formatting, and isort for sorting imports.
In a table below you’ll find these tools along with their versions at the time of writing, popular and (arguably) universal config files as well as their native configuration file formats.
| tool | version | setup.cfg | pyproject.toml | tox.ini | custom format |
|---|---|---|---|---|---|
| pytest | 5.3.2 | yes | no (WIP) | yes | pytest.ini |
| pytest-cov | 2.8.1 | yes | no | yes | .coveragerc, pytest conf |
| mypy | 0.750.0 | yes | no (won’t fix?) | no | mypy.ini (preferred) |
| flake8 | 3.7.9 | yes | yes (unofficially) | yes | .flake8 |
| black | 19.10b0 | no | yes | no | --config any TOML file |
| isort | 4.3.21 | yes | yes | yes | .isort.cfg |
If there’s something to take in from the table, then it’s the following. First, there’s no consensus in the Python community over the usage of pyproject.toml. The most notable discussion involves Guido van Rossum in mypy issue on the GitHub. PEP 518 is all about configuration for package building systems like distutils, setuptools, flit, bento. It says nothing about configuration for tools outside of the package builing context. But what if package building process involves type checking or code formatting? Also, setup.cfg is a part of the setuptools which is a building system too. Still, people use it for third-party tools configuration outside of the package building context, e.g. for imports sorting checks in their CI pipelines. All in all, mypy has set a precedent for not supporting pyproject.toml on purpose, so that some other tools developers hesitate either to openly support the new config file like flake8 does, or just postpone implementation until a consensus is emerged in the Python community.
Secondly, there’s always a black sheep of the family. Being a great formatting tool that eventually has ended so many holy wars about code formatting, black doesn’t support anything except for TOML config files. Even if you are more inclined towards ol’ reliable setup.cfg, you still has a black sheep in your setup.cfg-compatible flock.
Conclusion
TOML is a great human-readable, machine-editable, strong typed, well-defined configuration format. Having Python- and setuptools-independent configuration file pyproject.toml is a great idea. Still, as the usage of pyproject.toml for non-package building context is not defined, we are in the xkcd how-standards-proliferate situation. Until consensus is finally set, it’s better off either use setup.cfg with an annoying exception for black, or stick to the tools custom config formats and files.