Renaming

Overview¶

Python packages found on pypi.org and installed using pip often have the same name as packages found on anaconda.org and installed using conda. For instance, you can install pylint using:

$ pip install pylint

$ conda install -c conda-forge pylint

However, pip and conda use totally separate packaging systems and there is no inherent requirement that package names match between the two, and indeed there are quite a few exceptions, for example:

$ pip install numpy-quaternion 
$ pip install tables
$ pip install torch

$ conda install -c conda-forge quaternion 
$ conda install -c conda-forge pytables
$ conda install -c conda-forge pytorch

So when converting from wheels to conda packages, it is important that these differences be handled by the tool.

Standard renames¶

The whl2conda tool maintains a table of automatic renaming rules that is taken from mappings collected automatically from tools supporting the public conda-forge repository. whl2conda includes a static copy of this table collated when the whl2conda package was built, but also supports the ability to maintain and update a locally cached copy dynamically. If there is a package new to conda-forge that may have appeared since installing whl2conda, you can update your local cache using:

$ whl2conda config --update-std-renames

You can also configure whl2conda to automatically update the standard rename mappings prior to operations that require them via a persistent user setting:

$ whl2conda config --set auto-update-std-renames true

The cache file is kept in a location in your user directory that is specific to your operating system:

• Linux: ~/.cache/whl2conda/stdrename.json
• MacOS: ~/Library/Caches/whl2conda/stdrename.json
• Windows: ~\AppData\Local\whl2conda\Cache\stdrename.json

This file is simply a JSON dictionary mapping pypi names to conda names.

If you want to generate a copy of this file in another location (e.g. for use by other development tools), you can add a path to the --update-std-renames option:

$ whl2conda config --update-std-renames pypi-conda-renames.json

Manual rules¶

The implicit standard renaming support takes care of most renaming issues for publicly available packages, but may not work for packages that come from alternative channels, which may be private to your organization.

If you encounter these, you can add command line options to whl2conda convert to rename packages and also to add or drop packages. You may also rename the package you are building.

Adding extra packages¶

You can add one or more extra package dependencies using the -A / --add-dependency option. This can be a conda package name and version spec, e.g.:

$ whl2conda convert -A 'extra-package >=1.2.3' ...

You can use this to add dependencies for conda packages that perhaps do not exist on pypi.

Dropping a package¶

Likewise, you can drop packages using -D / --drop-dependency with just the package name:

$ whl2conda convert -D 'some-pypi-only-package'

This option also allows you to use python regular expressions to drop any package that matches a pattern:

$ whl2conda convert -D 'acme-.*'

Renaming dependencies¶

To rename dependencies, use -R / --dependency-rename with two arguments, the pypi name followed by the conda name. You can also use regular expressions with capture groups, where $<n> will be replaced with the nth capture and ${name} will be replaced with the named capture group with given name.

$ whl2conda convert -R acme-widgets acme.widgets

$ whl2conda convert -R 'acme-(.*)' 'acme.$1'

$ whl2conda convert -R 'acme-(?P<part>.*)' 'acme.${part}'

Dependencies with extras¶

Conda packages cannot express pip extras, so by default the extras are dropped with a warning from dependencies of the form name[extra,...] — only the base package dependency is kept.

Rename rules are matched against the bracketed form of such dependencies (as written in the wheel metadata) before the bare name, so where a corresponding conda package exists — for example, on conda-forge the dask metapackage corresponds to dask[complete], while plain dask corresponds to dask-core — you can map the dependency explicitly, which also suppresses the warning:

$ whl2conda convert -R 'dask\[complete\]' dask -R dask dask-core

Note that the square brackets are escaped because the pattern is a regular expression, in which unescaped brackets would denote a character class — this is independent of any shell quoting. Alternatively, the extra's dependencies can be added individually using -A / --add-dependency.

whl2conda also knows about a number of common extras that have a dedicated corresponding package on conda-forge, including:

When such a dependency is encountered, the warning will point out the corresponding conda package, and the --known-extras option will apply these replacements automatically:

$ whl2conda convert --known-extras ...

Extras not handled by any of the above can be resolved from pypi.org metadata using the --resolve-extras option, which requires network access:

$ whl2conda convert --resolve-extras ...

This reads the extra's dependencies from the newest release of the package that satisfies the dependency's version spec, converts them like regular dependencies, and recursively resolves any extras they use in turn.

Renaming converted package¶

By default, the name of the generated conda package will be taken from the project name in the pyproject.toml if there is one, otherwise from the name in the wheel. This can be overridden using the --name command line option:

$ whl2conda convert acme-widgets-1.2.3-py3-none-any.whl --name acme-pywidgets

Specifying rules in pyproject.toml file¶

If you are using a pyproject.toml file for your project, you can instead specify how dependencies are modified in the tool options. This is described in the next section.

Hiding pip dependencies in dist-info directory for conda¶

The standard method of building python conda package involves running pip install or the equivalent, which results in a the package's original pip dependencies and other metadata being saved to the METADATA file in the package's .dist-info directory in site-packages/ for the environment in which it is installed. However, in a conda environment, these dependencies may seem to be incompatible with what is actually installed by conda. This can sometimes cause serious problems if pip or other standard python packaging tools attempt to check or update these dependencies.

Since the actual dependencies for packages installed by conda are described by the conda package's metadata, and not the metadata saved in the .dist-info, whl2conda by default will turn all regular dependencies in the dist-info into extra dependencies using the name original. So if you look at the METADATA file in the dist-info of the generated conda package, you will see entries like:

Requires-Dist: some-package >=1.2; extra == 'original'

If you want to leave these dependencies unchanged you can use the --keep-pip-dependencies option to whl2conda convert. This is not recommended unless you know that you need them.