PythonUp — The Python Runtime Manager (Windows)

PythonUp helps your download, configure, install, and manage Python runtimes. It also provides utilities that can be integrated into your Python-related development workflows. This is the Windows version.

macOS or Linux user? Check out PythonUp for POSIX.

Distribution

PythonUp for Windows is officially distributed on GitHub. Download installers from Releases and run it. After installation, a pythonup command will be available in newly-opened command prompts.

Quick Start

Install Python 3.6:

$ pythonup install 3.6

Run Python:

$ python3

Install Pipenv to Python 3.6:

$ pip3.6 install pipenv

And use it immediately:

$ pipenv --version
pipenv, version 9.0.1

Install Python 3.5 (32-bit):

$ pythonup install 3.5-32

Switch to a specific version:

$ pythonup use 3.5-32
$ python3 --version
Python 3.5.4

Switch back to 3.6:

$ pythonup use 3.6
$ python3 --version
Python 3.6.4
$ python3.5 --version
Python 3.5.4

Uninstall Python:

$ pythonup uninstall 3.5-32

Use --help to find more:

$ pythonup --help
$ pythonup install --help

Or read the Documentation.

Now you’re ready to use CPython on Windows like a first-class citizen, and ignore people telling you to get a Mac.

🤔😉😆

Topics

Install Python

PythonUp installs a Python version with a single command. To install <version> on you machine: [1]

pythonup install <version>

[1]	Use pythonup list --all to find out what versions are available. See List Pythons for more information about the list command.

This automatically downloads the installer, and runs it with minimal user input possible. [2] If there’s need to install without Internet connection, you can download the installer yourself, and run:

pythonup install --file=path\to\installer.exe <version>

to install directly from that installer. Either way, PythonUp sets up the Python installation on its own, and let you start using it right away.

[2]	Generally only to confirm the UAC dialog, if needed.

PythonUp provides some extra executables for you. Say you have install Python 3.6 after you set up PythonUp. Now you can launch Python with:

python3

Install a package, say, Pipenv, with:

pip3 install pipenv

and have the pipenv command available immediately after.

Upgrade Python

PythonUp allows you to upgrade a Python installation to a newer micro version, e.g. 3.6.3 to 3.6.4. Run the following command to upgrade an installed version (if available):

pythonup upgrade <version>

It takes some time for the developers to update the definition to a newer version. If you find a newer version released on python.org that is not available in PythonUp, send a pull request to update the definition files!

Uninstall Python

You probably guessed it:

pythonup uninstall <version>

Similar to installing, this does nothing too fancy, but just runs the standard uninstaller. It does perform some additional cleanup if you are using the version. See Manage Pythons about how you can manage/use versions.

Manage Pythons

The real benefits of PythonUp shows when you install more than one Python versions. Instead of manipulating the PATH environment variable, simply run the command with the appropriate version name.

# Python 3.6 (64-bit on available hosts, 32-bit otherwise).
pythonup install 3.6
python3.6
pip3.6

# Python 3.5 (force 32-bit).
pythonup install 3.5-32
python3.5-32
pip3.5-32

# Python 2.7.
pythonup install 2.7
python2.7
pip2.7

No more python.exe shadowing because you have multiple versions in PATH.

Use Versions

When you have multiple Python versions installed, PythonUp only exposes the above two executables by default. If you want to expose other commands of a given Python version, You can tell PythonUp to use it: [1]

pythonup use 3.6

[1]	PythonUp does this automatically when you install your first ever Python version. This is why we had access to python3 without using 3.6 in Install Python.

So PythonUp publishes (almost) all commands available in Python 3.6, including

• python3.exe
• pip3.exe
• easy_install-3.6.exe
• All other scripts you installed via pip install.

As an exception, PythonUp blacklists python.exe, pip.exe, and easy_install.exe from being published, to encourage the best practice of always using versioned Python and Pip commands.

You can use multiple versions together to expose scripts installed across them:

pythonup use 3.6 2.7

This exposes commands from both Python 3.6 and 2.7, with 3.6 taking precedence, i.e. if a given command exists in both versions, the 3.6 one will be called.

Manage Used Versions

To see what versions are currently in use:

pythonup use

To reset using state (i.e. unuse all versions):

pythonup use --reset

Misc. Commands

Upgrade PythonUp Itself

A special upgrade syntax to upgrade not a Python version, but PythonUp itself:

pythonup upgrade self

This simply downloads the official installer and runs it.

List Pythons

This lists Python versions installed in your system:

pythonup list

To list all Python versions available, including uninstalled ones, use:

pythonup list --all

Either way, the output would be something like this:

o 2.7
o 3.4
  3.5
* 3.6

• The o prefix means the version is installed.
• * signifies an active version.
• No prefix if the version is not installed.

Download Python

pythonup download <version>

downloads the installer without exicuting it. The installer is saved to the current working directory by default, but you can also specify another directory with the --dest option.

Find Python Installation

pythonup where <version>

prints where the installed python.exe really is, usually something like:

C:\Users\username\AppData\Local\Programs\Python\PythonXY\python.exe

This is useful when you need to pass the real executable somewhere else, e.g. set the path to an environment variable in a Powershell script.

Link Individual Scripts

pythonup link <command>

links the specified command to your PATH. Nice to have when you accidetally break the system. There are --overwrite=yes and --all you can use for even better profit.

Release History

Next (not released)

Nothing yet.

Unstable

Nothing yet.

Version 3.0.10

• Upgrade Bleach to 3.3.0.
• Add Python 3.10 definition (3.10.0).
• Upgrade Python 3.9 definition to 3.9.8.
• Upgrade Python 3.8 definition to 3.8.10.

Version 3.0.9

• Upgrade Bleach to 3.1.4.
• Add Python 3.9 definition (3.9.0).
• Upgrade Python 3.8 definition to 3.8.6.
• Upgrade Python 3.7 definition to 3.7.9.
• The bundled Python (used to run PythonUp) is upgraded to 3.6.8.

Version 3.0.8

• Upgrade Python 3.8 definition to 3.8.5.
• Upgrade Python 3.7 definition to 3.7.8.

Version 3.0.7

• Upgrade Bleach to 3.1.1.
• Upgrade Python 3.8 definition to 3.8.2.
• Upgrade Python 3.7 definition to 3.7.7.

Version 3.0.6

• Fix attrs compatibility.

Version 3.0.5

Behavioural Changes

• Add Python 3.8 definition (3.8.0).
• Upgrade Python 3.7 definition to 3.7.5.

Version 3.0.4

Behavioural Changes

• Upgrade Python 3.7 definition to 3.7.3.

Verson 3.0.3

Behavioural Changes

• Upgrade Python 3.6 definition to 3.6.8.

UI Changes

• pip shims no longer displays “Not using any versions”.
• Fix a tense difference in command help messages.

Version 3.0.2

Behavioural Changes

• Upgrade Python 3.6 definition to 3.6.7.
• Upgrade Python 3.7 definition to 3.7.1.

Installer Changes

• The bundled Python (used to run PythonUp) is upgraded to 3.6.7.

Version 3.0.1

Fix pythonup upgrade self. The repository moved.

Version 3.0

Renamed project from SNAFU to PythonUp. The entry command is also renamed accordingly. Installation target is now %LOCALAPPDATA%\Programs\PythonUp.

UI Changes

• The main command is renamed to pythonup.

Behavioural Changes

• The scripts PATH is moved ahead of the cmd. This provides potential for more flexible customisations, i.e. if multiple sources install the same executable (CPython and Anaconda, for example), one (CPython) can take precedence in cmd, but allow the user to override this by the use command.
• PythonUp now works system-wide Python installations, and can publish shims for them as well as snafu install-ed ones. It still only supports installing them in per-user mode, but other commands should mostly work.
  EXCEPTION: Upgrading an MSI-based Python version (3.4 or earlier) is not supported.
• Bugs are fixed to correctly detect various registry values.
• The shims are updated to work independently from registry values. The active version registry values are removed in favour of an installation-local configuration file. This should not affect how the user interacts with PythonUp.
• Upgrade Python 2.7 definition to 2.7.15.
• Upgrade Python 3.6 definition to 3.6.6.
• Add Python 3.7 definition (3.7.0).

Installer Changes

• The bundled Python (used to run PythonUp) is upgraded to 3.6.6.

Version 2.0

UI Changes

• Remove confusing activate and deactivate commands in favour of the use command. The previous “reactivation” behaviour is now mapped to link --all.
• A newly installed Python will be used if there are no other versions detected.
• Add snafu upgrade self to perform in-place upgrade without manually downloading the installer.
• Add snafu download <version> to download installer of given Python version without installing.
• More complete help messages are provided to command arguments.
• The uninstalling processes are now as interaction-free as installing. Previously some user intervention was needed (especially the EXE-based versions).

Behavioural Changes

• snafu install now automatically uses the version if it is the only Python installation detected. This should simplify things even more for beginners.
• Minor update to Python 3.4 definition.
• Call snafu link --all automatically after installing with a shimmed pip command.
• Upgrade Python 3.4 definition to 3.4.4.
• Upgrade Python 3.6 definition to 3.6.4.
• Improve behaviour when uninstalling versions not installed by SNAFU.
• snafu link --all without active versions does not fail anymore. The warning message is still printed, but the exit code is now 0.
• Py Launcher usage is reduced in favour of reading the registry ourselves.
• Documentation is no longer installed with each Python version.

Installer Changes

• Add an option to install and use a Python version after SNAFU is set up.
• 64-bit variant does not carry x86 MSU files anymore, reducing installer size.
• Correctly install Windows update KB2999226 on 32-bit Windows.

Version 1.0

UI Changes

• New upgrade command to install a newer patch version on an installed version.
• New link command to manually publish a script from the active Python versions.

Behavioural Changes

• pip-X.Y are now published on install.
• Automatically deactivate an uninstalling active version.
• A Python exception will be raised early on download error, instead of failing later during installation.
• Uninstallation now skips gracefully if launcher scripts do not exist.

Installer Changes

• Bundle and trigger Windows update KB2999226 on installation to provide necessary runtime files on Windows Vista to 8.1 so the bundled Python 3.6 can run correctly.
• Environment variables changes can now propagate correctly without OS restart.

Version 0.2

UI Changes

• uninstall now attempts to use Windows’s uninstall feature to avoid re-download.
• install and uninstall receives a --file option to allow using local installers without re-downloading.
  IMPORTANT: Correctness of the installer is not checked. The user is responsible for choosing the correct and valid installer file. Results of installing a faulty installer is undefined.
• New command where to show where the actual interpreter is. This is useful if you need to pass it to another command (e.g. pipenv --python).
• list shows activeness.
• snafu --version shows program version.

Behavioural Changes

• activate writes a pin file showing the current active versions.
• Symbols in snafu list are changed.

Installer Changes

• Environment variables are now set up automatically during installation.
• Extract the py launcher MSI to make the distribution substentially smaller.
• The installer now comes with both 64- and 32-bit flavours.
• The uninstaller is now added to registry, so you can remove SNAFU in Control Panel.

Version 0.1

Initial release. Features I want the most are largely implemented. Only 64-bit Pythons are supported for now, and the installer is 64-bit-only.

An all-in-one installer that installs SNAFU into %LOCALAPPDATA%\Programs\SNAFU, and sets up the py launcher.

• snafu install/uninstall <version>
• snafu list [--all]
• snafu activate/deactivate <version> [<version> ...]

Contribute to PythonUp for Windows

Development happens on GitHub.

Development Guide

Requirements

• Windows
• Python 3.6
• Pipenv

Optional Dependencies

• Rust if you want to build the shims. The Rust development environment needs to be available in your shell. PythonUp targets the stable channel.
• NSIS 3.x if you want to build the installer. makensis needs to be available in your shell.

Project Setup

Download and enter the project:

git clone https://github.com/uranusjr/pythonup-windows.git
cd pythonup-windows

Set up environment:

pipenv install --dev

Run Tests

Run Python tests:

pipenv run pytest tests

Run Rust tests:

pipenv run invoke shims.test

Unfortunately there are only very limited tests right now.

Run In-Development PythonUp

pipenv run python -m pythonup [COMMAND] ...

This should have the same behaviour as an installed command, but within the confine of the Pipenv-managed virtual environment.

Warning

PythonUp depends a lot on the Windows Registry, so certain commands still have global implications. For example, the uninstall command will uninstall Python from your system, and use will affect your global using state!

Build Installer

pipenv run invoke installers.build

You can only build installers of your host’s architecture. Cross compilation is certainly possible, but I haven’t found the need to set it up.

After the command finishes you should get an EXE in the installers directory.

Build Documentation

pipenv run invoke docs.build

Documentation is managed with Sphinx, and hosted on Read the Docs with a custom domain.

Source Code Guideline

Try to follow the code style. For Python code, run the linter to check for issues before submitting:

pipenv run flake8 .

Format of text files are managed with EditorConfig. I recommend using one of the editor plugins to automatically format files. If you can’t/don’t want to do so, please at least make sure you’re using the correct format before sending in pull requests.

Explain PythonUp

The idea of PythonUp formed gradually from problems I faced when working on multiple development machines, constantly switching between Windows and macOS. I long a more consistent development experience, and a good way to manage my Python environment on Windows.

I tried out quite a few solutions. While a lot of them work for myself (to varying degree), none of them make me feel comfortable because I can’t teach anybody about them, which means they are all somehow wrong. Eventually I decided I need a complete managing system, which became PythonUp.

Below are some questions I either faced when discussing the Python setup problem with others, or through introspection. Each question represents an alternative solution I tried, but eventually couldn’t feel satisfied with. I hope they will answer why I think PythonUp is the best solution, and you should adapt it as well.

Why Not “Add Python to PATH”?

The standard CPython installer offers this option. It is not checked by default, but a lot of online tutorials tell you to do so. The CPython core team is correct; it shouldn’t be checked under most cercumstances.

CPython’s standard Windows build, unlike on Un*x, does not provide the “altinstall” build option. Every CPython distribution on Windows has only one Python executable, called python.exe, not versioned names such as python3.6.exe.

Adding Python to PATH stops being a good idea the moment you install a second Python. You can only access one Python at a time, and installed scripts from different versions start to mix, which is a bad thing. [1] It also become very tedious and delicate very quickly to manipulate the PATH environment variable.

[1]	This is not a Windows-only problem, but also exactly why tutorials these days don’t recommand installing Python via python.org, but with platform-specific tools instead. Windows is the only mainstream operation system without a good Python verions management tool.

Why Not the Python Launcher (py.exe)?

Python introduced PEP 397 partly to solve the python.exe problem (also to interpret the shebang line on Windows). It installs a py.exe to your PATH, and instead of invoking python.exe directly, you should use, for example:

py -3.5 foo.py

to run foo.py with Python 3.5.

This is such a good idea PythonUp installs the Py Launcher during setup, and I encourage you to use it. But PythonUp also solves a few additional use cases that py.exe doesn’t:

• Availability of versioned Python executables, e.g. python3.6.exe.
• Managing commands other than python.exe.

PythonUp’s implementation also relies on a lot of the same registry values read by py.exe, which are formally defined in PEP 514, so you can view PythonUp as a supplement to py.exe, not a replacement.

Why Not Install to All Users?

Authentication on Windows is difficult to manage, especially in the console. There really is no way to elevate priviledge inside a command prompt, and it is against common workflow to require opening a new console when you need to pip install something.

It is useful to have a system-wide Python setup. For developers, however, it is always recommended to install a seperate copy of Python for yourself, so you can manage it directly, without special priviledge.

Why Not Chocolatey?

Chocolatey is a package manager on Windows. A lot of PythonUp’s ideas are inspired by it: standard Windows installers, interaction-free installation, and shims for execution in command prompts. It is a very good tool, and I use it on my Windows machine—alongside PythonUp.

What PythonUp is to Chocolatey is similar to pyenv to Homebrew on macOS. The aims are similar, but slightly different, so we can take an approach tailored to Python distribution.

Also I’m not very satisfied with Chocolatey’s user story. The setup is slightly complicated (due to Powershell’s execution policy), and requires administration priviledge to install packages. This is because it is fulfilling a different goal from PythonUp’s, but still makes me feel uncomforatble enough not to teach it to others.

Why Not Anaconda?

I considered Anaconda very hardly, but eventually decided against it. The tipping point is how Anaconda manages Python versions similar to virtualenvs, with manipulation of PATH and other environment variables. This is the wrong way to do it, [2] and to this day there is still no first-party Powershell support. [3]

[2]	Virtualenv’s bin/activate is doing it wrong

[3]	Powershell activate and deactivate

There is also no simple way (AFAIK) to run a particular version of Python in Anaconda with just one command (except specifying the absolute path to python.exe). You always need to activate the environment (and remember to deactivate afterwards). It just feels so tedious to me.

In the end, the activate/deactivate approach requires more from the user mentally. It is an extra thing to wrap your head around; tedious for the experienced, and sometimes even too difficult for newcomers. This is why Pipenv catches on so fast—virtualenv is wonderful, but everybody has issues with the user interface they long for a better solution.

Anaconda is incredibly useful for those who need it, but its version management tooling is simply not good enough. Fortunately, since Anaconda already conforms to PEP 514, there’s nothing theoratically preventing PythonUp to co-live with Anaconda, so people can use PythonUp to manage Anaconda versions instead.

Implement PythonUp

Some implementation notes about PythonUp, in a series of questions. Nobody ever asked, but I want to answer them.

How are Pythons installed?

The official CPython installers are downloaded, and executed in a non-interactive manner. Check out the relevant documentation for more details:

• https://www.python.org/download/releases/2.5/msi/
• https://docs.python.org/3/using/windows.html#installing-without-ui

Where are Pythons installed?

%LOCALAPPDATA%\Programs\Python\<version>. This is the standard “only-for-me” installation location for Python 3.5+, and we retrofit this rule to older versions as well for consistency.

How are executables linked?

They are not. Scripts are copied. .py files works as well because they have appropriate shebang lines, and can be handled by the Python Launcher, as specified in PEP 397.

A few wrapper executables (shims) are distributed with PythonUp, and are published into PATH to stub a few special executables, such as python.exe and pip.exe. When invoked, these shims rely on the registry to launch their real conterparts, and bridge all user interaction to them.

The shims minimise the need to expose internal DLLs, and, in the case of pip.exe etc., provide a chance to hook into extra machinery when you alter Python installations. This is inspired by pyenv and Chocolatey, and provides a more seamless experience.