Deploying python applications
published on Wednesday, March 13, 2019
I've been using python for years, but never quite figured out (until now) how to create an easily deployable windows application without relying on internet access at installation time. The user should be able to just unzip or install in a certain location and it works.
There are a number of packagers such as PyInstaller, cx_Freeze, pyqtdeploy, py2exe, nuitka which apparently work very well for many. For me they didn't. Be it due to unclear documentation, complex package specification or build process, or even build or runtime errors, I gave up quickly. Apart from that, I have requirements that were usually not explained (or even supported): multiple executables, package data files, source code installation of selected packages (so it is possible to inspect and modify on the target machine). All this would probably be possible to setup with more tenacity by learning more about the specification scripts/languages, but I have found an easier solution for me!
If you're looking through the windows downloads list at python.org, you'll notice that there is now an embeddable zip file. This contains a small python distribution without any fuzz. It is just about 13MB big, after extraction. Perfect for redistribution with your application. There is already a great blog article, but I will go in a bit more detail here. The plan is as follows:
On the developer machine:
- download and extract embeddable python to pkg/
- install your python modules to pkg/lib/site-packages
- add launchers for your application pkg/*.exe
- zip up the pkg/ directory, or create an installer (e.g. with nsis)
On the user machine:
- extract or install the application
On the developer machine, I strongly recommend using miniconda (or anaconda) as build environment. This will make it very easy to acquire non-python dependencies and manage python environments.
Let's start by setting up our own python environment in a local folder. Open a conda terminal and type:
In the interactive terminal you don't have to type call here, but it becomes necessary when putting this in a .bat script. For consistency, I write it on other command executions as well.
Next, acquire the embeddable python runtime:
I like to extract everything into a pkg directory, that can later just be zipped up and distributed.
If you are hesitant to ship python along with the application (as I initially was), there are many advantages to this, such as:
- the user doesn't have to install python manually
- no interference from other installations
- no package conflicts with other applications
If your application has any python dependencies, you will probably want to install them as site-packages. In order to allow python to find the site-packages directory, we have to take care of the following detail:
Do not add quotes or spaces around 'import site' as you might do on linux! It will mess up the required format!
I prefer to do a two stage setup: First, download or create wheels for all required python packages, and then install the wheels into a target folder. This makes it easier to skip the whole download/wheel building step if you mess something up and in principle allows redistributing the wheels directly for installation on the target machine:
You can put only the name and version of your application in the requirements file, in which case pip will automatically install dependencies in the newest available version. If you want to lock dependencies from a tested configuration, the requirements file can be generated with pip freeze > requirements.txt.
Depending on your needs, there may be extra steps here. For example, if you haven't uploaded your package to PyPI, you might need additional build instructions or may need to bundle additional files.
We will now add an app.exe that launches your application. In the simplest case, where clicking your exe should do the same as typing python -m app on the command line, the following code is enough:
For more advanced use cases, read Embedding Python in Another Application.
You can compile and link this against python37.dll with the compiler of your choice. It is advisable to use the compiler that is officially used to build python on windows, see Which Microsoft Visual C++ compiler to use with a specific Python version? or Windows Compilers.
For unrelated reasons, I personally use mingwpy instead. This package can be conveniently installed via conda, however is available only up to python 3.4, one has to create a separate environment with mingwpy in it:
Next, compile as follows:
The -mwindows flag is used to prevent a console window from popping up with your application (assuming you run a GUI application, otherwise just remove this flag). If you want to pop up a console window only in certain cases, you could use the AllocConsole function in the WinAPI.
If you are following my (rather bad) example to use mingwpy, also read the section on CRT issues. Apart from that you now have a fully functional and portable application in the pkg/ folder that you can start using, or zip it up and transport to another machine.
Note that the version of mingw used above is tailored to python 3.4 and therefore links against msvcr100.dll that is not included with python 3.7 – which will result in startup errors if the DLL is not already present on the target machine by some happy accident. This problem can be alleviated by copying the DLL to your package distribution:
Furthermore, you have to make absolutetly sure not to pass CRT objects from your C code to python, this will crash your application at runtime.
If you are still unwilling to use the official compiler, I will now show the don't-do-this-at-home solution:
gcc can be instructed not to link against its default set of standard runtime libraries by passing the -nostdlib flag. However, in this case, no startup code will be executed, you are completely on your own with CRT initialization and global variable initialization. This is therefore not recommended if you are not aware that things can go terribly wrong... Let's get going!
Change your launcher's main function to WinMainCRTStartup, e.g.:
and add -nostdlib -lkernel32 -lshell32 to your linker flags:
In order to add an icon to your EXE, first create a app.rc file with the name of your icon, e.g.:
then compile this to a .res file:
All that is left to do, is to add app.res to the list of source files on your gcc command line when building the EXE, e.g.: