virtualenv is a tool to create isolated Python environments.
The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.7/site-packages (or whatever your platform’s standard location is), it’s easy to end up in a situation where you unintentionally upgrade an application that shouldn’t be upgraded.
Or more generally, what if you want to install an application and leave it be? If an application works, any change in its libraries or the versions of those libraries can break the application.
Also, what if you can’t install packages into the global site-packages directory? For instance, on a shared host.
In all these cases, virtualenv can help you. It creates an environment that has its own installation directories, that doesn’t share libraries with other virtualenv environments (and optionally doesn’t access the globally installed libraries either).
We advise installing virtualenv-1.9 or greater. Prior to version 1.9, the pip included in virtualenv did not not download from PyPI over SSL.
When using pip to install virtualenv, we advise using pip 1.3 or greater. Prior to version 1.3, pip did not not download from PyPI over SSL.
We advise against using easy_install to install virtualenv when using setuptools < 0.9.7, because easy_install didn’t download from PyPI over SSL and was broken in some subtle ways.
To install globally with pip (if you have pip 1.3 or greater installed globally):
$ [sudo] pip install virtualenv
Or to get the latest unreleased dev version:
$ [sudo] pip install https://github.com/pypa/virtualenv/tarball/develop
To install globally from source:
$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-X.X.tar.gz $ tar xvfz virtualenv-X.X.tar.gz $ cd virtualenv-X.X $ [sudo] python setup.py install
To use locally from source:
$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-X.X.tar.gz $ tar xvfz virtualenv-X.X.tar.gz $ cd virtualenv-X.X $ python virtualenv.py myVE
The virtualenv.py script is not supported if run without the necessary pip/setuptools/virtualenv distributions available locally. All of the installation methods above include a virtualenv_support directory alongside virtualenv.py which contains a complete set of pip and setuptools distributions, and so are fully supported.
The basic usage is:
$ virtualenv ENV
This creates ENV/lib/pythonX.X/site-packages, where any libraries you install will go. It also creates ENV/bin/python, which is a Python interpreter that uses this environment. Anytime you use that interpreter (including when a script has #!/path/to/ENV/bin/python in it) the libraries in that environment will be used.
It also installs Setuptools into the environment.
Virtualenv (<1.10) used to provide a --distribute option to use the setuptools fork Distribute. Since Distribute has been merged back into setuptools this option is now no-op, it will always use the improved setuptools releases.
A new virtualenv also includes the pip installer, so you can use ENV/bin/pip to install additional packages into the environment.
In a newly created virtualenv there will be a bin/activate shell script. For Windows systems, activation scripts are provided for CMD.exe and Powershell.
On Posix systems you can do:
$ source bin/activate
This will change your $PATH so its first entry is the virtualenv’s bin/ directory. (You have to use source because it changes your shell environment in-place.) This is all it does; it’s purely a convenience. If you directly run a script or the python interpreter from the virtualenv’s bin/ directory (e.g. path/to/env/bin/pip or /path/to/env/bin/python script.py) there’s no need for activation.
After activating an environment you can use the function deactivate to undo the changes to your $PATH.
The activate script will also modify your shell prompt to indicate which environment is currently active. You can disable this behavior, which can be useful if you have your own custom prompt that already displays the active environment name. To do so, set the VIRTUAL_ENV_DISABLE_PROMPT environment variable to any non-empty value before running the activate script.
On Windows you just do:
And type deactivate to undo the changes.
Based on your active shell (CMD.exe or Powershell.exe), Windows will use either activate.bat or activate.ps1 (as appropriate) to activate the virtual environment. If using Powershell, see the notes about code signing below.
If using Powershell, the activate script is subject to the execution policies on the system. By default on Windows 7, the system’s excution policy is set to Restricted, meaning no scripts like the activate script are allowed to be executed. But that can’t stop us from changing that slightly to allow it to be executed.
In order to use the script, you have to relax your system’s execution policy to AllSigned, meaning all scripts on the system must be digitally signed to be executed. Since the virtualenv activation script is signed by one of the authors (Jannis Leidel) this level of the execution policy suffices. As an administrator run:
PS C:\> Set-ExecutionPolicy AllSigned
Then you’ll be asked to trust the signer, when executing the script. You will be prompted with the following:
PS C:\> virtualenv .\foo New python executable in C:\foo\Scripts\python.exe Installing setuptools................done. Installing pip...................done. PS C:\> .\foo\scripts\activate Do you want to run software from this untrusted publisher? File C:\foo\scripts\activate.ps1 is published by Efirstname.lastname@example.org, CN=Jannis Leidel, L=Berlin, S=Berlin, C=DE, Description=581796-Gh7xfJxkxQSIO4E0 and is not trusted on your system. Only run scripts from trusted publishers. [V] Never run [D] Do not run [R] Run once [A] Always run [?] Help (default is "D"):A (foo) PS C:\>
If you select [A] Always Run, the certificate will be added to the Trusted Publishers of your user account, and will be trusted in this user’s context henceforth. If you select [R] Run Once, the script will be run, but you will be prometed on a subsequent invocation. Advanced users can add the signer’s certificate to the Trusted Publishers of the Computer account to apply to all users (though this technique is out of scope of this document).
Alternatively, you may relax the system execution policy to allow running of local scripts without verifying the code signature using the following:
PS C:\> Set-ExecutionPolicy RemoteSigned
Since the activate.ps1 script is generated locally for each virtualenv, it is not considered a remote script and can then be executed.
The --system-site-packages Option¶
If you build with virtualenv --system-site-packages ENV, your virtual environment will inherit packages from /usr/lib/python2.7/site-packages (or wherever your global site-packages directory is).
This can be used if you have control over the global site-packages directory, and you want to depend on the packages there. If you want isolation from the global system, do not use this flag.
Environment variables and configuration files¶
virtualenv can not only be configured by passing command line options such as --python but also by two other means:
Each command line option is automatically used to look for environment variables with the name format VIRTUALENV_<UPPER_NAME>. That means the name of the command line options are capitalized and have dashes ('-') replaced with underscores ('_').
For example, to automatically use a custom Python binary instead of the one virtualenv is run with you can also set an environment variable:
$ export VIRTUALENV_PYTHON=/opt/python-3.3/bin/python $ virtualenv ENV
It’s the same as passing the option to virtualenv directly:
$ virtualenv --python=/opt/python-3.3/bin/python ENV
This also works for appending command line options, like --find-links. Just leave an empty space between the passsed values, e.g.:
$ export VIRTUALENV_EXTRA_SEARCH_DIR="/path/to/dists /path/to/other/dists" $ virtualenv ENV
is the same as calling:
$ virtualenv --extra-search-dir=/path/to/dists --extra-search-dir=/path/to/other/dists ENV
virtualenv also looks for a standard ini config file. On Unix and Mac OS X that’s $HOME/.virtualenv/virtualenv.ini and on Windows, it’s %APPDATA%\virtualenv\virtualenv.ini.
The names of the settings are derived from the long command line option, e.g. the option --python would look like this:
[virtualenv] python = /opt/python-3.3/bin/python
Appending options like --extra-search-dir can be written on multiple lines:
[virtualenv] extra-search-dir = /path/to/dists /path/to/other/dists
Please have a look at the output of virtualenv --help for a full list of supported options.
Some paths within the virtualenv are slightly different on Windows: scripts and executables on Windows go in ENV\Scripts\ instead of ENV/bin/ and libraries go in ENV\Lib\ rather than ENV/lib/.
To create a virtualenv under a path with spaces in it on Windows, you’ll need the win32api library installed.
Creating Your Own Bootstrap Scripts¶
While this creates an environment, it doesn’t put anything into the environment. Developers may find it useful to distribute a script that sets up a particular environment, for example a script that installs a particular web application.
To create a script like this, call virtualenv.create_bootstrap_script(extra_text), and write the result to your new bootstrapping script. Here’s the documentation from the docstring:
Creates a bootstrap script, which is like this script but with extend_parser, adjust_options, and after_install hooks.
This returns a string that (written to disk of course) can be used as a bootstrap script with your own customizations. The script will be the standard virtualenv.py script, with your extra text added (your extra text should be Python code).
If you include these functions, they will be called:
- You can add or remove options from the parser here.
- adjust_options(options, args):
- You can change options here, or change the args (if you accept different kinds of arguments, be sure you modify args so it is only [DEST_DIR]).
After everything is installed, this function is called. This is probably the function you are most likely to use. An example would be:def after_install(options, home_dir): if sys.platform == 'win32': bin = 'Scripts' else: bin = 'bin' subprocess.call([join(home_dir, bin, 'easy_install'), 'MyPackage']) subprocess.call([join(home_dir, bin, 'my-package-script'), 'setup', home_dir])
This example immediately installs a package, and runs a setup script from that package.
Here’s a more concrete example of how you could use this:
import virtualenv, textwrap output = virtualenv.create_bootstrap_script(textwrap.dedent(""" import os, subprocess def after_install(options, home_dir): etc = join(home_dir, 'etc') if not os.path.exists(etc): os.makedirs(etc) subprocess.call([join(home_dir, 'bin', 'easy_install'), 'BlogApplication']) subprocess.call([join(home_dir, 'bin', 'paster'), 'make-config', 'BlogApplication', join(etc, 'blog.ini')]) subprocess.call([join(home_dir, 'bin', 'paster'), 'setup-app', join(etc, 'blog.ini')]) """)) f = open('blog-bootstrap.py', 'w').write(output)
Another example is available here.
Using Virtualenv without bin/python¶
Luckily, it’s easy. You must use the custom Python interpreter to install libraries. But to use libraries, you just have to be sure the path is correct. A script is available to correct the path. You can setup the environment like:
activate_this = '/path/to/env/bin/activate_this.py' execfile(activate_this, dict(__file__=activate_this))
This will change sys.path and even change sys.prefix, but also allow you to use an existing interpreter. Items in your environment will show up first on sys.path, before global items. However, global items will always be accessible (as if the --system-site-packages flag had been used in creating the environment, whether it was or not). Also, this cannot undo the activation of other environments, or modules that have been imported. You shouldn’t try to, for instance, activate an environment before a web request; you should activate one environment as early as possible, and not do it again in that process.
Making Environments Relocatable¶
Note: this option is somewhat experimental, and there are probably caveats that have not yet been identified.
The --relocatable option currently has a number of issues, and is not guaranteed to work in all circumstances. It is possible that the option will be deprecated in a future version of virtualenv.
Normally environments are tied to a specific path. That means that you cannot move an environment around or copy it to another computer. You can fix up an environment to make it relocatable with the command:
$ virtualenv --relocatable ENV
This will make some of the files created by setuptools use relative paths, and will change all the scripts to use activate_this.py instead of using the location of the Python interpreter to select the environment.
Note: scripts which have been made relocatable will only work if the virtualenv is activated, specifically the python executable from the virtualenv must be the first one on the system PATH. Also note that the activate scripts are not currently made relocatable by virtualenv --relocatable.
Note: you must run this after you’ve installed any packages into the environment. If you make an environment relocatable, then install a new package, you must run virtualenv --relocatable again.
Also, this does not make your packages cross-platform. You can move the directory around, but it can only be used on other similar computers. Some known environmental differences that can cause incompatibilities: a different version of Python, when one platform uses UCS2 for its internal unicode representation and another uses UCS4 (a compile-time option), obvious platform changes like Windows vs. Linux, or Intel vs. ARM, and if you have libraries that bind to C libraries on the system, if those C libraries are located somewhere different (either different versions, or a different filesystem layout).
If you use this flag to create an environment, currently, the --system-site-packages option will be implied.
The --extra-search-dir option¶
This option allows you to provide your own versions of setuptools and/or pip to use instead of the embedded versions that come with virtualenv.
To use this feature, pass one or more --extra-search-dir options to virtualenv like this:
$ virtualenv --extra-search-dir=/path/to/distributions ENV
The /path/to/distributions path should point to a directory that contains setuptools and/or pip wheels.
virtualenv will look for wheels in the specified directories, but will use pip’s standard algorithm for selecting the wheel to install, which looks for the latest compatible wheel.
As well as the extra directories, the search order includes:
- The virtualenv_support directory relative to virtualenv.py
- The directory where virtualenv.py is located.
- The current directory.
If no satisfactory local distributions are found, virtualenv will fail. Virtualenv will never download packages.
Compare & Contrast with Alternatives¶
There are several alternatives that create isolated environments:
workingenv (which I do not suggest you use anymore) is the predecessor to this library. It used the main Python interpreter, but relied on setting $PYTHONPATH to activate the environment. This causes problems when running Python scripts that aren’t part of the environment (e.g., a globally installed hg or bzr). It also conflicted a lot with Setuptools.
virtual-python is also a predecessor to this library. It uses only symlinks, so it couldn’t work on Windows. It also symlinks over the entire standard library and global site-packages. As a result, it won’t see new additions to the global site-packages.
This script only symlinks a small portion of the standard library into the environment, and so on Windows it is feasible to simply copy these files over. Also, it creates a new/empty site-packages and also adds the global site-packages to the path, so updates are tracked separately. This script also installs Setuptools automatically, saving a step and avoiding the need for network access.
zc.buildout doesn’t create an isolated Python environment in the same style, but achieves similar results through a declarative config file that sets up scripts with very particular packages. As a declarative system, it is somewhat easier to repeat and manage, but more difficult to experiment with. zc.buildout includes the ability to setup non-Python systems (e.g., a database server or an Apache instance).
I strongly recommend anyone doing application development or deployment use one of these tools.
Virtualenv’s release schedule is tied to pip’s – each time there’s a new pip release, there will be a new virtualenv release that bundles the new version of pip.
Files in the virtualenv_embedded/ subdirectory are embedded into virtualenv.py itself as base64-encoded strings (in order to support single-file use of virtualenv.py without installing it). If your patch changes any file in virtualenv_embedded/, run bin/rebuild-script.py to update the embedded version of that file in virtualenv.py; commit that and submit it as part of your patch / pull request.
Running the tests¶
Virtualenv’s test suite is small and not yet at all comprehensive, but we aim to grow it.
The easy way to run tests (handles test dependencies automatically):
$ python setup.py test
If you want to run only a selection of the tests, you’ll need to run them directly with nose instead. Create a virtualenv, and install required packages:
$ pip install nose mock
Or select just a single test file to run:
$ nosetests tests.test_virtualenv