Python Virtual Environment

Most beginner developers have a hard time understanding virtual environments, why they are useful and how to use them. I am going to explain why you want to use them, and then cover how to use them. There are a couple ways to use virtual environments, all solve the same problems but are set up differently.

What is a virtual environment?

When you install a package with pip (or conda) you install that package in the current environment. If you do not have a virtual environment, that ends up being root. Installing packages in root is considered bad practice as different packages (and your own code) may require different versions of a package that may not work with other packages in other projects.

Packages you will use globally on your operating system are typically installed in root, anything that is specific to a project should be installed in a virtual environment. A virtual environment simulates a fresh install of python where you can install only the packages required for your project and freeze the version as needed. Freezing a version means you install a version of a module like 1.5 and do not upgrade to future versions prior to testing the impact. A virtual environment also keeps all your modules from impacting other projects.

One project may need requests 2.18.4 (the current latest version), but that version may cause problems (until a particular bug that affects your project is fixed). Using virtual environments you can install two different versions and keep both projects happy. Your code is only part of the problems, your dependencies can have dependencies.

Setting up and using a virtual environment.

Before explaining how to set up a virtual environment, I will give an overview of the process, then I will cover each step in detail.

Using a virtual environment

Create Virtual Environment
Activate Virtual Environment
Install Packages Requirements

Creating a virtual environment

There are quite a few ways to set up a virtual environment. A few common ways are:

virtualenv
conda
pipenv

There are a few more but they are additive to the ones listed above.

virtualenv

virtualenv is the defacto standard for virtual environments. It is fairly easy to use but can be confusing to remember the syntax and until you have done it a lot of times you will need to look up the process. Normally I would recommend and teach you this method.

conda

If you are using Anaconda/Miniconda (I highly recommend you do!) then you have conda available to you. It is a package manager like pip but even more powerful. If you are using Anaconda I would typically recommend this over virtualenv, but this has changed with the final option available to us.

pipenv

This is the final option I am presenting to you, and it is the one I will cover in detail and recommend you use as well. Pipenv is a newer project that replaces virtualenv and makes the entire process easier and cleaner. It acts a bit like Nodejs's superiornpm package manager. pipenv combines pip and virtualenv into one easy to use tool. It also has some security benefits as it uses hashes, something you won't need to worry about usually.

You will most likely need to install pipenv as it is not usually included in most python distributions by default. This will be used as a systemwide package and will not be installed in a virtual environment.

Install `pipenv`

python3 -m pip install pipenv

Once pipenv is installed, creating and using environments is fun and easy.

Creating a virtual environment

Change to the directory of your project or create a new folder if you are starting a new project. From there you just use pipenv like you would use for pip. If it is the first time installing a package in that folder, a new Pipfile and Pipefile.lock file. These files will manage what packages and versions you have installed for your project (this folder and downward).

After you install the packages you need for your project using pipenv install [PACKAGE]. Once you have finished installing all the packages you need for your project you will need to activate your virtual environment. pipenv will remind you after each package is installed wiht the following message:

To activate this project's virtualenv, run the following:
$ pipenv shell

Just use pipenv shell to switch to your virtual environment. While using your virtual environment you will see the name of the virtual environment on the left of your command prompt.

That's it, you now have a virtual environment with just the packages you installed. The versions of all the packages is maintained by pipenv and will ensure no other project will impact this project.

You can use pipenv graph to list the packages you have installed in your virtual environment.

Exit virtual environment

When you are done with this project or want to switch virtual environments, you can just type in exit and you will be returned to a normal command prompt.

But wait! There's more!

One thing that pipenv brings to the table that Nodejs also does very well is dev dependencies. These are packages you only need during development but will not be installed during production.

When installing a package that should only be used for development (say for example a package that reloads a server when the code base changes or a test framework) you use the --dev parameter.

pipenv install pytest --dev

You don't want a test framework installed in production, this allows you to maintain a separate list of packages that will not be installed unless you use pipenv install --dev to install all de v dependencies. This is extremely useful when working on multiple machines or with multiple users.

This should give you everything you need to create and use virtual environments to create safe project environments during your python development. If you have any questions or need more information, check out the pipenv project page.