Customize Consent Preferences

We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.

The cookies that are categorized as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... 

Always Active

Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

No cookies to display.

Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

No cookies to display.

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

No cookies to display.

Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.

No cookies to display.

Advertisement cookies are used to provide visitors with customized advertisements based on the pages you visited previously and to analyze the effectiveness of the ad campaigns.

No cookies to display.

Using Container Software

One of the amazingly useful current trends in software development is “containerization“. This describes setting up a self-contained environment on a host computer system, that can run a separate operating system (OS), contain data and software not usually available otherwise. I find this very appealing e.g. for

  • testing a software package that is not available for my usual OS
  • testing my own software on a different OS
  • developing or running an analysis in fully reproducible settings
  • sharing software or analysis with reproducible settings

The main players in the field are Docker and Singularity. There are pros and cons for each, Singularity might be better suited for shared environments as you can run the containers with standard user rights.
The general idea of containerization is similar to virtual machines, this Docker page explains the differences.

Docker

To install Docker on an Ubuntu system, currently the following commands will do the trick:

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable" 
sudo apt-get update
sudo apt-get install -y docker-ce

Installation of Docker on Mac OSX is straightforward with the download of the free community edition (CE) from here. Make sure you either add the software as a log-in item or manually start it before you attempt to use it. (You will see the little Docker whale in the task bar. )

Some key Docker commands are

# show all images available locally:
docker images
# show all containers that are already running:
docker ps
# start a new container from an image that can be fetched from the remote Docker repository.
# a simple test:
docker run hello-world
# or a small Linux system:
docker run --rm -it phusion/baseimage:0.9.22 /sbin/my_init -- bash 

A good way to move data in and out of the container is by mounting a specific directory, e.g./home/felix/testproject to the tmp folder:

docker run --rm -it --mount type=bind,source=/home/felix/testproject,target=/tmp phusion/baseimage:0.9.22 /sbin/my_init -- bash

The standard way to create new images is by defining all installation steps in a Dockerfile. However, to share a pre-build environment with your own data it is sometimes easier to freeze the container you have build up. Your can do this by starting a base system, e.g. the phusion/baseimage mentioned above, installing all the software and data you like and exporting the container:

# find container ID:
docker ps
CONTAINER ID        IMAGE                      COMMAND                  CREATED       
c24c11050fc5        phusion/baseimage:0.9.22   "/sbin/my_init -- ..."   4 seconds ago 
# export the current state in a compressed archive:
docker export c24c11050fc5 | gzip > my_new_image.tar.gz
# import and run the image later or on a different computer:
docker import my_new_image.tar.gz my_container
docker run -it my_container /sbin/my_init -- bash

Singularity

Installation on Mac OSX requires the Vagrant and the VirtualBox system first. Instead of using the brew system often recommended, I found it better to install the dmg packages directly from the Vagrant site and the Oracle VirtualBox site. Good instructions for different systems are also given here.
After installation you can start the VirtualBox Manager and set up an Ubuntu image:

# get the image:
vagrant init ubuntu/trusty64
# start the virtual machine:
vagrant up
# get to the command line:
vagrant ssh

or directly start a Singularity image with:

vagrant init singularityware/singularity-2.4
vagrant up
vagrant ssh

You can stop a VM running in the background with

vagrant halt

It is possible to use Docker images in Singularity, you can pull from the Docker hub and build a local image:

sudo singularity build phusion-singularity-image.simg docker://phusion/baseimage:0.9.22

This should get you started with containers.
Make sure to keep an eye on disk consumption, in particular Docker data seems to grow significantly in the background (See issue here)! I prefer to move Docker (the “qcow2” file) to a fast external disk.