Customize Consent Preferences

We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.

The cookies that are categorized as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... 

Always Active

Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

No cookies to display.

Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

No cookies to display.

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

No cookies to display.

Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.

No cookies to display.

Advertisement cookies are used to provide visitors with customized advertisements based on the pages you visited previously and to analyze the effectiveness of the ad campaigns.

No cookies to display.

Creating a Singularity image when Docker is misbehaving

Today’s task was to update a docker image by adding some data, committing the changes to the Docker-Hub and creating image files for Docker and for Singularity (version 2.4-dist).
I’m usually performing these tasks on an Ubuntu machine on the AWS cloud. Trying to follow my steps from a previous time, I first start my container and perform the update within:

sudo docker run -it --rm -m 4g --mount type=bind,source=/home/ubuntu,target=/tmp  --name projectname-1 $DOCKER_ID_USER/projectname:2 /bin/bash
# ...

Checking the container ID and committing and pushing the changes to Docker-Hub all seems fine:

sudo docker ps
sudo docker commit 6a024ac35ab5 $DOCKER_ID_USER/projectname:2
sudo docker push $DOCKER_ID_USER/projectname:2

But trying to pull down the image using Docker or Singularity fails repeatedly:

sudo docker pull $DOCKER_ID_USER/projectname:2
invalid reference format
singularity pull docker://$DOCKER_ID_USER/projectname:2
Importing: base Singularity environment
Importing: /home/ubuntu/.singularity/docker/sha256:223cbef2a1193a2c9ab9dac0195ff0dcbbe2067e724f46a5fbe8473dda842b71.tar.gz
gzip: /home/ubuntu/.singularity/docker/sha256:223cbef2a1193a2c9ab9dac0195ff0dcbbe2067e724f46a5fbe8473dda842b71.tar.gz: not in gzip format
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

The name and format of my project have not changed however! After some head-scratching I found that I could actually perform the tasks of creating the Singularity images on my local machine (an Apple MacBook Pro running High Sierra), following this Singularity page:

cd /Users/fsk/singularity-vm/
vagrant destroy
rm Vagrantfile
vagrant up --provider virtualbox

Checking with singularity –version shows version 2.4-dist is running successfully!
Pulling with Singularity nicely creates my Singularity img file from the Docker image:

sudo singularity pull docker://$DOCKER_ID_USER/projectname:2
...
Singularity container built: ./projectname-2.img
...

What are the alternatives?

  • Using docker2singularity created a file twice the size and was therefore not useful!
  • Re-building from a Docker file or from a Singularity recipe file would have been very tedious, but would probably work.

To produce the Docker images, I was able to export from the running container on the Ubuntu machine:

sudo docker export 6v034ac35ab5 | gzip > projectname-2.tar.gz

All is good…