Today’s task was to update a docker image by adding some data, committing the changes to the Docker-Hub and creating image files for Docker and for Singularity (version 2.4-dist).
I’m usually performing these tasks on an Ubuntu machine on the AWS cloud. Trying to follow my steps from a previous time, I first start my container and perform the update within:
sudo docker run -it --rm -m 4g --mount type=bind,source=/home/ubuntu,target=/tmp --name projectname-1 $DOCKER_ID_USER/projectname:2 /bin/bash # ...
Checking the container ID and committing and pushing the changes to Docker-Hub all seems fine:
sudo docker ps sudo docker commit 6a024ac35ab5 $DOCKER_ID_USER/projectname:2 sudo docker push $DOCKER_ID_USER/projectname:2
But trying to pull down the image using Docker or Singularity fails repeatedly:
sudo docker pull $DOCKER_ID_USER/projectname:2
invalid reference format
singularity pull docker://$DOCKER_ID_USER/projectname:2
Importing: base Singularity environment Importing: /home/ubuntu/.singularity/docker/sha256:223cbef2a1193a2c9ab9dac0195ff0dcbbe2067e724f46a5fbe8473dda842b71.tar.gz gzip: /home/ubuntu/.singularity/docker/sha256:223cbef2a1193a2c9ab9dac0195ff0dcbbe2067e724f46a5fbe8473dda842b71.tar.gz: not in gzip format tar: This does not look like a tar archive tar: Exiting with failure status due to previous errors
The name and format of my project have not changed however! After some head-scratching I found that I could actually perform the tasks of creating the Singularity images on my local machine (an Apple MacBook Pro running High Sierra), following this Singularity page:
cd /Users/fsk/singularity-vm/ vagrant destroy rm Vagrantfile vagrant up --provider virtualbox
Checking with singularity –version shows version 2.4-dist is running successfully!
Pulling with Singularity nicely creates my Singularity img file from the Docker image:
sudo singularity pull docker://$DOCKER_ID_USER/projectname:2
... Singularity container built: ./projectname-2.img ...
What are the alternatives?
- Using docker2singularity created a file twice the size and was therefore not useful!
- Re-building from a Docker file or from a Singularity recipe file would have been very tedious, but would probably work.
To produce the Docker images, I was able to export from the running container on the Ubuntu machine:
sudo docker export 6v034ac35ab5 | gzip > projectname-2.tar.gz
All is good…