In this part of the story I’ll share what technology I found useful for doing my work for different projects as a consulting bioinformatics scientist. This is the current state as of the end of 2018. It might change, but it might be useful for people in similar situations.
Computer set-up:
I do most of the work remotely, i.e. from my office at home with some visits to clients where possible. There I’m using:
An Apple MacBook Pro running the latest OSX.
on a Griffin Elevator stand – When you’re sitting many hours you need to keep a good posture!
A number of external USB hard disks like this one with 2 TB – Don’t fill up your machine and make sure you do backups!
Either a TrackMan Marble (to avoid bending your wrist) or a Logitech M330 Silent Plus mouse – I don’t know why not everybody is using the silent mouse! The constant clicking and scrolling get too annoying!
An Apple wired aluminum keyboard (with numeric keypad)
The key is the Unix-based OS, I wouldn’t want to work without access to the powerful command-line tools, etc.
For small or visual Python projects the iPython Jupyter Notebooks are great
For larger Python (or other) projects I like PyCharm CE
To do any more data- or processing-intensive tasks I use machines of suitable size in the Amazon cloud. I keep an image (AMI) there which has the software installed that I usually need, so starting work there is quick and much cheaper than buying your own server. These cloud machines are also better secured than most on-site servers!
I also share data and results with my clients through S3 on the Amazon cloud. Alternatively I set up a Nextcloud storage on my web-hosting server.
For most code-reading and -writing as well as for note-taking I love the TextMate editor.
Part B: Project management, marketing, etc.
I maintain a WordPress-based website hosted at all-inkl, with some companion pages (1, 2) to drive traffic.
The profiles at LinkedIn and XING are of key importance in order to be found when people search for your bioinformatics service.
Tracking the time I spend on different projects is done with a slightly customized version of Anuko that I installed on my server.
Expenses and other money-related tracking for net income determination (Einnahmenüberschussrechnung) is done with MS Excel or LibreOffice first. It is then entered in the (cloud-based) tax software LexOffice for the regular VAT submissions (Umsatzsteuer-Zahlungen an das Finanzamt via ELSTER). This software is not perfect, but you can do a 30-day test (or a 1-year test if there are promotions) to try if it is for you.
As you can see many of the tools are open-source or at least free software solutions.
So you have some free time and energy to use and extend your bioinformatics skills? One main worry of a potential (or existing) freelancer is: How do I find work! How will companies know that I am here and I am available for hire?
In my opinion, there is enough work for all of us. The field of bioinformatics can be defined quite broadly and it is still expanding. The real issue seems to be that some companies are not used to working with external freelancers / consultants and are therefor not aware of this resource! For us to spread the word and find interesting projects my suggestions would be:
A – Your network: If you have worked in the field before or know people from uni, make use of these contacts! This is by far the best way to get started as you will be in a field you already know and potentially with people who like and value you already.
Let them know that you would be happy to work with them and what areas you are most skilled or interested in. Make sure they don’t feel pressured though.
B – Internet resources: Besides the direct advertising through your own web site(s), make sure you make full use of online social networks like LinkedIn and Xing. They have channeled many interested parties to me (along with random requests of course)!
In addition a whole number of freelancer platforms have been created in recent years that focus on bringing companies and freelancer together. You can also leave your CV with recruitment agencies like Hayes. Just remember that any of these platforms or agencies need to pay their employees as well – so they will take a (sometimes significant) cut of your earnings.
Make sure you add keywords to all sites and profiles that describe the kind of knowledge you have or the kind of work you are looking for!
Here is a list of potential sources of project work for you as a bioinformatics freelancer:
Barcodes are not just for supermarkets! Any medium to high-throughput laboratory can benefit greatly from employing barcodes. The process of identifying and tracking samples and all types of other things (please don’t barcode your technician!) through locations and processes can:
Save time typing the entries manually
Greatly reduce the risks of mis-identification
Allow longer IDs (i.e. more samples) or more other data without more effort
Traditional barcodes exist as one-dimensional and two-dimensional black-and-white images (some examples in fig. 1). Both types are read using an optical sensor at a set distance. This could be a dedicated barcode-scanner or the camera of your smartphone. One-Dimensional barcodes typically track just one piece of data, such as the sample number. Two-Dimensional barcodes are able to encode more data, including sample ID, customer ID, lot numbers, and more.
There are around 100 different encoding schemes for barcodes. The original standard are “EAN/UPC” linear barcodes, but “Code-128” is becoming more established as it allows letters as well as digits (all 128 ASCII code characters), includes a start and stop identification. The latter as well as a “quite zone” (white space) around the barcode and a checksum digit at the end of the code make the reading more reliable, reducing the risk of mis-identification.
An online resource to see all the different types of barcodes and try encoding information with them can be found here, a specification document is also provided. A whole lecture about the science of barcodes is provided here. Two examples of encoding “Gene-Test” can be seen in fig. 2 & 3:
2-D Codablock-F encoding
Linear EAN128 barcode
In the EAN-128 scheme there are four types of bar sizes, one character is represented by 3 bars and 3 spaces (total six elements). Using the table here we can decode this barcode:
One of the amazingly useful current trends in software development is “containerization“. This describes setting up a self-contained environment on a host computer system, that can run a separate operating system (OS), contain data and software not usually available otherwise. I find this very appealing e.g. for
testing a software package that is not available for my usual OS
testing my own software on a different OS
developing or running an analysis in fully reproducible settings
sharing software or analysis with reproducible settings
The main players in the field are Docker and Singularity. There are pros and cons for each, Singularity might be better suited for shared environments as you can run the containers with standard user rights.
The general idea of containerization is similar to virtual machines, this Docker page explains the differences.
Docker
To install Docker on an Ubuntu system, currently the following commands will do the trick:
Installation of Docker on Mac OSX is straightforward with the download of the free community edition (CE) from here. Make sure you either add the software as a log-in item or manually start it before you attempt to use it. (You will see the little Docker whale in the task bar. )
Some key Docker commands are
# show all images available locally:
docker images
# show all containers that are already running:
docker ps
# start a new container from an image that can be fetched from the remote Docker repository.
# a simple test:
docker run hello-world
# or a small Linux system:
docker run --rm -it phusion/baseimage:0.9.22 /sbin/my_init -- bash
A good way to move data in and out of the container is by mounting a specific directory, e.g./home/felix/testproject to the tmp folder:
docker run --rm -it --mount type=bind,source=/home/felix/testproject,target=/tmp phusion/baseimage:0.9.22 /sbin/my_init -- bash
The standard way to create new images is by defining all installation steps in a Dockerfile. However, to share a pre-build environment with your own data it is sometimes easier to freeze the container you have build up. Your can do this by starting a base system, e.g. the phusion/baseimage mentioned above, installing all the software and data you like and exporting the container:
# find container ID:
docker ps
CONTAINER ID IMAGE COMMAND CREATED
c24c11050fc5 phusion/baseimage:0.9.22 "/sbin/my_init -- ..." 4 seconds ago
# export the current state in a compressed archive:
docker export c24c11050fc5 | gzip > my_new_image.tar.gz
# import and run the image later or on a different computer:
docker import my_new_image.tar.gz my_container
docker run -it my_container /sbin/my_init -- bash
Singularity
Installation on Mac OSX requires the Vagrant and the VirtualBox system first. Instead of using the brew system often recommended, I found it better to install the dmg packages directly from the Vagrant site and the Oracle VirtualBox site. Good instructions for different systems are also given here.
After installation you can start the VirtualBox Manager and set up an Ubuntu image:
# get the image:
vagrant init ubuntu/trusty64
# start the virtual machine:
vagrant up
# get to the command line:
vagrant ssh
or directly start a Singularity image with:
vagrant init singularityware/singularity-2.4
vagrant up
vagrant ssh
You can stop a VM running in the background with
vagrant halt
It is possible to use Docker images in Singularity, you can pull from the Docker hub and build a local image:
This should get you started with containers.
Make sure to keep an eye on disk consumption, in particular Docker data seems to grow significantly in the background (See issue here)! I prefer to move Docker (the “qcow2” file) to a fast external disk.
It’s good practise to perform all kinds of tests on files that you are processing in a Bash script, in particular if it is a file that the user has provided. Here is a reminder of the test flags you can use as part of the test command:
Switches to check:
-b filename – Block special file -c filename – Special character file -d directoryname – Check for directory Existence -e filename – Check for file existence, regardless of type (node, directory, socket, etc.) -f filename – Check for regular file existence not a directory -G filename – Check if file exists and is owned by effective group ID -G filename set-group-id – True if file exists and is set-group-id -k filename – Sticky bit -L filename – Symbolic link -O filename – True if file exists and is owned by the effective user id -r filename – Check if file is a readable -S filename – Check if file is socket -s filename – Check if file is nonzero size -u filename – Check if file set-user-id bit is set -w filename – Check if file is writable -x filename – Check if file is executable
How to use:
#!/bin/bash
file=./file
if[[-e "${file}" ]] ;then
echo "File exists"else
echo "File does not exist"fi
A test expression can be negated by using the ! operator
#!/bin/bash
file=./file
if[[!-e "${file}" ]] ;then
echo "File does not exist"else
echo "File exists"fi
Source: GroundZero @ StackOverflow
Note: To be risk-adverse I like to use double brackets [[ ]] around the tests and brackets {} around the variable names.
Additionally it might be useful here to learn additional comparison operators:
[ ( EXPR ) ]
Returns the value of EXPR. This may be used to override the normal precedence of operators.
[ EXPR1 -a EXPR2 ]
True if both EXPR1 and EXPR2 are true.
[ EXPR1 -o EXPR2 ]
True if either EXPR1 or EXPR2 is true.
[ -z STRING ]
True of the length if “STRING” is zero.
[ -n STRING ] or [ STRING ]
True if the length of “STRING” is non-zero.
[ STRING1 == STRING2 ]
True if the strings are equal. “=” may be used instead of “==” for strict POSIX compliance.
[ STRING1 != STRING2 ]
True if the strings are not equal.
[ ARG1 OP ARG2 ]
“OP” is one of -eq, -ne, -lt, -le, -gt or -ge. These arithmetic binary operators return true if “ARG1” is equal to, not equal to, less than, less than or equal to, greater than, or greater than or equal to “ARG2”, respectively. “ARG1” and “ARG2” are integers.
We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.OkRead more
So you have some free time and energy to use and extend your bioinformatics skills? One main worry of a potential (or existing) freelancer is:
Traditional barcodes exist as one-dimensional and two-dimensional black-and-white images (some examples in fig. 1). Both types are read using an optical sensor at a set distance. This could be a dedicated barcode-scanner or the camera of your smartphone. One-Dimensional barcodes typically track just one piece of data, such as the sample number. Two-Dimensional barcodes are able to encode more data, including sample ID, customer ID, lot numbers, and more.
One of the amazingly useful current trends in software development is “containerization“. This describes setting up a self-contained environment on a host computer system, that can run a separate operating system (OS), contain data and software not usually available otherwise. I find this very appealing e.g. for
)