Docker usage information


1. Introduction

The Docker image contains the latest version of Giotto.

1.1 Starting and restarting container
By this time, you have installed the giotto Docker image. You exited Docker and you would like to know how to start Docker container again. Open a terminal (Windows Terminal or Terminal app, or GNOME terminal), and enter the following command: 
$ docker start giotto-1.0.2 
$ docker attach giotto-1.0.2 
dean@fbf953ebb7e6:/$
1.2 The user dean
As you log into Docker, you are logged in as user dean. The directory that appears upon launching the docker container is /. This is not where you will work. You will want to go to /data, where your files are shared with the host system: 
dean@fbf953ebb7e6:/$ cd /data
The user dean has administrative power to install packages with sudo utility, so users may choose to install additional packages. The user has an administrative password. The operating system currently is Ubuntu 18.04.2 LTS.


2. Working in Docker

The Docker image comes installed with Rstudio. Rstudio is a web based interface which runs R.

2.1 Start Rstudio
Enter the Rstudio environment. If you have not started Rstudio already (such as upon starting Docker image), do the following (you may need a password): 
dean@fbf953ebb7e6:~$ sudo rstudio-server stop
[sudo] password for dean: 
rsession: no process found
dean@fbf953ebb7e6:~$ sudo rstudio-server start
dean@fbf953ebb7e6:~$
Then open a web browser and navigate to http://localhost:8787/. You may be prompted for a password. You should see the familiar Rstudio web interface.

2.2 Working directory
The working directory in docker is /data. This is the directory that we designate to share between the docker image and the host system (the -v parameter in docker run -it command). Setting the working directory to the shared point allows easy exchange of files with the host.

It may be a good idea to change the current directory to the working directory in R/RStudio: 
> setwd("/data")
The working directory is a good place to store your datasets and intermediary files to be processed by Giotto, and results.

2.2.1 Communicate files between Docker and host: Where to find results
We specified during docker image installation a directory to share between docker image and host (through -v parameter in docker run -it command).

Because /data is shared with the host, users can extract the results from Docker and see it in the host operating system. For example, here are some processed files from running Giotto in Docker: 
dean@fbf953ebb7e6:/data$ ls -ltr
total 343080
-rw-r--r--  1 dean dean     20635 Oct 15 11:53 cortex_svz_centroids_coord.txt
-rw-r--r--  1 dean dean  18341006 Oct 15 11:53 cortex_svz_expression.txt
-rw-r--r--  1 dean dean     23040 Oct 15 11:53 cortex_svz_centroids_annot.txt
-rw-r--r--  1 dean dean    464934 Oct 15 12:17 2_spatplot.png
-rw-r--r--  1 dean dean    132137 Oct 15 12:17 3_a_HVGplot.png
-rw-r--r--  1 dean dean    134423 Oct 15 12:17 3_b_screeplot.png
-rw-r--r--  1 dean dean    167046 Oct 15 12:17 3_c_PCA_reduction.png
-rw-r--r--  1 dean dean    195095 Oct 15 12:17 3_d_UMAP_reduction.png
-rw-r--r--  1 dean dean    187440 Oct 15 12:17 3_e_tSNE_reduction.png
-rw-r--r--  1 dean dean   1177983 Oct 15 12:17 4_a_UMAP_leiden.png
On my host system, I have it set up so that /data is linked to /home/qzhu/Downloads/workplace on my host system (a Ubuntu). You can see that all files in /data are reflected in the linked location: 
qzhu@qzhu-Latitude-E5570:~/Downloads/workplace$ ls -ltr
-rw-r--r--  1 qzhu qzhu     20635 Oct 15 11:53 cortex_svz_centroids_coord.txt
-rw-r--r--  1 qzhu qzhu  18341006 Oct 15 11:53 cortex_svz_expression.txt
-rw-r--r--  1 qzhu qzhu     23040 Oct 15 11:53 cortex_svz_centroids_annot.txt
-rw-r--r--  1 qzhu qzhu    464934 Oct 15 12:17 2_spatplot.png
-rw-r--r--  1 qzhu qzhu    132137 Oct 15 12:17 3_a_HVGplot.png
-rw-r--r--  1 qzhu qzhu    134423 Oct 15 12:17 3_b_screeplot.png
-rw-r--r--  1 qzhu qzhu    167046 Oct 15 12:17 3_c_PCA_reduction.png
-rw-r--r--  1 qzhu qzhu    195095 Oct 15 12:17 3_d_UMAP_reduction.png
-rw-r--r--  1 qzhu qzhu    187440 Oct 15 12:17 3_e_tSNE_reduction.png
-rw-r--r--  1 qzhu qzhu   1177983 Oct 15 12:17 4_a_UMAP_leiden.png
-rw-r--r--  1 qzhu qzhu    250511 Oct 15 12:18 HVGplot.png
Similarly, the host can put files to shared directory to share files (such as input datasets) to /data in docker.

2.2.2 Giotto Analyzer: running dataset examples
The dataset examples should support running Giotto in Docker. You should note the following especially as you begin to follow the dataset examples using a Docker giotto.
  • Set the working directory to /data early on: 
    > setwd("/data")
> my_working_dir = "/data"
    The variable my_working_dir is used frequently throughout the dataset examples.
  • Set the python path early on: 
    > python_path = "/usr/bin/python3"
    The variable python_path is used frequently throughout the dataset examples. There is only one python location and it is /usr/bin/python3.
2.2.3 Giotto Viewer
You should know: how to launch a shell terminal in docker, because viewer requires entering linux commands outside of R.

Normally, you first run exportGiottoViewer() function in Analyzer. This function exports the required inputs for viewer. Next, follow this viewer tutorial on how to see the viewer (follow Simple version (no image)). If there are extra information, such as staining images, follow Advanced version section of tutorial.

  • If you want to run Giotto Viewer independently (i.e. without exportGiottoViewer function in Analyzer), you should prepare input files yourself.

2.2.4 Where to find installed R packages (advanced)
They are typically located in /home/dean/R/x86_64-pc-linux-gnu-library. Some may be installed system-wide.

2.2.5 Where to find installed python packages (advanced)
They are typically located in /home/dean/.local/lib/python3.6/site-packages/. Some may be installed system-wide.

2.2.6 What is the password for dean user
dean

3. Packages

Along with Giotto, this docker image also installed the following other packages:
  • DESeq2
  • trendsceek
  • smfishHmrf
  • spatstat
  • SPARK
  • leiden
  • magick
  • Seurat
  • RCTD
  • SpatialDE

4. Updating Giotto in docker

We will update the image on a regular basis. However, if the user wants to use the latest version of Giotto right away, the user can update the Giotto version within docker: 
> library(devtools)  # if not installed: install.packages('devtools')
> library(remotes)  # if not installed: install.packages('remotes')
> remotes::install_github("RubD/Giotto", ref="master")
These commands will update Giotto in /home/dean/R/x86_64-pc-linux-gnu-library.

To update the Giotto Viewer in Docker, enter the following commands in a docker terminal: 
dean@fbf953ebb7e6:/$ sudo pip3 install --upgrade --force-reinstall --no-cache-dir --no-deps giotto-viewer smfish-image-processing
[sudo] password for dean: (enter dean)
  • If you see "Requirements satisfied...", this indicates that the new versions are not installed. Try running same command again.
  • You should see "Successfully uninstalled...", followed by "Successfully installed ..." for each smfish_image_processing and giotto-viewer packages.
  • The above will update giotto viewer in /usr/local/lib, /usr/local/bin directories.


4.1 Packaging docker to share with others
Make sure the docker container is exited. Then, outside of docker, enter the following commands: 
$ docker export jovial_keldysh > docker-giotto-1.0.2
$ 7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on docker-giotto-1.0.2.7z docker-giotto-1.0.2
Note that this will need quite a bit of space (5GB) in the process of exporting image. Once created, the file docker-giotto-1.0.2.7z can be shared with others.