5. Developer guide

This developer guide is meant for software developers who would like to understand REANA source code and contribute to it.

5.1. Local development workflow

REANA cluster is composed of several micro-services with multiple independent source code repositories.

The main source code repository contains a Makefile which allows you to quickly clone all the necessary repositories and kick-start your REANA platform developments locally.

You can simply type make to see the available options and usage scenarios.


  This Makefile facilitates building and testing REANA on a local Minikube cluster.
  Useful for personal development and CI testing scenarios.

Available commands:

  build             Build REANA client and cluster components.
  ci                Perform full Continuous Integration build and test cycle. [main function]
  clone             Clone REANA source code repositories locally.
  deploy            Deploy/redeploy previously built REANA cluster.
  example           Run one or several demo examples.
  prefetch          Prefetch interesting Docker images. Useful to speed things later.
  setup             Prepare local host virtual environment and Minikube for REANA building and deployment.
  teardown          Destroy local host virtual environment and Minikube. All traces go.
  test              Run unit tests on the REANA package.

Configuration options:

  CLUSTER_CONFIG      Which cluster configuration to use for Minikube? [e.g. "dev" for live coding and debugging; default is production]
  DEMO                Which demo example to run? [e.g. "reana-demo-helloworld"; default is several]
  GITHUB_USER         Which GitHub user account to use for cloning for Minikube? [default=anonymous]
  MINIKUBE_CPUS       How many CPUs to allocate for Minikube? [default=2]
  MINIKUBE_DISKSIZE   How much disk size to allocate for Minikube? [default=40g]
  MINIKUBE_DRIVER     Which vm driver to use for Minikube? [default=virtualbox]
  MINIKUBE_MEMORY     How much memory to allocate for Minikube? [default=3072]
  MINIKUBE_PROFILE    Which Minikube profile to use? [default=minikube]
  MINIKUBE_KUBERNETES Which Kubernetes version to use with Minikube? [default=v1.16.3]
  TIMECHECK           Checking frequency in seconds when bringing cluster up and down? [default=5]
  TIMEOUT             Maximum timeout to wait when bringing cluster up and down? [default=300]
  VENV_NAME           Which Python virtual environment name to use? [default=reana]
  SERVER_URL          Setting a customized REANA Server hostname? [e.g. "https://myreanaserver.com"; default is Minikube IP]
  CLUSTER_FLAGS       Which flags need to be passed to reana-cluster command? [e.g. "--loglevel --ui"; no flags are passed by default]


  # Example 1: set up personal development environment
  $ GITHUB_USER=johndoe make setup clone prefetch

  # Example 2: build and deploy REANA in production mode
  $ make build deploy

  # Example 3: build and deploy REANA in development mode (with live code changes and debugging)
  $ minikube mount $(pwd)/..:/code
  $ CLUSTER_CONFIG=dev make build deploy

  # Example 4: build and deploy REANA with a custom hostname including REANA-UI
  $ CLUSTER_FLAGS=--ui SERVER_URL=https://reana-local.cern.ch make build deploy

  # Example 5: run one small demo example to verify the build
  $ DEMO=reana-demo-helloworld make example

  # Example 6: run several small examples to verify the build
  $ make example

  # Example 7: perform full CI build-and-test cycle
  $ make ci

  # Example 8: perform full CI build-and-test cycle in an independent cluster
  $ mkdir /tmp/nightlybuild && cd /tmp/nightlybuild
  $ git clone https://github.com/reanahub/reana && cd reana
  $ VENV_NAME=nightlybuild MINIKUBE_PROFILE=nightlybuild make ci
  $ VENV_NAME=nightlybuild MINIKUBE_PROFILE=nightlybuild make teardown
  $ cd /tmp && rm -rf /tmp/nightlybuild

In addition, REANA comes with a reana-dev helper development script that simplifies working with multiple repositories during local development and integration testing. You can use --help option to see the detailed usage instructions.

5.1.1. reana-dev

Run REANA development and integration commands.

How to prepare your environment:

$ # prepare directory that will hold sources
$ mkdir -p ~/project/reana/src
$ cd ~/project/reana/src
$ # create new virtual environment
$ virtualenv ~/.virtualenvs/myreana
$ source ~/.virtualenvs/myreana/bin/activate
$ # install reana-dev developer helper script
$ pip install git+git://github.com/reanahub/reana.git#egg=reana
$ # run ssh-agent locally to simplify GitHub interaction
$ eval "$(ssh-agent -s)"
$ ssh-add ~/.ssh/id_rsa

How to fork and clone all REANA repositories:

$ reana-dev git-fork -c ALL
$ eval "$(reana-dev git-fork -c ALL)"
$ reana-dev git-clone -c ALL -u tiborsimko

How to install latest master REANA cluster and client CLI scripts:

$ reana-dev install-client
$ reana-dev install-cluster

How to compile and deploy latest master REANA cluster:

$ # install minikube and set docker environment
$ minikube start --vm-driver=virtualbox \
$ eval $(minikube docker-env)
$ # option (a): cluster in production-like mode
$ reana-dev docker-build
$ reana-cluster -f reana-cluster-minikube.yaml init
$ # option (b): cluster in developer-like debug-friendly mode
$ reana-dev docker-build -b DEBUG=1
$ reana-cluster -f reana-cluster-minikube-dev.yaml init

How to set up your shell environment variables:

$ eval $(reana-dev setup-environment)

How to run full REANA example using a given workflow engine:

$ reana-dev run-example -c reana-demo-root6-roofit -w serial

How to test one component pull request:

$ cd reana-workflow-controller
$ reana-dev git-checkout -b . 72 --fetch
$ reana-dev docker-build -c .
$ reana-dev kubectl-delete-pod -c .

How to test multiple component branches:

$ reana-dev git-checkout -b reana-server 72
$ reana-dev git-checkout -b reana-workflow-controller 98
$ reana-dev git-status
$ reana-dev docker-build
$ reana-dev kubectl-delete-pod -c reana-server
$ reana-dev kubectl-delete-pod -c reana-workflow-controller

How to test multiple component branches with commits to shared modules:

$ reana-dev git-checkout -b reana-commons 72
$ reana-dev git-checkout -b reana-db 73
$ reana-dev git-checkout -b reana-workflow-controller 98
$ reana-dev git-checkout -b reana-server 112
$ reana-dev git-submodule --update
$ reana-dev install-client
$ reana-dev install-cluster
$ reana-dev docker-build
$ reana-cluster -f reana-cluster-minikube.yaml down
$ minikube ssh 'sudo rm -rf /var/reana'
$ reana-cluster -f reana-cluster-minikube.yaml init
$ eval $(reana-dev setup-environment)
$ reana-dev run-example -c r-d-helloworld
$ reana-dev git-submodule --delete
$ reana-dev git-status -s

How to release and push cluster component images:

$ reana-dev git-clean
$ reana-dev docker-build --no-cache
$ # we should now run one more test with non-cached ``latest``
$ # once it works, we can tag and push
$ reana-dev docker-build -t 0.3.0.dev20180625
$ reana-dev docker-push -t 0.3.0.dev20180625
$ # we should now make PR for ``reana-cluster.yaml`` to use given tag
reana-dev [OPTIONS] COMMAND [ARGS]...



Build REANA component images.


List REANA component images.


Pull REANA component images from DockerHub.


Push REANA component images to DockerHub.


Remove REANA component images.


Display information about locally checked-out…


Check out local branch corresponding to a…


Clean REANA source repository code tree.


Clone REANA source repositories from GitHub.


Diff checked-out REANA local source code…


Fetch REANA upstream source code repositories…


Display commands to fork REANA source code…


Show commit logs in given component…


Push REANA local repositories to GitHub…


Report status of REANA source repositories.


Sync REANA shared modules across all the…


Upgrade REANA local source code repositories…


Display usage help tips and tricks.


Install latest REANA client Python package…


Install latest REANA cluster Python package…


Delete REANA component’s pod.


Run given REANA example with given workflow…


Display commands to set up shell environment…


Show version.

5.2. Debugging

In order to debug a REANA component, you first have to mount REANA’s source code into Minikube’s /code directory and then build and re-deploy in development mode:

$ minikube mount $(pwd)/..:/code
$ CLUSTER_CONFIG=dev make build deploy

Let us now introduce wdb breakpoint as the first instruction of the first instruction of the get_workflows() function located in reana_server/rest/workflows.py:


We can check that the code has been in fact updated and make a request to the component:

$ kubectl logs --selector=app="server" -c server

DB Created.
 * Serving Flask app "/code/reana_server/app.py" (lazy loading)
 * Environment: production
 WARNING: Do not use the development server in a production environment.
 Use a production WSGI server instead.
 * Debug mode: on
 * Running on (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 221-564-335

$ curl $REANA_SERVER_URL/api/workflows?access_token=$REANA_ACCESS_TOKEN

After doing that we can go to the wdb dashboard:

$ firefox http://`minikube ip`:31984

And finally select the debugging session.


5.3. Port forwarding

If you ever need to access one specific microservice via HTTP there is a Kubernetes command that can help. The port-forward command connects a local port on the machine to a port on a Kubernetes pod. It directs the traffic reaching the local port to the pod port through an HTTP connection. Example:

$ kubectl port-forward --address <pod_name> <local_port>:<pod_port>

The --address flag defines the local IP address to listen on. Using makes the connection listen to all local IP addresses.