5. Developer guide

This developer guide is meant for software developers who would like to understand REANA source code and contribute to it.

5.1. Local development workflow

REANA cluster is composed of several micro-services with multiple independent source code repositories.

The main source code repository contains a Makefile which allows you to quickly clone all the necessary repositories and kick-start your REANA platform developments locally.

You can simply type make to see the available options and usage scenarios.


  This Makefile facilitates building and testing REANA on a local Minikube cluster.
  Useful for personal development and CI testing scenarios.

Available commands:

  build             Build REANA client and cluster components.
  ci                Perform full Continuous Integration build and test cycle. [main function]
  clone             Clone REANA source code repositories locally.
  deploy            Deploy/redeploy previously built REANA cluster.
  example           Run all or one particular demo example. By default all REANA examples are executed.
  prefetch          Prefetch interesting Docker images. Useful to speed things later.
  setup             Prepare local host virtual environment and Minikube for REANA building and deployment.
  teardown          Destroy local host virtual environment and Minikube. All traces go.
  test              Run unit tests on the REANA package.

Configuration options:

  GITHUB_USER       Which GitHub user account to use for cloning? [default=anonymous]
  MINIKUBE_DRIVER   Which Minikube driver to use? [default=kvm2]
  MINIKUBE_PROFILE  Which Minikube profile to use? [default=minikube]
  TIMECHECK         Checking frequency in seconds when bringing cluster up and down? [default=5]
  TIMEOUT           Maximum timeout to wait when bringing cluster up and down? [default=300]
  VENV_NAME         Which Python virtual environment name to use? [default=reana]
  DEMO              Which demo example to run? [e.g. reana-demo-helloworld; default=several runable examples]
  CLUSTER_CONFIG    REANA cluster environment mode. Use "dev" for live coding and debugging.


  # how to set up personal development environment:
  $ GITHUB_USER=johndoe MINIKUBE_DRIVER=virtualbox make setup clone

  # how to build and deploy REANA in production mode:
  make build
  make deploy

  # how to deploy REANA in development mode, with application
  # autoreload on code changes and debugging capabilities:
  $ minikube mount $(pwd)/..:/code
  $ cd reana-server && pip install -e . && cd ..  # workaround necessary for reanahub/reana-workflow-controller#64
  $ CLUSTER_CONFIG=dev make build
  $ CLUSTER_CONFIG=dev make deploy

  # how to build latest checked-out sources and run one small demo example:
  $ DEMO=reana-demo-helloworld make ci

  # how to build latest checked-out sources and run several runable demo examples:
  $ make ci

  # how to perform an independent automated CI test run:
  $ mkdir /tmp/nightlybuild && cd /tmp/nightlybuild
  $ git clone https://github.com/reanahub/reana && cd reana
  $ VENV_NAME=nightlybuild MINIKUBE_PROFILE=nightlybuild make ci
  $ VENV_NAME=nightlybuild MINIKUBE_PROFILE=nightlybuild make teardown
  $ cd /tmp && rm -rf /tmp/nightlybuild

In addition, REANA comes with a reana-dev helper development script that simplifies working with multiple repositories during local development and integration testing. You can use --help option to see the detailed usage instructions.

5.1.1. reana-dev

Run REANA development and integration commands.

How to prepare your environment:

$ # prepare directory that will hold sources
$ mkdir -p ~/project/reana/src
$ cd ~/project/reana/src
$ # create new virtual environment
$ virtualenv ~/.virtualenvs/myreana
$ source ~/.virtualenvs/myreana/bin/activate
$ # install reana-dev developer helper script
$ pip install git+git://github.com/reanahub/reana.git#egg=reana
$ # run ssh-agent locally to simplify GitHub interaction
$ eval "$(ssh-agent -s)"
$ ssh-add ~/.ssh/id_rsa

How to fork and clone all REANA repositories:

$ reana-dev git-fork -c ALL
$ eval "$(reana-dev git-fork -c ALL)"
$ reana-dev git-clone -c ALL -u tiborsimko

How to install latest master REANA cluster and client CLI scripts:

$ reana-dev install-client
$ reana-dev install-cluster

How to compile and deploy latest master REANA cluster:

$ # install minikube and set docker environment
$ minikube start --vm-driver=kvm2 \
$ eval $(minikube docker-env)
# deploy helm inside the Cluster
$ helm init
$ # option (a): cluster in production-like mode
$ reana-dev docker-build
$ reana-cluster -f reana-cluster-minikube.yaml init --traefik
$ # option (b): cluster in developer-like debug-friendly mode
$ reana-dev docker-build -b DEBUG=1
$ reana-cluster -f reana-cluster-dev.yaml init --traefik

How to set up your shell environment variables:

$ eval $(reana-dev setup-environment)

How to run full REANA example using a given workflow engine:

$ reana-dev run-example -c reana-demo-root6-roofit -w serial

How to test one component pull request:

$ cd reana-workflow-controller
$ reana-dev git-checkout -b . 72 --fetch
$ reana-dev docker-build -c .
$ reana-dev kubectl-delete-pod -c .

How to test multiple component branches:

$ reana-dev git-checkout -b reana-server 72
$ reana-dev git-checkout -b reana-workflow-controller 98
$ reana-dev git-status
$ reana-dev docker-build
$ reana-dev kubectl-delete-pod -c reana-server
$ reana-dev kubectl-delete-pod -c reana-workflow-controller

How to test multiple component branches with commits to shared modules:

$ reana-dev git-checkout -b reana-commons 72
$ reana-dev git-checkout -b reana-db 73
$ reana-dev git-checkout -b reana-workflow-controller 98
$ reana-dev git-checkout -b reana-server 112
$ reana-dev git-submodule --update
$ reana-dev install-client
$ reana-dev install-cluster
$ reana-dev docker-build
$ reana-cluster -f reana-cluster-minikube.yaml down
$ minikube ssh 'sudo rm -rf /var/reana'
$ reana-cluster -f reana-cluster-minikube.yaml init --traefik
$ eval $(reana-dev setup-environment)
$ reana-dev run-example -c r-d-helloworld
$ reana-dev git-submodule --delete
$ reana-dev git-status -s

How to release and push cluster component images:

$ reana-dev git-clean
$ reana-dev docker-build --no-cache
$ # we should now run one more test with non-cached ``latest``
$ # once it works, we can tag and push
$ reana-dev docker-build -t 0.3.0.dev20180625
$ reana-dev docker-push -t 0.3.0.dev20180625
$ # we should now make PR for ``reana-cluster.yaml`` to use given tag
reana-dev [OPTIONS] COMMAND [ARGS]...



Build REANA component images.


List REANA component images.


Pull REANA component images from DockerHub.


Push REANA component images to DockerHub.


Remove REANA component images.


Display information about locally checked-out…


Check out local branch corresponding to a…


Clean REANA source repository code tree.


Clone REANA source repositories from GitHub.


Diff checked-out REANA local source code…


Fetch REANA upstream source code repositories…


Display commands to fork REANA source code…


Show commit logs in given component…


Push REANA local repositories to GitHub…


Report status of REANA source repositories.


Sync REANA shared modules across all the…


Upgrade REANA local source code repositories…


Display usage help tips and tricks.


Install latest REANA client Python package…


Install latest REANA cluster Python package…


Delete REANA component’s pod.


Run given REANA example with given workflow…


Display commands to set up shell environment…


Return REANA version.

5.2. Debugging

In order to debug a REANA component, you first have to install REANA cluster in the development mode (see reana-cluster documentation). Once you have done this, you have to build the image of the component you are working on in development mode and we restart the corresponding pod:

$ cd src/reana-server
$ reana-dev docker-build -t latest -c . -b DEBUG=1
$ reana-dev kubectl-delete-pod -c .

Let us now introduce wdb breakpoint as the first instruction of the first instruction of the get_workflows() function located in reana_server/rest/workflows.py:


We can check that the code has been in fact updated and make a request to the component:

$ kubectl logs --selector=app="server"

DB Created.
 * Serving Flask app "/code/reana_server/app.py" (lazy loading)
 * Environment: production
 WARNING: Do not use the development server in a production environment.
 Use a production WSGI server instead.
 * Debug mode: on
 * Running on (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 221-564-335
$ curl $REANA_SERVER_URL/api/workflows?access_token=$REANA_ACCESS_TOKEN

After doing that we can go to the wdb dashboard:

$ firefox http://`minikube ip`:31984

And finally select the debugging session.



It is not possible to get live code updates in workflow engine components since celery option –autoreload doesn’t work and it is deprecated. To debug celery right now:

  • Set breakpoint: import wdb; wdb.set_trace()
  • Kill the workflow engine container: kubectl delete pod cwl-default-worker-2461563162-r4hgg