5. Developer guide

This developer guide is meant for software developers who would like to understand REANA source code and contribute to it.

5.1. Local development workflow

REANA cluster is composed of several micro-services with multiple independent source code repositories. reana is a helper development script that facilitates working with multiple repositories for local development and integration testing purposes.

5.1.1. reana

Run REANA development and integration commands.

How to configure your environment:

$ export REANA_SRCDIR=~/project/reana/src
$ export REANA_GITHUB_USER=tiborsimko

How to prepare your environment:

$ # prepare directoru that will hold sources
$ # install reana developer helper script
$ mkvirtualenv reana
$ pip install git+git://github.com/reanahub/reana.git#egg=reana
$ # run ssh-agent locally to simplify GitHub interaction
$ eval "$(ssh-agent -s)"
$ ssh-add ~/.ssh/id_rsa

How to fork and clone all REANA repositories:

$ reana git-fork -c ALL
$ eval "$(reana git-fork -c ALL)"
$ reana git-clone -c ALL

How to install latest master REANA cluster and client CLI scripts:

$ workon reana
$ reana install-client
$ reana install-cluster

How to compile and deploy latest master REANA cluster:

$ minikube start --kubernetes-version="v1.11.2" --vm-driver=kvm2
$ eval $(minikube docker-env)
$ reana docker-build
$ reana docker-images
$ pip install reana-cluster
$ reana-cluster -f reana-cluster-latest.yaml init

How to set up your shell environment variables:

$ eval $(reana setup-environment)

How to run full REANA example using a given workflow engine:

$ reana run-example -c reana-demo-root6-roofit -w serial -s 10

How to test one component pull request:

$ cd reana-job-controller
$ reana git-checkout -b . 72 --fetch
$ reana docker-build -c .
$ kubectl delete pod -l app=job-controller

How to test multiple component branches:

$ reana git-checkout -b reana-job-controller 72
$ reana git-checkout -b reana-workflow-controller 98
$ reana git-status
$ reana docker-build
$ kubectl delete pod -l app=job-controller
$ kubectl delete pod -l app=workflow-controller

How to release and push cluster component images:

$ reana git-clean
$ reana docker-build --no-cache
$ # we should now run one more test with non-cached ``latest``
$ # once it works, we can tag and push
$ reana docker-build -t 0.3.0.dev20180625
$ reana docker-push -t 0.3.0.dev20180625
$ # we should now make PR for ``reana-cluster.yaml`` to use given tag



Build REANA component images.


List REANA component images.


Pull REANA component images from DockerHub.


Push REANA component images to DockerHub.


Remove REANA component images.


Check out local branch corresponding to a…


Clean REANA source repository code tree.


Clone REANA source repositories from GitHub.


Diff checked-out REANA local source code…


Fetch REANA upstream source code repositories…


Display commands to fork REANA source code…


Push REANA local repositories to GitHub…


Report status of REANA source repositories.


Upgrade REANA local source code repositories.


Display usage help tips and tricks.


Install latest REANA client Python package…


Install latest REANA cluster Python package…


Run given REANA example with given workflow…


Display commands to set up shell environment…


Return REANA version.

You can use --help option to see the detailed help for each command.

5.2. Debugging

In order to debug a REANA component, you first have to install REANA cluster in the development mode (see reana-cluster documentation). Once you have done this, you have to build the image of the component you are working on in development mode and we restart the corresponding pod:

$ cd src/reana-server
$ # if we are not connected to the minikube docker daemon we should do it
$ eval "$(minikube docker-env)"
$ docker build . --build-arg DEBUG=true -t reanahub/reana-server:latest
$ kubectl delete pod --selector=app="server"

Let us now introduce wdb breakpoint as the first instruction of the first instruction of the get_analyses() function located in reana_server/rest/analyses.py:


We can check that the code has been in fact updated and make a request to the component:

$ kubectl logs --selector=app="server"

 * Running on (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 310-304-952 - - [15/Feb/2018 12:43:49] "GET /api/ping HTTP/1.1" 200 -
 * Detected change in '/code/reana_server/rest/analyses.py', reloading
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 310-304-952
$ curl ""

After doing that we can go to the wdb dashboard (you can get wdb address using reana-cluster get wdb).


And finally select the debugging session.



It is not possible to get live code updates in workflow engine components since celery option –autoreload doesn’t work and it is deprecated. To debug celery right now:

  • Set breakpoint: import wdb; wdb.set_trace()
  • Kill the workflow engine container: kubectl delete pod cwl-default-worker-2461563162-r4hgg