5. Developer guide

This developer guide is meant for software developers who would like to understand REANA source code and contribute to it.

5.1. Local development workflow

REANA cluster is composed of several micro-services with multiple independent source code repositories.

The main source code repository contains a Makefile which allows you to quickly clone all the necessary repositories and kick-start your REANA platform developments locally.

You can simply type make to see the available options and usage scenarios.

Description:

  This Makefile facilitates building and testing REANA on a local Minikube cluster.
  Useful for personal development and CI testing scenarios.

Available commands:

  build             Build REANA client and cluster components.
  ci                Perform full Continuous Integration build and test cycle. [main function]
  clone             Clone REANA source code repositories locally.
  deploy            Deploy/redeploy previously built REANA cluster.
  example           Run one or several demo examples.
  prefetch          Prefetch interesting Docker images. Useful to speed things later.
  setup             Prepare local host virtual environment and Minikube for REANA building and deployment.
  teardown          Destroy local host virtual environment and Minikube. All traces go.
  test              Run unit tests on the REANA package.

Configuration options:

  CLUSTER_CONFIG    Which cluster configuration to use for Minikube? [e.g. "dev" for live coding and debugging; default is production]
  DEMO              Which demo example to run? [e.g. "reana-demo-helloworld"; default is several]
  GITHUB_USER       Which GitHub user account to use for cloning for Minikube? [default=anonymous]
  MINIKUBE_CPUS     How many CPUs to allocate for Minikube? [default=2]
  MINIKUBE_DISKSIZE How much disk size to allocate for Minikube? [default=40g]
  MINIKUBE_DRIVER   Which vm driver to use for Minikube? [default=kvm2]
  MINIKUBE_MEMORY   How much memory to allocate for Minikube? [default=3072]
  MINIKUBE_PROFILE  Which Minikube profile to use? [default=minikube]
  TIMECHECK         Checking frequency in seconds when bringing cluster up and down? [default=5]
  TIMEOUT           Maximum timeout to wait when bringing cluster up and down? [default=300]
  VENV_NAME         Which Python virtual environment name to use? [default=reana]

Examples:

  # Example 1: set up personal development environment
  $ GITHUB_USER=johndoe MINIKUBE_DRIVER=virtualbox make setup prefetch clone

  # Example 2: build and deploy REANA in production mode
  $ make build deploy

  # Example 3: build and deploy REANA in development mode (with live code changes and debugging)
  $ minikube mount $(pwd)/..:/code
  $ cd reana-server && pip install -e . && cd ..  # workaround for reana-workflow-controller#64
  $ CLUSTER_CONFIG=dev make build deploy

  # Example 4: run one small demo example to verify the build
  $ DEMO=reana-demo-helloworld make example

  # Example 5: run several small examples to verify the build
  $ make example

  # Example 6: perform full CI build-and-test cycle
  $ make ci

  # Example 7: perform full CI build-and-test cycle in an independent cluster
  $ mkdir /tmp/nightlybuild && cd /tmp/nightlybuild
  $ git clone https://github.com/reanahub/reana && cd reana
  $ VENV_NAME=nightlybuild MINIKUBE_PROFILE=nightlybuild make ci
  $ VENV_NAME=nightlybuild MINIKUBE_PROFILE=nightlybuild make teardown
  $ cd /tmp && rm -rf /tmp/nightlybuild

In addition, REANA comes with a reana-dev helper development script that simplifies working with multiple repositories during local development and integration testing. You can use --help option to see the detailed usage instructions.

5.1.1. reana-dev

Run REANA development and integration commands.

How to prepare your environment:


$ # prepare directory that will hold sources
$ mkdir -p ~/project/reana/src
$ cd ~/project/reana/src
$ # create new virtual environment
$ virtualenv ~/.virtualenvs/myreana
$ source ~/.virtualenvs/myreana/bin/activate
$ # install reana-dev developer helper script
$ pip install git+git://github.com/reanahub/reana.git#egg=reana
$ # run ssh-agent locally to simplify GitHub interaction
$ eval "$(ssh-agent -s)"
$ ssh-add ~/.ssh/id_rsa

How to fork and clone all REANA repositories:


$ reana-dev git-fork -c ALL
$ eval "$(reana-dev git-fork -c ALL)"
$ reana-dev git-clone -c ALL -u tiborsimko

How to install latest master REANA cluster and client CLI scripts:


$ reana-dev install-client
$ reana-dev install-cluster

How to compile and deploy latest master REANA cluster:


$ # install minikube and set docker environment
$ minikube start --vm-driver=kvm2 \
                 --feature-gates="TTLAfterFinished=true"
$ eval $(minikube docker-env)
# deploy helm inside the Cluster
$ helm init
$ # option (a): cluster in production-like mode
$ reana-dev docker-build
$ reana-cluster -f reana-cluster-minikube.yaml init \
                --traefik --generate-db-secrets
$ # option (b): cluster in developer-like debug-friendly mode
$ reana-dev docker-build -b DEBUG=1
$ reana-cluster -f reana-cluster-minikube-dev.yaml init \
                --traefik --generate-db-secrets

How to set up your shell environment variables:


$ eval $(reana-dev setup-environment)

How to run full REANA example using a given workflow engine:


$ reana-dev run-example -c reana-demo-root6-roofit -w serial

How to test one component pull request:


$ cd reana-workflow-controller
$ reana-dev git-checkout -b . 72 --fetch
$ reana-dev docker-build -c .
$ reana-dev kubectl-delete-pod -c .

How to test multiple component branches:


$ reana-dev git-checkout -b reana-server 72
$ reana-dev git-checkout -b reana-workflow-controller 98
$ reana-dev git-status
$ reana-dev docker-build
$ reana-dev kubectl-delete-pod -c reana-server
$ reana-dev kubectl-delete-pod -c reana-workflow-controller

How to test multiple component branches with commits to shared modules:


$ reana-dev git-checkout -b reana-commons 72
$ reana-dev git-checkout -b reana-db 73
$ reana-dev git-checkout -b reana-workflow-controller 98
$ reana-dev git-checkout -b reana-server 112
$ reana-dev git-submodule --update
$ reana-dev install-client
$ reana-dev install-cluster
$ reana-dev docker-build
$ reana-cluster -f reana-cluster-minikube.yaml down
$ minikube ssh 'sudo rm -rf /var/reana'
$ reana-cluster -f reana-cluster-minikube.yaml init \
                --traefik --generate-db-secrets
$ eval $(reana-dev setup-environment)
$ reana-dev run-example -c r-d-helloworld
$ reana-dev git-submodule --delete
$ reana-dev git-status -s

How to release and push cluster component images:


$ reana-dev git-clean
$ reana-dev docker-build --no-cache
$ # we should now run one more test with non-cached ``latest``
$ # once it works, we can tag and push
$ reana-dev docker-build -t 0.3.0.dev20180625
$ reana-dev docker-push -t 0.3.0.dev20180625
$ # we should now make PR for ``reana-cluster.yaml`` to use given tag
reana-dev [OPTIONS] COMMAND [ARGS]...

Commands

docker-build

Build REANA component images.

docker-images

List REANA component images.

docker-pull

Pull REANA component images from DockerHub.

docker-push

Push REANA component images to DockerHub.

docker-rmi

Remove REANA component images.

git-branch

Display information about locally checked-out…

git-checkout

Check out local branch corresponding to a…

git-clean

Clean REANA source repository code tree.

git-clone

Clone REANA source repositories from GitHub.

git-diff

Diff checked-out REANA local source code…

git-fetch

Fetch REANA upstream source code repositories…

git-fork

Display commands to fork REANA source code…

git-log

Show commit logs in given component…

git-push

Push REANA local repositories to GitHub…

git-status

Report status of REANA source repositories.

git-submodule

Sync REANA shared modules across all the…

git-upgrade

Upgrade REANA local source code repositories…

help

Display usage help tips and tricks.

install-client

Install latest REANA client Python package…

install-cluster

Install latest REANA cluster Python package…

kubectl-delete-pod

Delete REANA component’s pod.

run-example

Run given REANA example with given workflow…

setup-environment

Display commands to set up shell environment…

version

Show version.

5.2. Debugging

In order to debug a REANA component, you first have to install REANA cluster in the development mode (see reana-cluster documentation). Once you have done this, you have to build the image of the component you are working on in development mode and we restart the corresponding pod:

$ cd src/reana-server
$ reana-dev docker-build -t latest -c . -b DEBUG=1
$ reana-dev kubectl-delete-pod -c .

Let us now introduce wdb breakpoint as the first instruction of the first instruction of the get_workflows() function located in reana_server/rest/workflows.py:

_images/setting-the-breakpoint.png

We can check that the code has been in fact updated and make a request to the component:

$ kubectl logs --selector=app="server"

DB Created.
 * Serving Flask app "/code/reana_server/app.py" (lazy loading)
 * Environment: production
 WARNING: Do not use the development server in a production environment.
 Use a production WSGI server instead.
 * Debug mode: on
 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 221-564-335
$ curl $REANA_SERVER_URL/api/workflows?access_token=$REANA_ACCESS_TOKEN

After doing that we can go to the wdb dashboard:

$ firefox http://`minikube ip`:31984
_images/wdb-active-sessions.png

And finally select the debugging session.

_images/wdb-debugging-ui.png

Limitations

It is not possible to get live code updates in workflow engine components since celery option –autoreload doesn’t work and it is deprecated. To debug celery right now:

  • Set breakpoint: import wdb; wdb.set_trace()
  • Kill the workflow engine container: kubectl delete pod cwl-default-worker-2461563162-r4hgg

5.3. Port forwarding

If you ever need to access one specific microservice via HTTP there is a Kubernetes command that can help. The port-forward command connects a local port on the machine to a port on a Kubernetes pod. It directs the traffic reaching the local port to the pod port through an HTTP connection. Example:

$ kubectl port-forward --address 0.0.0.0 <pod_name> <local_port>:<pod_port>

The --address flag defines the local IP address to listen on. Using 0.0.0.0 makes the connection listen to all local IP addresses.