3. User guide

3.1. Environment variables

3.1.1. REANA_SERVER_URL

You can set this environment variable in order to specify to which REANA cloud instance your client should connect. For example:

$ export REANA_SERVER_URL=https://reana.cern.ch

3.1.2. REANA_ACCESS_TOKEN

You should specify valid access token for the REANA cloud instance you would like to use. For example:

$ export REANA_ACCESS_TOKEN=XXXXXXX

The token should be provided to you by the REANA cluster administrators.

3.1.3. REANA_WORKON

You can set this environment variable in order to specify a concrete workflow you would like to work on. (As an alternative to specifying --workflow option in commands.) For example:

$ export REANA_WORKON=myfirstanalysis

will work on the latest run of your “myfirstanalysis” workflow.

Note that you can also specify a concrete run number:

$ export REANA_WORKON=myfirstanalysis.3

which will permit to work on the third run of the “myfirstanalysis” workflow, for example to check out past input and output files.

You can list all your workflow runs and their statuses by doing:

$ reana-client list

and set REANA_WORKON to the one you would like to work on.

3.2. Usage

3.2.1. Overview

Please see the Getting Started section for a basic reana-client use case scenario.

3.2.2. Uploading analysis assets

Uploading files or directories to the analysis workspace is simple as:

$ reana-client upload mydata.csv mycode.C mytmp
File mydata.csv was successfully uploaded.
File mycode.C was successfully uploaded.
File mytmp/myfiltereddata.csv was successfully uploaded.

If you want to upload all input files defined in the reana.yaml of the analysis, you can just run:

$ reana-client upload
File mydata.csv was successfully uploaded.
File mycode.C was successfully uploaded.

Directory structures are maintained, i.e. mytmp exists in the workspace.

Note that symbolic links are resolved at the moment of upload so that a hard copy of the link target is uploaded to the cloud storage workspace. The link is not maintained throughout the workflow execution.

3.2.3. Deleting analysis assets

The deletion of files contained in the analysis workspace is possible through the remove command:

$ reana-client rm mydata.csv
File mydata.csv was successfully deleted.
25356 bytes freed up.

If you want to delete more than one file at once it is possible to use globbing:

$ reana-client rm '**/*.csv'
File mydata.csv was successfully deleted.
File mytmp/myfiltereddata.csv was successfully deleted.
79736 bytes freed up.

3.2.4. Moving analysis assets

The movement of file(s) or folders within the analysis workspace is possible through the mv command:

$ reana-client mv data/mydata.csv mydata.csv
File mydata.csv was successfully deleted.

3.2.5. Overriding default input parameters

If you want to run a workflow with different input parameters than the ones in reana.yaml, you can do it by running reana-client start with flag -p and specifying parameters that you want to override.

Note that parameters passed with -p flag must exist in reana.yaml. Non-existing parameters will be skipped.

$ reana-client start -p myparam1=myval1 -p myparam2=myval2
workflow.1 has been started.

3.2.6. Downloading outputs

Downloading files from an analysis workspace:

$ reana-client download result.png
File plot.png downloaded to /myfirstanalysis.

In the same way you can download all outputs defined in the reana.yaml file of the analysis, by just running:

$ reana-client download

Note that downloading directories is not yet supported.

3.2.7. Running analysis

If you have fully working analysis with reana.yaml, you can run the workflow easily via the run wrapper command, which will create a new workflow, upload analysis inputs, and start the workflow run.

$ vim reana.yaml
$ reana-client run -n myanalysis
[INFO] Creating a workflow...
myanalysis.1
[INFO] Uploading files...
File code/helloworld.py was successfully uploaded.
File data/names.txt was successfully uploaded.
[INFO] Starting workflow...
myanalysis.1 has been started.
$ export REANA_WORKON=myanalysis
$ reana-client status
NAME         RUN_NUMBER   CREATED               STATUS    PROGRESS
myanalysis   1            2018-11-07T12:45:18   running   1/1
$ reana-client download results/plot.png

3.2.8. Opening interactive sessions

While your analysis workflows are running, you may want to open interactive session processes on the workspace, such as Jupyter notebooks, via the open command. This will allow to quickly explore the generated data while the analysis is in progress, or even run your analyses from within the notebook environment spawned on the remote containerised platform.

$ reana-client open -w myanalysis.1 jupyter
https://reana.cern.ch/7cd4d23e-48d1-4f7f-8a3c-3a6d256fb8bc?token=P-IkL_7w25IDHhes8I7DtICWLNQm2WAZ9gkoKC2vq10
It could take several minutes to start the interactive session.

Open the link returned by the command in order to access the interactive notebook session.

_images/interactive-session.png

REANA currently supports Jupyter notebooks. Note that you can pass any notebook image you are interested to run on the workspace, such as PySpark, or even your own image, by using the –image option.

3.2.9. Closing interactive sessions

Once you finished working on your interactive session notebook, you can close it via close command.

$ reana-client close -w myanalysis.1
Interactive session for workflow myanalysis.1 was successfully closed

3.2.10. Deleting workflows

You can mark a workflow as deleted with:

$ reana-client delete

Passing no argument will mark the workflow selected by your REANA_WORKON variable as deleted. To specify a different workflow than your currently selected one use the -w/–workflow flag and set the workflow name and run number. If e.g. workflow run number 123 of your analysis failed, you can delete it as follows:

$ reana-client delete --workflow=myanalysis.123

After simple deletion the workspace can be accessed to retrieve files uploaded there. If you are sure the workspace can also be deleted pass the –include-workspace flag.

$ reana-client delete --workflow=myanalysis.123 --include-workspace

To delete all runs of a given workflow, pass the –include-all-runs flag and run:

$ reana-client delete --workflow=myanalysis --include-all-runs

and to completely remove a workflow run and its workspace from REANA pass the –include-records flag:

$ reana-client delete --workflow=myanalysis.1 --include-records

3.2.11. Stopping workflows

You can stop a workflow with:

$ reana-client stop --force

The workflow assigned to REANA_WORKON variable will be stopped. To specify a different workflow than your currently selected one use the -w/–workflow flag and set the workflow name or UUID.

$ reana-client stop --force --workflow=otherworkflow.1

Note that currently only force stop is implemented.

3.3. Examples

You can get inspiration on how to structure your REANA-compatible research data analysis from several reana-demo-... examples provided on GitHub.

3.4. Commands

The full list of reana-client commands with their documented options is available in the CLI API documentation.