REANA-Client

REANA-Client is a component of the REANA reusable and reproducible research data analysis platform. It provides a command-line tool that allows researchers to submit, run, and manage their computational workflows.

• seed workspace with input code and data

• run computational workflows on remote compute clouds

• list submitted workflows and enquire about their statuses

• download results of finished workflows

Installation

$ # create new virtual environment
$ virtualenv ~/.virtualenvs/reana
$ source ~/.virtualenvs/reana/bin/activate
$ # install reana-client
$ pip install reana-client

Usage

The detailed information on how to install and use REANA can be found in docs.reana.io.

CLI API

Usage: reana-client [OPTIONS] COMMAND [ARGS]...

  REANA client for interacting with REANA server.

Options:
  -l, --loglevel [DEBUG|INFO|WARNING]
                                  Sets log level
  --help                          Show this message and exit.

Quota commands:
  quota-show  Show user quota.

Configuration commands:
  info     List cluster general information.
  ping     Check connection to REANA server.
  version  Show version.

Workflow management commands:
  create  Create a new workflow.
  delete  Delete a workflow.
  diff    Show diff between two workflows.
  list    List all workflows and sessions.

Workflow execution commands:
  logs      Get workflow logs.
  restart   Restart previously run workflow.
  run       Shortcut to create, upload, start a new workflow.
  start     Start previously created workflow.
  status    Get status of a workflow.
  stop      Stop a running workflow.
  validate  Validate workflow specification file.

Workspace interactive commands:
  close  Close an interactive session.
  open   Open an interactive session inside the workspace.

Workspace file management commands:
  download  Download workspace files.
  du        Get workspace disk usage.
  ls        List workspace files.
  mv        Move files within workspace.
  prune     Prune workspace files.
  rm        Delete files from workspace.
  upload    Upload files and directories to workspace.

Workspace file retention commands:
  retention-rules-list  List the retention rules for a workflow.

Secret management commands:
  secrets-add     Add secrets from literal string or from file.
  secrets-delete  Delete user secrets by name.
  secrets-list    List user secrets.

reana-client

REANA client for interacting with REANA server.

reana-client [OPTIONS] COMMAND [ARGS]...

Options

-l, --loglevel <loglevel>

Sets log level

Options:

DEBUG | INFO | WARNING

close

Close an interactive session.

The close command allows to shut down any interactive sessions that you may have running. You would typically use this command after you finished exploring data in the Jupyter notebook and after you have transferred any code created in your interactive session.

Examples:

$ reana-client close -w myanalysis.42

reana-client close [OPTIONS]

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-t, --access-token <access_token>

Access token of the current user.

create

Create a new workflow.

The create command allows to create a new workflow from reana.yaml specifications file. The file is expected to be located in the current working directory, or supplied via command-line -f option, see examples below.

Examples:

$ reana-client create

$ reana-client create -w myanalysis

$ reana-client create -w myanalysis -f myreana.yaml

reana-client create [OPTIONS]

Options

-f, --file <file>

REANA specification file describing the workflow to execute. [default=reana.yaml]

-n, -w, --name, --workflow <name>

Optional name of the workflow. [default is “workflow”]

--skip-validation

If set, specifications file is not validated before submitting it’s contents to REANA server.

-t, --access-token <access_token>

Access token of the current user.

delete

Delete a workflow.

The delete command removes workflow run(s) from the database. Note that the workspace and any open session attached to it will always be deleted, even when --include-workspace is not specified. Note also that you can remove all past runs of a workflow by specifying --include-all-runs flag.

Example:

$ reana-client delete -w myanalysis.42

$ reana-client delete -w myanalysis.42 –include-all-runs

reana-client delete [OPTIONS]

Options

--include-all-runs

Delete all runs of a given workflow.

--include-workspace

Delete workspace from REANA.

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-t, --access-token <access_token>

Access token of the current user.

diff

Show diff between two workflows.

The diff command allows to compare two workflows, the workflow_a and workflow_b, which must be provided as arguments. The output will show the difference in workflow run parameters, the generated files, the logs, etc.

Examples:

$ reana-client diff myanalysis.42 myotheranalysis.43

$ reana-client diff myanalysis.42 myotheranalysis.43 –brief

reana-client diff [OPTIONS] WORKFLOW_A WORKFLOW_B

Options

-q, --brief

If not set, differences in the contents of the files in the two workspaces are shown.

-u, -U, --unified <context_lines>

Sets number of context lines for workspace diff output.

-t, --access-token <access_token>

Access token of the current user.

Arguments

WORKFLOW_A

Required argument

WORKFLOW_B

Required argument

download

Download workspace files.

The download command allows to download workspace files and directories. By default, the files specified in the workflow specification as outputs are downloaded. You can also specify the individual files you would like to download, see examples below.

Examples:

$ reana-client download # download all output files

$ reana-client download mydata.tmp outputs/myplot.png

$ reana-client download -o - data.txt # write data.txt to stdout

reana-client download [OPTIONS] FILES

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-o, --output-directory <output_directory>

Path to the directory where files will be downloaded. If - is specified as path, the files will be written to the standard output.

-t, --access-token <access_token>

Access token of the current user.

Arguments

FILES

Optional argument(s)

du

Get workspace disk usage.

The du command allows to check the disk usage of given workspace.

Examples:

$ reana-client du -w myanalysis.42 -s

$ reana-client du -w myanalysis.42 -s –human-readable

$ reana-client du -w myanalysis.42 –filter name=data/

reana-client du [OPTIONS]

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-t, --access-token <access_token>

Access token of the current user.

-s, --summarize

Display total.

--filter <filters>

Filter results to show only files that match certain filtering criteria such as file name or size.Use –filter <columm_name>=<column_value> pairs. Available filters are name and size.

-h, --human-readable

Show disk size in human readable format.

info

List cluster general information.

The info command lists general information about the cluster.

Lists all the available workspaces. It also returns the default workspace defined by the admin.

Examples:

$ reana-client info

reana-client info [OPTIONS]

Options

--json

Get output in JSON format.

-t, --access-token <access_token>

Access token of the current user.

list

List all workflows and sessions.

The list command lists workflows and sessions. By default, the list of workflows is returned. If you would like to see the list of your open interactive sessions, you need to pass the --sessions command-line option.

Example:

$ reana-client list –all

$ reana-client list –sessions

$ reana-client list –verbose –bytes

reana-client list [OPTIONS]

Options

-w, --workflow <workflow>

List all runs of the given workflow.

-s, --sessions

List all open interactive sessions.

--format <_format>

Format output according to column titles or column values. Use <columm_name>=<column_value> format. E.g. display workflow with failed status and named test_workflow –format status=failed,name=test_workflow.

--json

Get output in JSON format.

--all

Show all workflows including deleted ones.

-v, --verbose

Print out extra information: workflow id, user id, disk usage, progress, duration.

-h, --human-readable

Show disk size in human readable format.

--sort <sort_column_name>

Sort the output by specified column

--filter <filters>

Filter workflow that contains certain filtering criteria. Use –filter <columm_name>=<column_value> pairs. Available filters are name and status.

--include-duration

Include the duration of the workflows in seconds. In case a workflow is in progress, its duration as of now will be shown.

--include-progress

Include progress information of the workflows.

--include-workspace-size

Include size information of the workspace.

--show-deleted-runs

Include deleted workflows in the output.

-t, --access-token <access_token>

Access token of the current user.

--page <page>

Results page number (to be used with –size).

--size <size>

Size of results per page (to be used with –page).

logs

Get workflow logs.

The logs command allows to retrieve logs of running workflow. Note that only finished steps of the workflow are returned, the logs of the currently processed step is not returned until it is finished.

Examples:

$ reana-client logs -w myanalysis.42 $ reana-client logs -w myanalysis.42 -s 1st_step

reana-client logs [OPTIONS]

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

--json

Get output in JSON format.

-t, --access-token <access_token>

Access token of the current user.

--filter <filters>

Filter job logs to include only those steps that match certain filtering criteria. Use –filter name=value pairs. Available filters are compute_backend, docker_img, status and step.

--page <page>

Results page number (to be used with –size).

--size <size>

Size of results per page (to be used with –page).

ls

List workspace files.

The ls command lists workspace files of a workflow specified by the environment variable REANA_WORKON or provided as a command-line flag --workflow or -w. The SOURCE argument is optional and specifies a pattern matching files and directories.

Examples:

$ reana-client ls –workflow myanalysis.42

$ reana-client ls –workflow myanalysis.42 –human-readable

$ reana-client ls –workflow myanalysis.42 ‘data/root’

$ reana-client ls –workflow myanalysis.42 –filter name=hello

reana-client ls [OPTIONS] SOURCE

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

--format <_format>

Format output according to column titles or column values. Use <column_name>=<column_value> format. For E.g. display FILES named data.txt –format name=data.txt.

--json

Get output in JSON format.

--url

Get URLs of output files.

--filter <filters>

Filter results to show only files that match certain filtering criteria such as file name, size or modification date.Use –filter <columm_name>=<column_value> pairs. Available filters are name, size and last-modified.

-h, --human-readable

Show disk size in human readable format.

-t, --access-token <access_token>

Access token of the current user.

--page <page>

Results page number (to be used with –size).

--size <size>

Size of results per page (to be used with –page).

Arguments

SOURCE

Optional argument

mv

Move files within workspace.

The mv command allows to move files within a workspace. Note that the workflow might fail if files are moved during its execution.

Examples:

$ reana-client mv data/input.txt input/input.txt

reana-client mv [OPTIONS] SOURCE TARGET

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-t, --access-token <access_token>

Access token of the current user.

Arguments

SOURCE

Required argument

TARGET

Required argument

open

Open an interactive session inside the workspace.

The open command allows to open interactive session processes on top of the workflow workspace, such as Jupyter notebooks. This is useful to quickly inspect and analyse the produced files while the workflow is stlil running.

Examples:

$ reana-client open -w myanalysis.42 jupyter

reana-client open [OPTIONS] interactive-session-type

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-i, --image <image>

Docker image which will be used to spawn the interactive session. Overrides the default image for the selected type.

-t, --access-token <access_token>

Access token of the current user.

Arguments

interactive-session-type

Optional argument

ping

Check connection to REANA server.

The ping command allows to test connection to REANA server.

Examples:

$ reana-client ping

reana-client ping [OPTIONS]

Options

-t, --access-token <access_token>

Access token of the current user.

prune

Prune workspace files.

The prune command deletes all the intermediate files of a given workflow that are not present in the input or output section of the workflow specification.

Examples:

$ reana-client prune -w myanalysis.42

$ reana-client prune -w myanalysis.42 –include-inputs

reana-client prune [OPTIONS]

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-t, --access-token <access_token>

Access token of the current user.

--include-inputs

Delete also the input files of the workflow. Note that this includes the workflow specification file.

--include-outputs

Delete also the output files of the workflow.

quota-show

Show user quota.

The quota-show command displays quota usage for the user.

Examples:

$ reana-client quota-show –resource disk –report limit

$ reana-client quota-show –resource disk –report usage

$ reana-client quota-show –resource disk

$ reana-client quota-show –resources

reana-client quota-show [OPTIONS]

Options

--resource <resource>

Specify quota resource. e.g. cpu, disk.

--resources

Print available resources

--report <report>

Specify quota report type. e.g. limit, usage.

Options:

limit | usage

-h, --human-readable

Show disk size in human readable format.

-t, --access-token <access_token>

Access token of the current user.

restart

Restart previously run workflow.

The restart command allows to restart a previous workflow on the same workspace.

Note that workflow restarting can be used in a combination with operational options FROM and TARGET. You can also pass a modified workflow specification with -f or --file flag.

You can furthermore use modified input prameters using -p or --parameters flag and by setting additional operational options using -o or --options. The input parameters and operational options can be repetitive.

Examples:

$ reana-client restart -w myanalysis.42 -p sleeptime=10 -p myparam=4

$ reana-client restart -w myanalysis.42 -p myparam=myvalue

$ reana-client restart -w myanalysis.42 -o TARGET=gendata

$ reana-client restart -w myanalysis.42 -o FROM=fitdata

reana-client restart [OPTIONS]

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-t, --access-token <access_token>

Access token of the current user.

-p, --parameter <parameters>

Additional input parameters to override original ones from reana.yaml. E.g. -p myparam1=myval1 -p myparam2=myval2.

-o, --option <options>

Additional operational options for the workflow execution. E.g. CACHE=off. (workflow engine - serial) E.g. –debug (workflow engine - cwl)

-f, --file <file>

REANA specification file describing the workflow to execute. [default=reana.yaml]

retention-rules-list

List the retention rules for a workflow.

Example:

$ reana-client retention-rules-list -w myanalysis.42

reana-client retention-rules-list [OPTIONS]

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-t, --access-token <access_token>

Access token of the current user.

--format <_format>

Format output according to column titles or column values. Use <columm_name>=<column_value> format. E.g. display pattern and status of active retention rules –format workspace_files,status=active.

--json

Get output in JSON format.

rm

Delete files from workspace.

The rm command allow to delete files and directories from workspace. Note that you can use glob to remove similar files.

Examples:

$ reana-client rm -w myanalysis.42 data/mydata.csv

$ reana-client rm -w myanalysis.42 ‘data/root’

reana-client rm [OPTIONS] SOURCES

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-t, --access-token <access_token>

Access token of the current user.

Arguments

SOURCES

Optional argument(s)

run

Shortcut to create, upload, start a new workflow.

The run command allows to create a new workflow, upload its input files and start it in one command.

Examples:

$ reana-client run -w myanalysis-test-small -p myparam=mysmallvalue

$ reana-client run -w myanalysis-test-big -p myparam=mybigvalue

reana-client run [OPTIONS]

Options

-f, --file <file>

REANA specification file describing the workflow to execute. [default=reana.yaml]

-n, -w, --name, --workflow <name>

Optional name of the workflow. [default is “workflow”]

--skip-validation

If set, specifications file is not validated before submitting it’s contents to REANA server.

-p, --parameter <parameters>

Additional input parameters to override original ones from reana.yaml. E.g. -p myparam1=myval1 -p myparam2=myval2.

-o, --option <options>

Additional operational options for the workflow execution. E.g. CACHE=off.

--follow

If set, follows the execution of the workflow until termination.

-t, --access-token <access_token>

Access token of the current user.

secrets-add

Add secrets from literal string or from file.

Examples:

$ reana-client secrets-add –env RUCIO_USERNAME=ruciouser

$ reana-client secrets-add –file userkey.pem

$ reana-client secrets-add –env VOMSPROXY_FILE=x509up_u1000

–file /tmp/x509up_u1000

reana-client secrets-add [OPTIONS]

Options

--env <env>

Secrets to be uploaded from literal string. E.g. RUCIO_USERNAME=ruciouser

--file <file>

Secrets to be uploaded from file.

--overwrite

Overwrite the secret if already present

-t, --access-token <access_token>

Access token of the current user.

secrets-delete

Delete user secrets by name.

Examples:

$ reana-client secrets-delete RUCIO_USERNAME

reana-client secrets-delete [OPTIONS] SECRETS...

Options

-t, --access-token <access_token>

Access token of the current user.

Arguments

SECRETS

Required argument(s)

secrets-list

List user secrets.

Examples:

$ reana-client secrets-list

reana-client secrets-list [OPTIONS]

Options

-t, --access-token <access_token>

Access token of the current user.

start

Start previously created workflow.

The start command allows to start previously created workflow. The workflow execution can be further influenced by passing input prameters using -p or --parameters flag and by setting additional operational options using -o or --options. The input parameters and operational options can be repetitive. For example, to disable caching for the Serial workflow engine, you can set -o CACHE=off.

Examples:

$ reana-client start -w myanalysis.42 -p sleeptime=10 -p myparam=4

$ reana-client start -w myanalysis.42 -p myparam1=myvalue1 -o CACHE=off

reana-client start [OPTIONS]

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-t, --access-token <access_token>

Access token of the current user.

-p, --parameter <parameters>

Additional input parameters to override original ones from reana.yaml. E.g. -p myparam1=myval1 -p myparam2=myval2.

-o, --option <options>

Additional operational options for the workflow execution. E.g. CACHE=off. (workflow engine - serial) E.g. –debug (workflow engine - cwl)

--follow

If set, follows the execution of the workflow until termination.

status

Get status of a workflow.

The status command allow to retrieve status of a workflow. The status can be created, queued, running, failed, etc. You can increase verbosity or filter retrieved information by passing appropriate command-line options.

Examples:

$ reana-client status -w myanalysis.42

$ reana-client status -w myanalysis.42 -v –json

reana-client status [OPTIONS]

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

--format <_format>

Format output by displaying only certain columns. E.g. –format name,status.

--json

Get output in JSON format.

--include-duration

Include the duration of the workflow in seconds. In case the workflow is in progress, its duration as of now will be shown.

-t, --access-token <access_token>

Access token of the current user.

-v, --verbose

Set status information verbosity.

stop

Stop a running workflow.

The stop command allows to hard-stop the running workflow process. Note that soft-stopping of the workflow is currently not supported. This command should be therefore used with care, only if you are absolutely sure that there is no point in continuing the running the workflow.

Example:

$ reana-client stop -w myanalysis.42 –force

reana-client stop [OPTIONS]

Options

--force

Stop a workflow without waiting for jobs to finish.

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-t, --access-token <access_token>

Access token of the current user.

upload

Upload files and directories to workspace.

The upload command allows to upload workflow input files and directories. The SOURCES argument can be repeated and specifies which files and directories are to be uploaded, see examples below. The default behaviour is to upload all input files and directories specified in the reana.yaml file.

Examples:

$ reana-client upload -w myanalysis.42

$ reana-client upload -w myanalysis.42 code/mycode.py

reana-client upload [OPTIONS] SOURCES

Options

-w, --workflow <workflow>

Name or UUID of the workflow. Overrides value of REANA_WORKON environment variable.

-t, --access-token <access_token>

Access token of the current user.

Arguments

SOURCES

Optional argument(s)

validate

Validate workflow specification file.

The validate command allows to check syntax and validate the reana.yaml workflow specification file.

Examples:

$ reana-client validate -f reana.yaml

reana-client validate [OPTIONS]

Options

-f, --file <file>

REANA specification file describing the workflow to execute. [default=reana.yaml]

--environments

If set, check all runtime environments specified in REANA specification file. [default=False]

--pull

If set, try to pull remote environment image from registry to perform validation locally. Requires --environments flag. [default=False]

--server-capabilities

If set, check the server capabilities such as workspace validation. [default=False]

-t, --access-token <access_token>

Access token of the current user.

version

Show version.

The version command shows REANA client version.

Examples:

$ reana-client version

reana-client version [OPTIONS]

API docs

REANA REST API client.

reana_client.api.client.add_secrets(secrets, overwrite, access_token)

Add new secrets.

Parameters:

• secrets –

  dictionary containing all the secrets to be sent. The dictionary has the secret names for keys and for each key there is a dictionary with two fields:

  • value: a base64 encoded file or literal string

  • type: "file" or "env"

• overwrite – whether secrets should be overwritten when they already exist.

• access_token – access token of the current user.

Returns:

a dictionary containing the message key with a success message.

reana_client.api.client.close_interactive_session(workflow, access_token)

Close an interactive workflow session.

Parameters:

• workflow – name or id of the workflow to close.

• access_token – workflow owner REANA access token.

Returns:

the relative path to the interactive session.

reana_client.api.client.create_workflow(reana_specification, name, access_token)

Create a workflow.

Parameters:

• reana_specification – a dictionary representing the REANA specification of the workflow.

• name – name of the workflow.

• access_token – access token of the current user.

Returns:

if the workflow was created successfully, a dictionary with the information about the workflow_id and workflow_name, along with a message of success.

reana_client.api.client.create_workflow_from_json(name, access_token, workflow_json=None, workflow_file=None, parameters=None, workflow_engine='yadage', outputs=None, workspace_path=None)

Create a workflow from JSON specification.

Parameters:

• name – name or UUID of the workflow to be started.

• access_token – access token of the current user.

• workflow_json – workflow specification in JSON format.

• workflow_file – workflow specification file path. Ignores workflow_json if provided.

• parameters – workflow input parameters dictionary.

• workflow_engine – one of the workflow engines (yadage, serial, cwl)

• outputs – dictionary with expected workflow outputs.

• workspace_path – path to the workspace where the workflow is located.

Returns:

if the workflow was created successfully, a dictionary with the information about the workflow_id and workflow_name, along with a message of success.

Example:

create_workflow_from_json(
    workflow_json=workflow_json,
    name='workflow_name.1',
    access_token='access_token',
    parameters={'files': ['file.txt'],
        'parameters': {'key': 'value'}},
    workflow_engine='serial')

reana_client.api.client.delete_file(workflow, file_name, access_token)

Delete the requested file if it exists.

Parameters:

• workflow – name or id of the workflow.

• file_name – file name or path to the file requested.

• access_token – access token of the current user.

Returns:

a dictionary with two keys: deleted and failed. Each of this keys contains another dictionary with the name of the file as key and info about the size as value.

reana_client.api.client.delete_secrets(secrets, access_token)

Delete a list of secrets.

Parameters:

• secrets – list of secret names to be deleted.

• access_token – access token of the current user.

Returns:

a list with the names of the deleted secrets.

reana_client.api.client.delete_workflow(workflow, all_runs: bool, workspace: bool, access_token: str)

Delete a workflow.

Please note that the workspace will always be deleted, even if workspace is set to False.

Parameters:

• workflow – name or id of the workflow.

• all_runs – whether to delete all the runs of the workflow.

• workspace – whether to delete the workspace of the workflow.

• access_token – access token of the current user.

Returns:

a dictionary that cointains info about the deleted workflow (workflow_id, workflow_name, status, user), and a message key.

reana_client.api.client.diff_workflows(workflow_id_a, workflow_id_b, brief, access_token, context_lines)

Return the list of differences between two workflows.

Parameters:

• workflow_id_a – UUID which identifies the first workflow.

• workflow_id_b – UUID which identifies the second workflow.

• brief – Flag specifying desired detail in diff.

• context_lines – Optional parameter to set the number of context lines shown in the diff output.

• access_token – API token of user requesting diff.

Returns:

a list of dictionaries composed by asset, type, lines, a and b. Asset refers to the workflow asset where a difference was found, type refers to the asset type, lines refer to the lines of the file where the differences are and a, b fields are the actual lines that differ.

reana_client.api.client.download_file(workflow, file_name, access_token)

Download the requested file if it exists.

Parameters:

• workflow – name or id of the workflow.

• file_name – file name or path to the file requested.

• access_token – access token of the current user.

Returns:

a tuple containing file binary content, filename and whether the returned file is a zip archive containing multiple files.

reana_client.api.client.get_user_quota(access_token)

Retrieve user quota usage and limits.

Parameters:

access_token – access token of the current user.

Returns:

a dictionary with the information about the usage and limits of the user’s quota. The keys are cpu and disk, and refer to the respective usage and limits.

reana_client.api.client.get_workflow_disk_usage(workflow, parameters, access_token)

Display disk usage workflow.

Parameters:

• workflow – name or id of the workflow.

• parameters –

  a dictionary to customize the response. It has the following (optional) keys:

  • summarize: a boolean value to indicate whether to summarize the response to include only the total workspace disk usage

  • search: a string to filter the response by file name

• access_token – access token of the current user.

Returns:

a dictionary containing the workflow_id, workflow_name, and the user ID, with a disk_usage_info keys that contains a list of dictionaries, each of one corresponding to a file, with the name and size keys.

reana_client.api.client.get_workflow_logs(workflow, access_token, steps=None, page=None, size=None)

Get logs from a workflow engine.

Parameters:

• workflow – name or id of the workflow.

• access_token – access token of the current user.

• steps – list of step names to get logs for.

• page – page number of returned log list.

• size – page size of returned log list.

Returns:

a dictionary with a logs key containing a JSON string that contains the requested logs.

reana_client.api.client.get_workflow_parameters(workflow, access_token)

Get parameters of previously created workflow.

Parameters:

• workflow – name or id of the workflow.

• access_token – access token of the current user.

Returns:

a dictionary that cointains info about the workflow (name, type), and a dictionary of workflow parameters under the parameters key.

reana_client.api.client.get_workflow_retention_rules(workflow, access_token)

Get the retention rules of a workflow.

Parameters:

• workflow – name or id of the workflow.

• access_token – access token of the current user.

Returns:

a dictionary containing the workflow_id, workflow_name, and the retention_rules key with a list of dictionaries representing the retention rules of the workflow. Each dictionary contains info about the affected workspace files, and the schedule of the retention rule.

reana_client.api.client.get_workflow_specification(workflow, access_token)

Get specification of previously created workflow.

Parameters:

• workflow – name or id of the workflow.

• access_token – access token of the current user.

Returns:

a dictionary that cointains two top-level keys: parameters, and specification (which contains a dictionary created from the workflow specification).

reana_client.api.client.get_workflow_status(workflow, access_token)

Get status of previously created workflow.

Parameters:

• workflow – name or id of the workflow.

• access_token – access token of the current user.

Returns:

a dictionary with the information about the workflow status. The dictionary has the following keys: id, logs, name, progress, status, user.

reana_client.api.client.get_workflows(access_token, type, verbose=False, page=None, size=None, status=None, search=None, include_progress=None, include_workspace_size=None, workflow=None)

List all existing workflows.

Parameters:

• access_token – access token of the current user.

• type – type of workflow to be listed: "interactive" if you want to list only the workflows that have an interactive session attached, with the info about the session, or "batch" (default) otherwise.

• verbose – show detailed information about workflows.

• page – page number of the paginated list of workflows.

• size – number of workflows per page.

• status – filter workflows by status.

• search – search workflows by name.

• include_progress – include progress information in the response.

• include_workspace_size – include workspace size information in the response.

• workflow – name or id of the workflow.

Returns:

a list of dictionaries with the information about the workflows. The information includes the workflow name, id, status, size, user (given as the user’s ID), and info about the interactive session if present.

reana_client.api.client.info(access_token)

List general information about the cluster.

Parameters:

access_token – access token of the current user.

Returns:

a dictionary containing relevant values and configuration options about the cluster. Each key contains a dictionary with the title key, explaining the meaning of the value, and the value key, containing the value itself. Example of the returned keys include compute_backends, default_kubernetes_memory_limit, and maximum_interactive_session_inactivity_period.

reana_client.api.client.list_files(workflow, access_token, file_name=None, page=None, size=None, search=None)

Return the list of files for a given workflow workspace.

Parameters:

• workflow – name or id of the workflow.

• access_token – access token of the current user.

• file_name – file name(s) (glob) to list.

• page – page number of returned file list.

• size – page size of returned file list.

• search – filter search results by parameters.

Returns:

a list of dictionaries that have the name, size and last-modified keys.

reana_client.api.client.list_secrets(access_token)

List user secrets.

Parameters:

access_token – access token of the current user.

Returns:

a list of dictionaries, each of one corresponding to a secret, with the name and type keys.

reana_client.api.client.mv_files(source, target, workflow, access_token)

Move target file(s) within workspace.

Parameters:

• source – source filename or path.

• target – target filename or path.

• workflow – name or id of the workflow.

• access_token – token of user.

Returns:

a dictionary containing the workflow_id, workflow_name, and a message about the success of the operation.

reana_client.api.client.open_interactive_session(workflow, access_token, interactive_session_type, interactive_session_configuration)

Open an interactive session inside the workflow workspace.

Parameters:

• workflow – name or id of the workflow whose workspace will be available inside the interactive session.

• access_token – Workflow owner REANA access token.

• interactive_session_type – Type of interactive session to spawn.

• interactive_session_configuration – Specific configuration for the interactive session.

Returns:

the relative path to the interactive session.

reana_client.api.client.ping(access_token)

Check if the REANA server is reachable and the user is correctly authenticated.

Parameters:

access_token – access token of the current user.

Returns:

a dictionary with the status key ("Connected" if the server is reachable, the error message if there is a problem), the error key (True if there is an error, False otherwise), and info about the current user in full_name and email.

reana_client.api.client.prune_workspace(workflow, include_inputs, include_outputs, access_token)

Prune workspace files.

Parameters:

• workflow – name or id of the workflow.

• include_inputs – whether to also delete inputs.

• include_outputs – whether to also delete outputs.

• access_token – access token of the current user.

Returns:

a dictionary containing the workflow_id, workflow_name, and a message key with the result of the operation.

reana_client.api.client.start_workflow(workflow, access_token, parameters)

Start a workflow.

Parameters:

• workflow – name or id of previously created workflow.

• access_token – access token of the current user.

• parameters – dict of workflow parameters to override the original ones (after workflow creation).

Returns:

if the workflow was started successfully, a dictionary with the information about the workflow_id, workflow_name, run_number, status, and user, along with a message of success.

reana_client.api.client.stop_workflow(workflow, force_stop, access_token)

Stop a workflow.

Parameters:

• workflow – name or id of the workflow.

• force_stop – whether to stop the workflow immediately, without waiting for the jobs to finish.

• access_token – access token of the current user.

Returns:

a dictionary that cointains info about the stopped workflow (workflow_id, workflow_name, status, user), and a message key.

reana_client.api.client.upload_file(workflow, file_, file_name, access_token)

Upload file to workflow workspace.

Parameters:

• workflow – name or id of the workflow.

• file – content of a file that will be uploaded.

• file_name – name of a file that will be uploaded.

• access_token – access token of the current user.

Returns:

if the file was uploaded successfully, a dictionary with a message of success.

reana_client.api.client.upload_to_server(workflow, paths, access_token)

Upload file or directory to REANA server.

Shared e.g. by code upload and inputs upload.

Parameters:

• workflow – name or id of workflow whose workspace should be used to store the files.

• paths – absolute filepath(s) of files to be uploaded.

• access_token – access token of the current user.

Returns:

the list of path of files that were uploaded.

Changes

Version 0.9.2 (UNRELEASED)

• Fixes create_workflow_from_json API command to always send the workflow specification to the server.

• Fixes list command to be case-insensitive when using the --sort flag to sort the workflow runs by a specific column name.

• Fixes run wrapper command for workflows that do not contain inputs clause in their specification.

Version 0.9.1 (2023-09-27)

• Adds support for Python 3.12.

• Adds prune command to delete all intermediate files of a given workflow. Use with care.

• Changes open command to inform user about the auto-closure of interactive sessions after a certain inactivity timeout.

• Changes validate command to display non-critical validation warnings when checking the REANA specification file.

• Fixes list command to correctly list workflows when sorting them by their run number or by the size of their workspace.

• Fixes du command help message typo.

• Fixes validation --environments command to correctly handle fully-qualified image names.

Version 0.9.0 (2023-01-26)

• Adds support for Python 3.11.

• Adds support for .gitignore and .reanaignore to specify files that should not be uploaded to REANA.

• Adds retention-rules-list command to list the retention rules of a workflow.

• Changes REANA specification loading and validation functionalities by porting some of the logic to reana-commons.

• Changes create and restart commands to always upload the REANA specification file.

• Changes delete command to always delete the workflow’s workspace.

• Changes delete_workflow Python API function to always delete the workflow’s workspace.

• Changes download command to add the possibility to write files to the standard output via -o - option.

• Changes list command to hide deleted workflows by default.

• Changes list command to allow displaying deleted workflows via --all and --show-deleted-runs options.

• Changes list and status commands to allow displaying the duration of workflows with the --include-duration option.

• Changes mv command to allow moving files while a workflow is running.

• Changes upload command to prevent uploading symlinks.

• Changes validation --environment command to use Docker registry v2 APIs to check that a Docker image exists in DockerHub.

• Fixes list command to highlight the workflow specified in REANA_WORKON correctly.

• Fixes secrets-delete command error message when deleting non existing secrets.

• Fixes start command to report failed workflows as errors.

• Fixes start and run commands to correctly follow the execution of the workflow until termination.

• Fixes status command to respect output format provided by the --format option.

• Fixes upload command to report when input directories are listed under the files section in the REANA specification file and vice versa.

• Fixes validate --environment command to detect illegal whitespace characters in Docker image names.

Version 0.8.1 (2022-02-15)

• Adds support for creating reana-client standalone AppImage executables.

• Adds support for Python 3.10.

• Adds workflow name validation for create_workflow_from_json() Python API function.

• Fixes formatting of error messages and sets appropriate exit status codes.

Version 0.8.0 (2021-11-24)

• Adds support for running and validating Snakemake workflows.

• Adds support for outputs.directories in reana.yaml allowing to easily download output directories.

• Adds new command quota-show to retrieve information about total CPU and Disk usage and quota limits.

• Adds new command info that retrieves general information about the cluster, such as available workspace path settings.

• Changes validate command to add the possibility to check the workflow against server capabilities such as desired workspace path via –server-capabilities option.

• Changes list command to add the possibility to filter by workflow status and search by workflow name via --filter option.

• Changes list command to add the possibility to filter and display all the runs of a given workflow via -w option.

• Changes list command to stop including workflow progress and workspace size by default. Please use new options –include-progress and –include-workspace-size to show this information.

• Changes list --sessions command to display the status of interactive sessions.

• Changes logs command to display also the start and finish times of individual jobs.

• Changes ls command to add the possibility to filter by file name, size and last-modified values via --filter option.

• Changes du command to add the possibility filter by file name and size via --filter option.

• Changes delete command to prevent hard-deletion of workflows.

• Changes Yadage workflow specification loading to be done in reana-commons.

• Changes CWL workflow engine to cwltool version 3.1.20210628163208.

• Removes support for Python 2.7. Please use Python 3.6 or higher from now on.

Version 0.7.5 (2021-07-05)

• Changes workflow validation to display more granular output.

• Changes workflow parameters validation to warn about misused parameters for each step.

• Changes dependencies to unpin six so that client may be installed in more contexts.

• Fixes environment image validation not to test repetitively the same image.

• Fixes upload_to_server() Python API function to silently skip uploading in case of none-like inputs.

Version 0.7.4 (2021-04-28)

• Adds support of wildcard patterns to ls command.

• Adds support of directory download and wildcard patterns to download command.

• Changes list command to include deleted workflows by default.

• Fixes environment image validation info message where UIDs were switched.

Version 0.7.3 (2021-03-24)

• Adds validation of workflow input parameters to the validate command.

• Adds optional validation of workflow environment images (--environments) to the validate command.

Version 0.7.2 (2021-01-15)

• Adds support for Python 3.9.

• Fixes exception handling when uploading files.

• Fixes minor code warnings.

• Fixes traling slash issue from user exported REANA_SERVER_URL.

Version 0.7.1 (2020-11-10)

• Changes ping command output to include REANA client and server version information.

• Fixes upload command to properly display errors.

Version 0.7.0 (2020-10-20)

• Adds option to logs command to filter job logs according to compute backend, docker image, status and step name.

• Adds new restart command to restart previously run or failed workflows.

• Adds possibility to specify operational options in the reana.yaml of the workflow.

• Fixes user experience by preventing dots as part of the workflow name to avoid confusion with restart runs.

• Changes du command output format.

• Changes file loading to optimise CLI performance.

• Changes logs command to enhance formatting using marks and colours.

• Changes from Bravado to requests to improve download performance.

• Changes ping command to perform user access token validation.

• Changes defaults to accept both reana.yaml and reana.yml filenames.

• Changes diff command to improve output formatting.

• Changes code formatting to respect black coding style.

• Changes documentation to single-page layout.

Version 0.6.1 (2020-06-09)

• Fixes installation troubles for REANA 0.6.x release series by pinning several dependencies.

Version 0.6.0 (2019-12-27)

• Introduces user secrets management commands secrets-add, secrets-list and secrets-delete.

• Enhances run and create commands to allow specifying workfow via the --workflow flag.

• Introduces new command version to report client version.

• Fixes upload command behaviour for uploading very large files.

• Simplifies run command by removing free upload parameters.

• Upgrades cwltool to 1.0.20191022103248.

• Disables SSL verification warnings when talking to self-signed server certificates.

Version 0.5.0 (2019-04-24)

• Introduces new resources field in reana.yaml specification file allowing to declare computing resources needed for workflow runs, such as the CVMFS repositories via cvmfs subfield.

• Improves reana-client embedded command-line documentation (-help) by grouping commands and providing concrete usage examples for all commands.

• Enhances workflow start command allowing to override input parameters (--parameter) and to specify additional operational options (--option).

• Introduces new workflow run wrapper command that creates workflow, uploads its input data and code and starts its execution.

• Introduces new workflow stop command for stopping a running workflow.

• Enhances workflow logs command output capabilities via new --json option.

• Introduces new workflow diff command for comparing two workflow runs.

• Introduces new workflow delete command for deleting one or more workflow runs.

• Introduces new session open command allowing to run interactive sessions such as Jupyter notebook upon workflow workspace.

• Introduces new session close command for closing interactive sessions.

• Renames past workflows command to list allowing to list both workflow runs and interactive sessions.

• Introduces new workspace du command for checking workspace disk usage.

• Introduces new workspace mv command for moving files within workspace.

• Introduces new workspace rm command for removing files within workspace.

• Renames past workspace list command to ls allowing to list workspace files. Enhances its output capabilities via new --format option.

• Introduces new API function create_workflow_from_json() which allows developers and third-party systems to create workflows directly from JSON specification.

Version 0.4.0 (2018-11-07)

• Enhances test suite and increases code coverage.

• Changes license to MIT.

Version 0.3.1 (2018-09-25)

• Amends upload and download commands that will now upload/download all the files specified in reana.yaml in case no arguments are provided.

• Fixes status command’s JSON output mode.

• Upgrades CWL reference implementation to version 1.0.20180912090223.

• Renames Serial workflow operational parameter from CACHING``to ``CACHE.

• Adds support for Python 3.7.

Version 0.3.0 (2018-08-10)

• Adds support for Serial workflows.

• CLI refactored to a flat design:

  • inputs/outputs/code removed, everything is a file managed with upload/download/list commands.

  • Removes workflow command, workflows are managed with create/start/status.

• Removes analyes command, now validate is first level command.

• status now shows the selected workflow progress and current command on verbose mode.

• Requires the usage of an access token to talk to REANA Server.

• Fixes bug when uploading binary files.

• Supports addition of workflow engine parameters when using start for serial workflows.

• Improves error messages.

Version 0.2.0 (2018-04-20)

• Adds support for Common Workflow Language workflows.

• Adds support for persistent user-selected workflow names.

• Enables file and directory input uploading using absolute paths.

• Adds new status command to display the current status of the client.

• Reduces verbosity level for commands and improves error messages.

Version 0.1.0 (2018-01-30)

• Initial public release.

Please beware

Please note that REANA is in an early alpha stage of its development. The developer preview releases are meant for early adopters and testers. Please don’t rely on released versions for any production purposes yet.

Contributing

Bug reports, issues, feature requests, and other contributions are welcome. If you find a demonstrable problem that is caused by the REANA code, please:

1. Search for already reported problems.

2. Check if the issue has been fixed or is still reproducible on the latest master branch.

3. Create an issue, ideally with a test case.

If you create a pull request fixing a bug or implementing a feature, you can run the tests to ensure that everything is operating correctly:

$ ./run-tests.sh

Each pull request should preserve or increase code coverage.

License

MIT License

Copyright (C) 2017, 2018, 2019, 2020, 2021, 2022, 2023 CERN.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

In applying this license, CERN does not waive the privileges and immunities granted to it by virtue of its status as an Intergovernmental Organization or submit itself to any jurisdiction.

Authors

The list of contributors in alphabetical order:

• Adelina Lintuluoto

• Audrius Mecionis

• Anton Khodak

• Camila Diaz

• Daniel Prelipcean

• Diego Rodriguez

• Dinos Kousidis

• Giuseppe Steduto

• Harri Hirvonsalo

• Jan Okraska

• Leticia Wanderley

• Marco Donadoni

• Marco Vidal

• Matthew Feickert

• Rokas Maciulaitis

• Sinclert Perez

• Tibor Simko

• Vladyslav Moisieienkov