Blue Cloud - D4.4: Blue Cloud VRE Common Facilities (Release 2)

Assante M., Candela L., Pagano P., Cirillo R., Dell'Amico A., Frosini L., Lelii L., Lettere M., Mangiacrapa F., Panichi G., Sinibaldi F.

Blue-Cloud Services  Virtual Research Environments 

The Blue-Cloud project is piloting a cyber platform bringing together and providing access to multidisciplinary data from observations and models, analytical tools, and computing facilities essential to support research to understand better and manage the many aspects of ocean sustainability. This goal is realised by developing, deploying and operating the Blue-Cloud platform whose architecture consists of two major families of components: (a) the Blue Cloud Data Discovery and Access System to serve federated discovery and access to 'blue data' infrastructures; and (b) the Blue Cloud Virtual Research Environment (VRE) component to provide a Blue Cloud VRE as a federation of computing platforms and analytical services. This Deliverable D4.4 "Blue Cloud VRE Common Facilities (Release 2)" is the revised version of the D4.2 "Blue Cloud VRE Common Facilities (Release 1)". This revised version of the document covers the second period of the project, from M13 up to M27, including the up-to-date information of the services reported on D4.2 and the new services that have been developed and added to the VRE common facilities in the reporting period to serve the needs of the Blue Cloud community. The major changes and new services this deliverable introduces are: an Orchestrator (cf. Sec. 3.3), i.e. a software that allows for a declarative, technology agnostic definition of workflows to coordinate the execution of tasks across diverse services and systems; enhancements to the Workspace service to support tailored storage persistence and satisfy different application scenarios (cf. Sec. 4.1); enhancements in the Publishing Framework (cf. Sec. 6), namely the catalogue extension to deposit catalogue items to Zenodo and the facility to publish geospatial data from the workspace; the facility to interface with the Data Discovery & Access System (cf. Sec. 7.1) to transfer datasets of interest into the workspace for future uses; the notebook to facilitate the exploitation of the WEkEO Harmonised Data Access (HDA) API (cf. Sec. 7.2). This deliverable also updates the Identity and Access Management (cf. Sec. 3.1) and the Analytics Framework (cf. Sec. 5.1 and Sec 5.2) with minor changes reflecting the activities performed in the reporting period. A description of all the services previously documented in D4.2, not modified in the period, is preserved for this document to be self-contained and provide the reader with an overall description of the whole VRE Common Facilities offering. A total of 15 services and components are described in this deliverable by reporting their design principles, architecture and main features. These services and components contribute functionalities to the Blue Cloud VRE Enabling Framework (Identity and Access Management, VRE Management, Orchestrator), Collaborative framework (Workspace and Social Networking), Analytics Framework (Software and Algorithm Importer, Smart Executor), Publishing Framework (Catalogue Service) and improved support for RStudio, JupyterHub, ShinyProxy, and Docker Applications. Additionally, two new VRE services, aiming at bridging two VRE external systems such as the the WEkEO1 catalogue from Copernicus and the Data Discovery and Access from Blue-Cloud with the VRE tools are described. Services and components discussed in this deliverable have contributed to 14 gCube releases, from gCube 4.26 (November 2020) to gCube 5.6.0 (November 2021). They have been used to develop and operate the Virtual Laboratories of the Blue Cloud gateway https://blue-cloud.d4science.org and its underlying infrastructure. At the time of this deliverable the Blue-Cloud gateway and its services are serving more than 730 users with a total of 19000+ working sessions.

Source: ISTI Project report, Blue Cloud, D4.4, 2021