SoBigData++ - SoBigData e-Infrastructure Operation Report 3

Assante M., Candela L., Dell'Amico A., Frosini L., Mangiacrapa F., Molinaro E., Oliviero A., Pagano P., Panichi G., Piccioli T.

VRE, Operation, BigData, SoBigData 

This Deliverable builds upon and updates the previous reports, D9.2 - “SoBigData e-Infrastructure Operation Report 2” [5] and D9.1 - “SoBigData e-Infrastructure Operation Report 1” [3]. The SoBigData e-Infrastructure has been pivotal in enabling the core services and research support required for the SoBigData++ project, including Virtual Research Environments (VREs), the Catalogue, and Analytics Services. It is accessible through the SoBigData gateway (https://sobigdata.d4science.org), which provides end-users with seamless access to tools, datasets, and services. The SoBigData e-Infrastructure is built upon the D4Science infrastructure, offering a comprehensive platform that facilitates collaborative, transparent, and interdisciplinary research. The deployment and operation of VREs followed a well-defined procedure, leveraging the consolidated process inherited from D4Science. Throughout the 60 months of the project, a total of 27 VREs were created and operated to meet project and community needs. These VREs were classified into five categories: Exploratories, Applications, Virtual Labs, Training, and Management. Notable examples include, (i) SoBigDataLab and SoBigDataLab-PlusPlus for method development and experiments, (ii) Training VREs created for events like Summer Schools and specialised workshops, and (iii) Research spaces (formerly known as Exploratories) supporting targeted domains, such as Migration Studies, Sports Data Science, and Social Impacts of AI. The SoBigData Catalogue (https://sobigdata.d4science.org/catalogue-sobigdata) emerged as a critical resource for both human users and integrated services, enabling access to datasets, services, and analytical methods. The catalogue supports customisable item profiles enriched with metadata fields, controlled vocabularies, and validation rules. By end of term, the Catalogue recorded significant growth, particularly in key item types such as Methods (192 items) and Datasets (250 items). This expansion underscores the Catalogue’s role in promoting resource discoverability and supporting research workflows. Its usage indicators demonstrate its active adoption, with 31,909 total accesses, 29,595 metadata views, and 4,171 resource views recorded. Monthly trends reveal consistent engagement, highlighting its importance in the research ecosystem. The Social Mining Analytics Engine (SMAE) transitioned through the development of a new service, namely Cloud Computing Platform (CCP), offering enhanced scalability and automation through container orchestrations. Methods hosted on the SMAE span multiple categories, such as Text Processing, Web Analytics, and Image Analysis. Over the last year, the platform executed an average of 6.4 million method invocations per month, peaking at 16 million executions in July 2024. As of mid-December ’24, the e-infrastructure serves more than 13,000 users, with an overall trend in the use of the SoBigData VREs from January 2020 to December 2024, highlighting their importance for the research community. The steady engagement through 2023 and 2024, with peaks like July 2024 (2,592 sessions), underscores the VREs continued relevance and utility.