not yet published
Conference article
Restricted
AdapTSoS: adaptive testing of systems of systems
Santos Da Silva Samira, Lonetti Francesca, Bertolino Antonia
Testing of System of Systems (SoS) presents many challenges due to the dynamic and continuous evolution of its Constituent Systems (CSs). Existing SoS testing strategies support the definition of test cases but fail to address the dynamic changes of CSs. New test cases need to be added or existing ones must be updated to cope with the variable functionalities of CSs. Adaptive testing is a strategy to dynamically adapt a set of test configurations or test cases to changes in the system under test over time. The main idea of this paper is to provide a new adaptive testing strategy, AdapTSoS, for the adaptive testing of SoSs in order to deal with the continuous changes of variable functionalities of the CSs. The proposed strategy leverages a variability model representing the variable functionalities of the CSs and a Discrete Time Markov Chain simulating the SoS changes over time, to then provide a set of adaptive test configurations to be executed on the SoS. We illustrate the application of AdapTSoS to an existing SoS example and discuss the benefits provided by the proposed approach.Source: LECTURE NOTES IN COMPUTER SCIENCE, vol. 16315. Lisbon, Portugal, 3-5 september 2025
Santos Da Silva Samira, Lonetti Francesca, Bertolino Antonia
Testing of System of Systems (SoS) presents many challenges due to the dynamic and continuous evolution of its Constituent Systems (CSs). Existing SoS testing strategies support the definition of test cases but fail to address the dynamic changes of CSs. New test cases need to be added or existing ones must be updated to cope with the variable functionalities of CSs. Adaptive testing is a strategy to dynamically adapt a set of test configurations or test cases to changes in the system under test over time. The main idea of this paper is to provide a new adaptive testing strategy, AdapTSoS, for the adaptive testing of SoSs in order to deal with the continuous changes of variable functionalities of the CSs. The proposed strategy leverages a variability model representing the variable functionalities of the CSs and a Discrete Time Markov Chain simulating the SoS changes over time, to then provide a set of adaptive test configurations to be executed on the SoS. We illustrate the application of AdapTSoS to an existing SoS example and discuss the benefits provided by the proposed approach.Source: LECTURE NOTES IN COMPUTER SCIENCE, vol. 16315. Lisbon, Portugal, 3-5 september 2025
2026
Journal article
Open Access
Recent developments in software engineering for systems-of-systems and software ecosystems
Lonetti Francesca, Bertolino Antonia, Antonino Pablo, Bae Doo-Hwan
The increasing scale, distribution, and interconnection of software-intensive systems continue to change the landscape of software engineering research and practice, bringing the diffusion of two similar paradigms: Systems-of-Systems (SoS) and Software Ecosystems (SECO). SoS are characterized by the integration of operationally and managerially independent systems that join together to accomplish a common mission. SoS are central to domains such as transportation, healthcare, defense, smart cities, and industrial automation where heterogeneous systems must cooperate, exchange information, and adapt to evolving missions. On the other hand, SECO describe environments in which a platform and its surrounding network of developers, partners, and organizations co-create software offerings. In SECO, technical artifacts interact with economic and social processes. Modern digital platforms, mobile operating systems, cloud services, and enterprise platforms leverage the capabilities of their ecosystems. This editorial brings together perspectives that explore the shared challenges and complementary insights of SoS and SECO research, aiming to foster a richer understanding of complex software-intensive systems and highlight new opportunities for collaboration across communities. From the long-running, successful series of the International Workshop on Software Engineering for Systems-of-Systems and Software Ecosystems (SESoS), co-located with the IEEE/ACM International Conference on Software Engineering (ICSE), we present this special issue of the Journal of Systems and Software on the topics of SESoS 2024 in Lisbon, Portugal. From a total of 18 submissions, 7 articles were accepted in this special issue. The articles in this collection address fundamental questions of SoS and SECO, including the evolution of functional relations and experimentation practices, the key factors affecting developer experience, also related to women’s inclusion, as well as the adoption of GenAI-driven approaches for vulnerability fixing. These articles offer updates on current advances of SoS and SECO engineering to researchers and practitioners, highlighting opportunities for future research.Source: THE JOURNAL OF SYSTEMS AND SOFTWARE, vol. 236 (issue 112819)
DOI: 10.1016/j.jss.2026.112819
DOI: 10.1016/j.jss.2026.112814
Metrics:
Lonetti Francesca, Bertolino Antonia, Antonino Pablo, Bae Doo-Hwan
The increasing scale, distribution, and interconnection of software-intensive systems continue to change the landscape of software engineering research and practice, bringing the diffusion of two similar paradigms: Systems-of-Systems (SoS) and Software Ecosystems (SECO). SoS are characterized by the integration of operationally and managerially independent systems that join together to accomplish a common mission. SoS are central to domains such as transportation, healthcare, defense, smart cities, and industrial automation where heterogeneous systems must cooperate, exchange information, and adapt to evolving missions. On the other hand, SECO describe environments in which a platform and its surrounding network of developers, partners, and organizations co-create software offerings. In SECO, technical artifacts interact with economic and social processes. Modern digital platforms, mobile operating systems, cloud services, and enterprise platforms leverage the capabilities of their ecosystems. This editorial brings together perspectives that explore the shared challenges and complementary insights of SoS and SECO research, aiming to foster a richer understanding of complex software-intensive systems and highlight new opportunities for collaboration across communities. From the long-running, successful series of the International Workshop on Software Engineering for Systems-of-Systems and Software Ecosystems (SESoS), co-located with the IEEE/ACM International Conference on Software Engineering (ICSE), we present this special issue of the Journal of Systems and Software on the topics of SESoS 2024 in Lisbon, Portugal. From a total of 18 submissions, 7 articles were accepted in this special issue. The articles in this collection address fundamental questions of SoS and SECO, including the evolution of functional relations and experimentation practices, the key factors affecting developer experience, also related to women’s inclusion, as well as the adoption of GenAI-driven approaches for vulnerability fixing. These articles offer updates on current advances of SoS and SECO engineering to researchers and practitioners, highlighting opportunities for future research.Source: THE JOURNAL OF SYSTEMS AND SOFTWARE, vol. 236 (issue 112819)
DOI: 10.1016/j.jss.2026.112819
DOI: 10.1016/j.jss.2026.112814
Metrics:
See at:
Journal of Systems and Software 
| CNR IRIS | www.sciencedirect.com | Journal of Systems and Software 
| CNR IRIS
2025
Journal article
Open Access
Artificial neural networks applied to olive oil production and characterization: a systematic review
Lonetti F., Martelli F., Resta G.
In recent years, the olive oil sector has experienced growth due to the health benefits associated with olive oil and its increasing demand in international markets. Artificial Neural Networks (ANNs) have emerged as powerful tools in various scientific domains, enhancing both the efficiency and the accuracy of analyses in the olive oil sector. This paper aims to comprehensively review the adoption of ANNs in the assessment of olive oil across production and post-production stages. To achieve this goal, we followed the well-known guidelines of Kitchenham (2004) for performing systematic reviews. This up-to-date review examines literature from the last seven years, analyzing 628 publications and finally selecting 79 primary studies. Through a systematic and comprehensive analysis, this review seeks to provide insights into the current state of research, identify gaps in knowledge, and offer recommendations for future directions in harnessing ANNs to optimize the production and post-production analyses of olive oil.Source: INTELLIGENT SYSTEMS WITH APPLICATIONS, vol. 26 (issue 200525)
DOI: 10.1016/j.iswa.2025.200525
Metrics:
Lonetti F., Martelli F., Resta G.
In recent years, the olive oil sector has experienced growth due to the health benefits associated with olive oil and its increasing demand in international markets. Artificial Neural Networks (ANNs) have emerged as powerful tools in various scientific domains, enhancing both the efficiency and the accuracy of analyses in the olive oil sector. This paper aims to comprehensively review the adoption of ANNs in the assessment of olive oil across production and post-production stages. To achieve this goal, we followed the well-known guidelines of Kitchenham (2004) for performing systematic reviews. This up-to-date review examines literature from the last seven years, analyzing 628 publications and finally selecting 79 primary studies. Through a systematic and comprehensive analysis, this review seeks to provide insights into the current state of research, identify gaps in knowledge, and offer recommendations for future directions in harnessing ANNs to optimize the production and post-production analyses of olive oil.Source: INTELLIGENT SYSTEMS WITH APPLICATIONS, vol. 26 (issue 200525)
DOI: 10.1016/j.iswa.2025.200525
Metrics:
See at:
Intelligent Systems with Applications | CNR IRIS | www.sciencedirect.com | CNR IRIS
2025
Other
Open Access
ISTI-day 2025 Proceedings
Del Corso G., Pedrotti A., Federico G., Gennaro C., Carrara F., Amato G., Di Benedetto M., Gabrielli E., Belli D., Matrullo Zoe, Miori V., Tolomei Gabriele, Waheed T., Marchetti E., Calabrò Antonello., Rossetti G., Stella Massimo, Cazabet Rémy, Abramski K., Cau E., Citraro S., Failla A., Mesina V., Morini V., Pansanella V., Colantonio S., Germanese D., Pascali M. A., Bianchi L., Messina N., Falchi F., Barsellotti L., Pacini G., Cassese M., Puccetti G., Esuli A., Volpi L., Moreo Alejandro, Sebastiani F., Sperduti G., Nguyen Dong, Broccia G., Ter Beek M. H., Ferrari A., Massink M., Belmonte Gina, Ciancia V., Papini O., Canapa G., Catricalà B., Manca M., Paternò F., Santoro C., Zedda E., Gallo S., Maenza S., Mattioli A., Simeoli L., Rucci D., Carlini E., Dazzi P., Kavalionak H., Mordacchini M., Rulli C., Muntean Cristina Ioana, Nardini F. M., Perego R., Rocchietti G., Lettich F., Renso C., Pugliese C., Casini G., Haldimann Jonas, Meyer Thomas, Assante M., Candela L., Dell'Amico A., Frosini L., Mangiacrapa F., Oliviero A., Pagano P., Panichi G., Peccerillo B., Procaccini M., Mannocci A., Manghi P., Lonetti F., Kang Dongjae, Di Giandomenico F., Jee Eunkyoung, Lazzini G., Conti F., Scopigno R., D'Acunto M., Moroni D., Cafiso M., Paradisi P., Callieri M., Pavoni G., Corsini M., De Falco A., Sala F., Saraceni Q., Gattiglia Gabriele
ISTI-Day is an annual information and networking event organized by the Institute of Information Science and Technologies "A. Faedo" (ISTI) of the Italian National Research Council (CNR). This event features an opening talk of the Director of the Dept. DIITET (Emilio F. Campana) as well as an overview of the Institute's activities presented by the ISTI Director (Roberto Scopigno). Those institutional segments are complemented by dedicated presentations and round tables featuring former staff members, as well as internal and external collaborators. To foster a network of knowledge and collaboration among newcomers, the 2025 ISTI Day edition also includes a large poster session that provides a comprehensive overview of current research activities. Each of the 13 laboratories contributes 1–3 posters, highlighting the most innovative work and offering early-career researchers a platform for discussion. Thus these proceedings include the posters selected for ISTI-Day 2025, reflecting the diverse and innovative nature of the Institute's research.
Del Corso G., Pedrotti A., Federico G., Gennaro C., Carrara F., Amato G., Di Benedetto M., Gabrielli E., Belli D., Matrullo Zoe, Miori V., Tolomei Gabriele, Waheed T., Marchetti E., Calabrò Antonello., Rossetti G., Stella Massimo, Cazabet Rémy, Abramski K., Cau E., Citraro S., Failla A., Mesina V., Morini V., Pansanella V., Colantonio S., Germanese D., Pascali M. A., Bianchi L., Messina N., Falchi F., Barsellotti L., Pacini G., Cassese M., Puccetti G., Esuli A., Volpi L., Moreo Alejandro, Sebastiani F., Sperduti G., Nguyen Dong, Broccia G., Ter Beek M. H., Ferrari A., Massink M., Belmonte Gina, Ciancia V., Papini O., Canapa G., Catricalà B., Manca M., Paternò F., Santoro C., Zedda E., Gallo S., Maenza S., Mattioli A., Simeoli L., Rucci D., Carlini E., Dazzi P., Kavalionak H., Mordacchini M., Rulli C., Muntean Cristina Ioana, Nardini F. M., Perego R., Rocchietti G., Lettich F., Renso C., Pugliese C., Casini G., Haldimann Jonas, Meyer Thomas, Assante M., Candela L., Dell'Amico A., Frosini L., Mangiacrapa F., Oliviero A., Pagano P., Panichi G., Peccerillo B., Procaccini M., Mannocci A., Manghi P., Lonetti F., Kang Dongjae, Di Giandomenico F., Jee Eunkyoung, Lazzini G., Conti F., Scopigno R., D'Acunto M., Moroni D., Cafiso M., Paradisi P., Callieri M., Pavoni G., Corsini M., De Falco A., Sala F., Saraceni Q., Gattiglia Gabriele
ISTI-Day is an annual information and networking event organized by the Institute of Information Science and Technologies "A. Faedo" (ISTI) of the Italian National Research Council (CNR). This event features an opening talk of the Director of the Dept. DIITET (Emilio F. Campana) as well as an overview of the Institute's activities presented by the ISTI Director (Roberto Scopigno). Those institutional segments are complemented by dedicated presentations and round tables featuring former staff members, as well as internal and external collaborators. To foster a network of knowledge and collaboration among newcomers, the 2025 ISTI Day edition also includes a large poster session that provides a comprehensive overview of current research activities. Each of the 13 laboratories contributes 1–3 posters, highlighting the most innovative work and offering early-career researchers a platform for discussion. Thus these proceedings include the posters selected for ISTI-Day 2025, reflecting the diverse and innovative nature of the Institute's research.
See at:
CNR IRIS | www.isti.cnr.it | CNR IRIS
2025
Journal article
Open Access
Using metamorphic relations in redundancy-based fault/intrusion tolerance
Di Giandomenico F., Masetti G., Lonetti F., Bertolino A.
Redundancy is widely used as a method for fault and intrusion tolerance. However, if the redundant components lack sufficient diversity, potentially dangerous common mode failures may go undetected. To address this issue, the design diversity approach has been proposed in the literature for decades. In this paper, we take an innovative approach to this problem by introducing a broader notion of diversity, which leverages Metamorphic Relations (MRs), i.e., necessary properties that must hold among diverse inputs and diverse outputs. We define two generic categories of MRs that establish data diversity and functional diversity. Furthermore, we elaborate on two corresponding logical architectures, paying particular attention to the necessary conditions for the adjudicator component. Finally, we present an initial evaluation of the proposed architectures, which points out the advantages with respect to their counterparts based on the traditional design diversity method, and discuss future research directions for this novel conceptual approach to redundancy-based fault/intrusion tolerance.Source: ACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY
DOI: 10.1145/3772722
Metrics:
Di Giandomenico F., Masetti G., Lonetti F., Bertolino A.
Redundancy is widely used as a method for fault and intrusion tolerance. However, if the redundant components lack sufficient diversity, potentially dangerous common mode failures may go undetected. To address this issue, the design diversity approach has been proposed in the literature for decades. In this paper, we take an innovative approach to this problem by introducing a broader notion of diversity, which leverages Metamorphic Relations (MRs), i.e., necessary properties that must hold among diverse inputs and diverse outputs. We define two generic categories of MRs that establish data diversity and functional diversity. Furthermore, we elaborate on two corresponding logical architectures, paying particular attention to the necessary conditions for the adjudicator component. Finally, we present an initial evaluation of the proposed architectures, which points out the advantages with respect to their counterparts based on the traditional design diversity method, and discuss future research directions for this novel conceptual approach to redundancy-based fault/intrusion tolerance.Source: ACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY
DOI: 10.1145/3772722
Metrics:
See at:
dl.acm.org | CNR IRIS | ACM Transactions on Software Engineering and Methodology | CNR IRIS
2024
Journal article
Open Access
A framework for the design of fault-tolerant systems-of-systems
Cerdeira Ferreira F. H., Nakagawa E. Y., Bertolino A., Lonetti F., De Oliveira Neves V., Pereira Dos Santos R.
Context: Systems-of-Systems (SoS) increasingly permeate everyday life in various critical domains. Due to their dynamic nature, guaranteeing their fault tolerance is challenging. Fault-tolerant SoS must deal with behavioral changes in constituent systems, whether accidental or deliberate. Goal: This work proposes ReViTA, a framework to assist professionals in designing fault-tolerant SoS that can continue to provide their function even in the presence of disturbances, i.e., events that affect the ability of an SoS to fulfill its mission. Methods: By adopting ReViTA, fault tolerance can be achieved by reconfiguring an SoS architecture to meet the critical mission requirements. Results: We performed two studies to evaluate the ReViTA acceptance by professionals. In the former, we gathered perceptions and suggestions from 14 professionals through individual interviews. In the latter, we involved a group of four professionals who applied ReViTA to a real-world scenario. Conclusion: The results demonstrate that ReViTA can effectively support professionals in designing faulttolerant SoS. Employing ReViTA also brings insights into costs and planning that are crucial for implementing fault-tolerance strategies. Using ReViTA facilitates a comprehensive understanding of conflicts and weaknesses in constituent systems and fosters collaboration between domain experts and decision-makers. Employing ReViTA also improves stakeholder communication and enhances resource utilization.Source: THE JOURNAL OF SYSTEMS AND SOFTWARE, vol. 211
DOI: 10.1016/j.jss.2024.112010
Metrics:
Cerdeira Ferreira F. H., Nakagawa E. Y., Bertolino A., Lonetti F., De Oliveira Neves V., Pereira Dos Santos R.
Context: Systems-of-Systems (SoS) increasingly permeate everyday life in various critical domains. Due to their dynamic nature, guaranteeing their fault tolerance is challenging. Fault-tolerant SoS must deal with behavioral changes in constituent systems, whether accidental or deliberate. Goal: This work proposes ReViTA, a framework to assist professionals in designing fault-tolerant SoS that can continue to provide their function even in the presence of disturbances, i.e., events that affect the ability of an SoS to fulfill its mission. Methods: By adopting ReViTA, fault tolerance can be achieved by reconfiguring an SoS architecture to meet the critical mission requirements. Results: We performed two studies to evaluate the ReViTA acceptance by professionals. In the former, we gathered perceptions and suggestions from 14 professionals through individual interviews. In the latter, we involved a group of four professionals who applied ReViTA to a real-world scenario. Conclusion: The results demonstrate that ReViTA can effectively support professionals in designing faulttolerant SoS. Employing ReViTA also brings insights into costs and planning that are crucial for implementing fault-tolerance strategies. Using ReViTA facilitates a comprehensive understanding of conflicts and weaknesses in constituent systems and fosters collaboration between domain experts and decision-makers. Employing ReViTA also improves stakeholder communication and enhances resource utilization.Source: THE JOURNAL OF SYSTEMS AND SOFTWARE, vol. 211
DOI: 10.1016/j.jss.2024.112010
Metrics:
See at:
IRIS Cnr | IRIS Cnr | IRIS Cnr | Journal of Systems and Software | CNR IRIS | CNR IRIS | CNR IRIS
2023
Conference article
Open Access
Cross-coverage testing of functionally equivalent programs
Bertolino A, De Angelis G, Di Giandomenico F, Lonetti F
Cross-coverage of a program P refers to the test coverage measured over a different program Q that is functionally equivalent to P. The novel concept of cross-coverage can find useful applications in the test of redundant software. We apply here cross-coverage for test suite augmentation and show that additional test cases generated from the coverage of an equivalent program, referred to as cross tests, can increase the coverage of a program in more effective way than a random baseline. We also observe that -contrary to traditional coverage testing-cross coverage could help finding (artificially created) missing functionality faults.DOI: 10.1109/ast58925.2023.00014
Metrics:
Bertolino A, De Angelis G, Di Giandomenico F, Lonetti F
Cross-coverage of a program P refers to the test coverage measured over a different program Q that is functionally equivalent to P. The novel concept of cross-coverage can find useful applications in the test of redundant software. We apply here cross-coverage for test suite augmentation and show that additional test cases generated from the coverage of an equivalent program, referred to as cross tests, can increase the coverage of a program in more effective way than a random baseline. We also observe that -contrary to traditional coverage testing-cross coverage could help finding (artificially created) missing functionality faults.DOI: 10.1109/ast58925.2023.00014
Metrics:
See at:
CNR IRIS | ieeexplore.ieee.org | ISTI Repository | CNR IRIS | CNR IRIS
2023
Conference article
Open Access
Automated derivation of test requirements for systems of systems
Azevedo Gonçalves J, Lonetti F, De Oliveira Neves V
Testing of Systems of Systems (SoS) is challenging and improving its cost-effectiveness is a relevant research topic. In this paper, we propose TESoS (Test Engine for Systems of Systems), a systematic approach that selects from SoS models, defined in mKAOS language, the functionalities to be tested and then automatically derives a set of test requirements. TESoS allows to classify test requirements according to unit, integration, and system testing levels. Moreover, it helps test planning by providing the tester with automated facilities for supporting the unit testing of constituent systems and computing the percentage of test requirements that are satisfied with a given test suite. We illustrate the TESoS application on an SoS case study in the educational domainDOI: 10.1109/sose59841.2023.10178516
Metrics:
Azevedo Gonçalves J, Lonetti F, De Oliveira Neves V
Testing of Systems of Systems (SoS) is challenging and improving its cost-effectiveness is a relevant research topic. In this paper, we propose TESoS (Test Engine for Systems of Systems), a systematic approach that selects from SoS models, defined in mKAOS language, the functionalities to be tested and then automatically derives a set of test requirements. TESoS allows to classify test requirements according to unit, integration, and system testing levels. Moreover, it helps test planning by providing the tester with automated facilities for supporting the unit testing of constituent systems and computing the percentage of test requirements that are satisfied with a given test suite. We illustrate the TESoS application on an SoS case study in the educational domainDOI: 10.1109/sose59841.2023.10178516
Metrics:
See at:
CNR IRIS | ieeexplore.ieee.org | ISTI Repository | CNR IRIS | CNR IRIS
2023
Journal article
Open Access
Model-based security testing in IoT systems: a rapid review
Lonetti F, Bertolino A, Di Giandomenico F
Context: Security testing is a challenging and effort-demanding task in IoT scenarios. The heterogeneous devices expose different vulnerabilities that can influence the methods and cost of security testing. Model-based security testing techniques support the systematic generation of test cases for the assessment of security requirements by leveraging the specifications of the IoT system model and of the attack templates. Objective: This paper aims to review the adoption of model-based security testing in the context of IoT, and then provides the first systematic and up-to-date comprehensive classification and analysis of research studies in this topic. Method: We conducted a systematic literature review analysing 803 publications and finally selecting 17 primary studies, which satisfied our inclusion criteria and were classified according to a set of relevant analysis dimensions. Results: We report the state-of-the-art about the used formalisms, the test techniques, the objectives, the target applications and domains; we also identify the targeted security attacks, and discuss the challenges, gaps and future research directions. Conclusion: Our review represents the first attempt to systematically analyze and classify existing studies on model-based security testing for IoT. According to the results, model-based security testing has been applied in core IoT domains. Models complexity and the need of modeling evolving scenarios that include heterogeneous open software and hardware components remain the most important shortcomings. Our study shows that model-based security testing of IoT applications is a promising research direction. The principal future research directions deal with: extending the existing modeling formalisms in order to capture all peculiarities and constraints of complex and large scale IoT networks; the definition of context-aware and dynamic evolution modelling approaches of IoT entities; and the combination of model-based testing techniques with other security test strategies such as penetration testing or learning techniques for model inference.Source: INFORMATION AND SOFTWARE TECHNOLOGY, vol. 164
DOI: 10.1016/j.infsof.2023.107326
Metrics:
Lonetti F, Bertolino A, Di Giandomenico F
Context: Security testing is a challenging and effort-demanding task in IoT scenarios. The heterogeneous devices expose different vulnerabilities that can influence the methods and cost of security testing. Model-based security testing techniques support the systematic generation of test cases for the assessment of security requirements by leveraging the specifications of the IoT system model and of the attack templates. Objective: This paper aims to review the adoption of model-based security testing in the context of IoT, and then provides the first systematic and up-to-date comprehensive classification and analysis of research studies in this topic. Method: We conducted a systematic literature review analysing 803 publications and finally selecting 17 primary studies, which satisfied our inclusion criteria and were classified according to a set of relevant analysis dimensions. Results: We report the state-of-the-art about the used formalisms, the test techniques, the objectives, the target applications and domains; we also identify the targeted security attacks, and discuss the challenges, gaps and future research directions. Conclusion: Our review represents the first attempt to systematically analyze and classify existing studies on model-based security testing for IoT. According to the results, model-based security testing has been applied in core IoT domains. Models complexity and the need of modeling evolving scenarios that include heterogeneous open software and hardware components remain the most important shortcomings. Our study shows that model-based security testing of IoT applications is a promising research direction. The principal future research directions deal with: extending the existing modeling formalisms in order to capture all peculiarities and constraints of complex and large scale IoT networks; the definition of context-aware and dynamic evolution modelling approaches of IoT entities; and the combination of model-based testing techniques with other security test strategies such as penetration testing or learning techniques for model inference.Source: INFORMATION AND SOFTWARE TECHNOLOGY, vol. 164
DOI: 10.1016/j.infsof.2023.107326
Metrics:
See at:
CNR IRIS | ISTI Repository | www.sciencedirect.com | CNR IRIS
2022
Journal article
Open Access
Designing and testing systems of systems: from variability models to test cases passing through desirability assessment
Lonetti F, De Oliveira Neves V, Bertolino A
In the early stages of a system of systems (SoS) conception, several constituent systems could be available that provide similar functionalities. An SoS design methodology should provide adequate means to model variability in order to support the opportunistic selection of the most desirable SoS configuration. We propose the VANTESS approach that (i) supports SoS modeling taking into account the variation points implied by the considered constituent systems; (ii) includes a heuristics to weight benefits and costs of potential architectural choices (called as SoS variants) for the selection of the constituent systems; and finally (iii) also helps test planning for the selected SoS variant by deriving a simulation model on which test objectives and scenarios can be devised. We illustrate an application example of VANTESS to the "educational" SoS and discuss its pros and cons within a focus group.Source: JOURNAL OF SOFTWARE (MALDEN, MASS. ONLINE)
DOI: 10.1002/smr.2427
Metrics:
Lonetti F, De Oliveira Neves V, Bertolino A
In the early stages of a system of systems (SoS) conception, several constituent systems could be available that provide similar functionalities. An SoS design methodology should provide adequate means to model variability in order to support the opportunistic selection of the most desirable SoS configuration. We propose the VANTESS approach that (i) supports SoS modeling taking into account the variation points implied by the considered constituent systems; (ii) includes a heuristics to weight benefits and costs of potential architectural choices (called as SoS variants) for the selection of the constituent systems; and finally (iii) also helps test planning for the selected SoS variant by deriving a simulation model on which test objectives and scenarios can be devised. We illustrate an application example of VANTESS to the "educational" SoS and discuss its pros and cons within a focus group.Source: JOURNAL OF SOFTWARE (MALDEN, MASS. ONLINE)
DOI: 10.1002/smr.2427
Metrics:
See at:
CNR IRIS | onlinelibrary.wiley.com | ISTI Repository | CNR IRIS | CNR IRIS
2022
Journal article
Open Access
A formal validation approach for XACML 3.0 access control policy
Caserio C, Lonetti F, Marchetti E
Access control systems represent a security mechanism to regulate the access to system resources, and XACML is the standard language for specifying, storing and deploying access control policies. The verbosity and complexity of XACML syntax as well as the natural language semantics provided by the standard make the verification and testing of these policies difficult and error-prone. In the literature, analysis techniques and access control languages formalizations are provided for verifiability and testability purposes. This paper provides three contributions: it provides a comprehensive formal specification of XACML 3.0 policy elements; it leverages the existing policy coverage criteria to be suitable for XACML 3.0; and it introduces a new set of coverage criteria to better focus the testing activities on the peculiarities of XACML 3.0. The application of the proposed coverage criteria to a policy example is described, and hints for future research directions are discussed.Source: SENSORS (BASEL), vol. 22 (issue 8)
DOI: 10.3390/s22082984
Project(s): BIECO
, CyberSec4Europe
Metrics:
Caserio C, Lonetti F, Marchetti E
Access control systems represent a security mechanism to regulate the access to system resources, and XACML is the standard language for specifying, storing and deploying access control policies. The verbosity and complexity of XACML syntax as well as the natural language semantics provided by the standard make the verification and testing of these policies difficult and error-prone. In the literature, analysis techniques and access control languages formalizations are provided for verifiability and testability purposes. This paper provides three contributions: it provides a comprehensive formal specification of XACML 3.0 policy elements; it leverages the existing policy coverage criteria to be suitable for XACML 3.0; and it introduces a new set of coverage criteria to better focus the testing activities on the peculiarities of XACML 3.0. The application of the proposed coverage criteria to a policy example is described, and hints for future research directions are discussed.Source: SENSORS (BASEL), vol. 22 (issue 8)
DOI: 10.3390/s22082984
Project(s): BIECO
, CyberSec4Europe Metrics:
See at:
Sensors | CNR IRIS | ISTI Repository | www.mdpi.com | CNR IRIS
2021
Conference article
Open Access
About the assessment of Grey Literature in Software Engineering
De Angelis G, Lonetti F
There is an ongoing interest in the Software Engineering field for multivocal literature reviews including grey literature. However, at the same time, the role of the grey literature is still controversial, and the benefits of its inclusion in systematic reviews are object of discussion. Some of these arguments concern the quality assessment methods for grey literature entries, which is often considered a challenging and critical task. On the one hand, apart from a few proposals, there is a lack of an acknowledged methodological support for the inclusion of Software Engineering grey literature in systematic surveys. On the other hand, the unstructured shape of the grey literature contents could lead to bias in the evaluation process impacting on the quality of the surveys. This work leverages an approach on fuzzy Likert scales, and it proposes a methodology for managing the explicit uncertainties emerging during the assessment of entries from the grey literature. The methodology also strengthens the adoption of consensus policies that take into account the individual confidence level expressed for each of the collected scores.DOI: 10.1145/3463274.3463362
Metrics:
De Angelis G, Lonetti F
There is an ongoing interest in the Software Engineering field for multivocal literature reviews including grey literature. However, at the same time, the role of the grey literature is still controversial, and the benefits of its inclusion in systematic reviews are object of discussion. Some of these arguments concern the quality assessment methods for grey literature entries, which is often considered a challenging and critical task. On the one hand, apart from a few proposals, there is a lack of an acknowledged methodological support for the inclusion of Software Engineering grey literature in systematic surveys. On the other hand, the unstructured shape of the grey literature contents could lead to bias in the evaluation process impacting on the quality of the surveys. This work leverages an approach on fuzzy Likert scales, and it proposes a methodology for managing the explicit uncertainties emerging during the assessment of entries from the grey literature. The methodology also strengthens the adoption of consensus policies that take into account the individual confidence level expressed for each of the collected scores.DOI: 10.1145/3463274.3463362
Metrics:
See at:
dl.acm.org | CNR IRIS | ISTI Repository | CNR IRIS | CNR IRIS
2020
Conference article
Restricted
A Framework for the Validation of Access Control Systems
Daoudagh S, Lonetti F, Marchetti E
In modern pervasive applications, it is important to validate Access Control (AC) mechanisms that are usually defined by means of the XACML standard. Mutation analysis has been applied on Access Control Policies (ACPs) for measuring the adequacy of a test suite. This paper provides an automatic framework for realizing mutations of the code of the Policy Decision Point (PDP) that is a critical component in AC systems. The proposed framework allows the test strategies assessment and the analysis of test data by leveraging mutation-based approaches. We show how to instantiate the proposed framework and provide also some examples of its application.DOI: 10.1007/978-3-030-39749-4_3
Project(s): CyberSec4Europe
Metrics:
Daoudagh S, Lonetti F, Marchetti E
In modern pervasive applications, it is important to validate Access Control (AC) mechanisms that are usually defined by means of the XACML standard. Mutation analysis has been applied on Access Control Policies (ACPs) for measuring the adequacy of a test suite. This paper provides an automatic framework for realizing mutations of the code of the Policy Decision Point (PDP) that is a critical component in AC systems. The proposed framework allows the test strategies assessment and the analysis of test data by leveraging mutation-based approaches. We show how to instantiate the proposed framework and provide also some examples of its application.DOI: 10.1007/978-3-030-39749-4_3
Project(s): CyberSec4Europe
Metrics:
See at:
Lecture Notes in Computer Science | CNR IRIS | CNR IRIS | link.springer.com
2020
Conference article
Open Access
Assessing testing strategies for access control systems: a controlled experiment
Daoudagh S, Lonetti F, Marchetti E
This paper presents a Controlled Experiment (CE) for assessing testing strategies in the context of Access Control (AC); more precisely, the CE is performed by considering the AC Systems (ACSs) based on the XACML Standard. We formalized the goal of the CE, and we assessed two available test cases generation strategies in terms of three metrics: Effectiveness, Size and Average Percentage Faults Detected (APFD). The experiment operation is described and the main results are analyzed.DOI: 10.5220/0008974201070118
Project(s): CyberSec4Europe
Metrics:
Daoudagh S, Lonetti F, Marchetti E
This paper presents a Controlled Experiment (CE) for assessing testing strategies in the context of Access Control (AC); more precisely, the CE is performed by considering the AC Systems (ACSs) based on the XACML Standard. We formalized the goal of the CE, and we assessed two available test cases generation strategies in terms of three metrics: Effectiveness, Size and Average Percentage Faults Detected (APFD). The experiment operation is described and the main results are analyzed.DOI: 10.5220/0008974201070118
Project(s): CyberSec4Europe
Metrics:
See at:
doi.org | CNR IRIS | ISTI Repository | www.scitepress.org | CNR IRIS | www.scopus.com
2020
Journal article
Open Access
XACMET: XACML testing & modeling: an automated model-based testing solution for access control systems
Daoudagh S, Lonetti F, Marchetti E
In the context of access control systems, testing activity is among the most adopted means to assure that sensible information or resources are correctly accessed. In XACML-based access control systems, incoming access requests are transmitted to the policy decision point (PDP) that grants or denies the access based on the defined XACML policies. The criticality of a PDP component requires an intensive testing activity consisting in probing such a component with a set of requests and checking whether its responses grant or deny the requested access as specified in the policy. Existing approaches for improving manual derivation of test requests such as combinatorial ones do not consider policy function semantics and do not provide a verdict oracle. In this paper, we introduce XACMET, a novel approach for systematic generation of XACML requests as well as automated model-based oracle derivation. The main features of XACMET are as follows: (i) it defines a typed graph, called the XAC-Graph, that models the XACML policy evaluation; (ii) it derives a set of test requests via full-path coverage of this graph; (iii) it derives automatically the expected verdict of a specific request execution by executing the corresponding path in such graph; (iv) it allows us to measure coverage assessment of a given test suite. Our validation of the XACMET prototype implementation confirms the effectiveness of the proposed approach.Source: SOFTWARE QUALITY JOURNAL, vol. 28 (issue 1), pp. 249-282
DOI: 10.1007/s11219-019-09470-5
Project(s): CyberSec4Europe
Metrics:
Daoudagh S, Lonetti F, Marchetti E
In the context of access control systems, testing activity is among the most adopted means to assure that sensible information or resources are correctly accessed. In XACML-based access control systems, incoming access requests are transmitted to the policy decision point (PDP) that grants or denies the access based on the defined XACML policies. The criticality of a PDP component requires an intensive testing activity consisting in probing such a component with a set of requests and checking whether its responses grant or deny the requested access as specified in the policy. Existing approaches for improving manual derivation of test requests such as combinatorial ones do not consider policy function semantics and do not provide a verdict oracle. In this paper, we introduce XACMET, a novel approach for systematic generation of XACML requests as well as automated model-based oracle derivation. The main features of XACMET are as follows: (i) it defines a typed graph, called the XAC-Graph, that models the XACML policy evaluation; (ii) it derives a set of test requests via full-path coverage of this graph; (iii) it derives automatically the expected verdict of a specific request execution by executing the corresponding path in such graph; (iv) it allows us to measure coverage assessment of a given test suite. Our validation of the XACMET prototype implementation confirms the effectiveness of the proposed approach.Source: SOFTWARE QUALITY JOURNAL, vol. 28 (issue 1), pp. 249-282
DOI: 10.1007/s11219-019-09470-5
Project(s): CyberSec4Europe
Metrics:
See at:
CNR IRIS | link.springer.com | ISTI Repository | Software Quality Journal | CNR IRIS | CNR IRIS
2020
Conference article
Open Access
EDUFYSoS: a factory of educational system of systems case studies
Bertolino A, De Angelis G, Lonetti F, De Oliveira Neves V, Olivero Ma
We propose a factory of educational System of Systems (SoS) case studies that can be used for evaluating SoS research results, in particular in SoS testing. The factory includes a first set of constituent systems that can collaborate within different SoS architectures to accomplish different missions. In the paper, we introduce three possible SoSs and outline their missions. For more detailed descriptions, diagrams and the source code, we refer to the online repository of EDUFYSoS. The factory is meant to provide an extensible playground, which we aim to grow to include more systems and other missions with the support of the community.DOI: 10.1109/sose50414.2020.9130551
Metrics:
Bertolino A, De Angelis G, Lonetti F, De Oliveira Neves V, Olivero Ma
We propose a factory of educational System of Systems (SoS) case studies that can be used for evaluating SoS research results, in particular in SoS testing. The factory includes a first set of constituent systems that can collaborate within different SoS architectures to accomplish different missions. In the paper, we introduce three possible SoSs and outline their missions. For more detailed descriptions, diagrams and the source code, we refer to the online repository of EDUFYSoS. The factory is meant to provide an extensible playground, which we aim to grow to include more systems and other missions with the support of the community.DOI: 10.1109/sose50414.2020.9130551
Metrics:
See at:
CNR IRIS | idUS. Depósito de Investigación Universidad de Sevilla | ieeexplore.ieee.org | ISTI Repository | doi.org | CNR IRIS | CNR IRIS
2020
Conference article
Open Access
Quality-of-Experience driven configuration of WebRTC services through automated testing
Bertolino A., Calabrò A., De Angelis G., Gortázar F., Lonetti F., Maes M., Tuñón G.
Quality of Experience (QoE) refers to the end users level of satisfaction with a real-time service, in particular in relation to its audio and video quality. Advances in WebRTC technology have favored the spread of multimedia services through use of any browser. Provision of adequate QoE in such services is of paramount importance. The assessment of QoE is costly and can be done only late in the service lifecycle. In this work we propose a simple approach for QoE-driven non-functional testing of WebRTC services that relies on the ElasTest open-source platform for end-to-end testing of large complex systems. We describe the ElasTest platform, the proposed approach and an experimental study. In this study, we compared qualitatively and quantitatively the effort required in the ElasTest supported scenario with respect to a "traditional" solution, showing great savings in terms of effort and time.DOI: 10.1109/qrs51102.2020.00031
Project(s): ELASTEST
Metrics:
Bertolino A., Calabrò A., De Angelis G., Gortázar F., Lonetti F., Maes M., Tuñón G.
Quality of Experience (QoE) refers to the end users level of satisfaction with a real-time service, in particular in relation to its audio and video quality. Advances in WebRTC technology have favored the spread of multimedia services through use of any browser. Provision of adequate QoE in such services is of paramount importance. The assessment of QoE is costly and can be done only late in the service lifecycle. In this work we propose a simple approach for QoE-driven non-functional testing of WebRTC services that relies on the ElasTest open-source platform for end-to-end testing of large complex systems. We describe the ElasTest platform, the proposed approach and an experimental study. In this study, we compared qualitatively and quantitatively the effort required in the ElasTest supported scenario with respect to a "traditional" solution, showing great savings in terms of effort and time.DOI: 10.1109/qrs51102.2020.00031
Project(s): ELASTEST
Metrics:
See at:
CNR IRIS | ISTI Repository | qrs20.techconf.org | doi.org | CNR IRIS | CNR IRIS
2020
Conference article
Open Access
Standing on the Shoulders of Software Product Line Research for Testing Systems of Systems
Bertolino A, Lonetti F, De Oliveira Neves V
The complex and dynamic nature of Systems of Systems (SoSs) poses many challenges on their validation and testing, but so far few effective test strategies exist to address them. On the other hand, extensive research has been conducted in the testing of Software Product Lines (SPLs), which present interesting convergence points with SoSs, as both disciplines aim at reducing development costs and time-to-market thanks to extensive reuse of existing artifacts. In this paper, we outline commonalities and differences between the SoS and SPL paradigms from the point of view of testing and investigate how existing methods and tools from SPL testing could be leveraged to address the challenges of SoS testing.DOI: 10.1109/issrew51248.2020.00074
Metrics:
Bertolino A, Lonetti F, De Oliveira Neves V
The complex and dynamic nature of Systems of Systems (SoSs) poses many challenges on their validation and testing, but so far few effective test strategies exist to address them. On the other hand, extensive research has been conducted in the testing of Software Product Lines (SPLs), which present interesting convergence points with SoSs, as both disciplines aim at reducing development costs and time-to-market thanks to extensive reuse of existing artifacts. In this paper, we outline commonalities and differences between the SoS and SPL paradigms from the point of view of testing and investigate how existing methods and tools from SPL testing could be leveraged to address the challenges of SoS testing.DOI: 10.1109/issrew51248.2020.00074
Metrics:
See at:
CNR IRIS | ieeexplore.ieee.org | ISTI Repository | doi.org | CNR IRIS | CNR IRIS
2020
Conference article
Restricted
Continuous Development and Testing of Access and Usage Control: A Systematic Literature Review
Daoudagh S, Lonetti F, Marchetti E
Context: Development and testing of access/usage control systems is a growing research area. With new trends in software development such as DevOps, the development of access/usage control also has to evolve. Objective: The main aim of this paper is to provide an overview of research proposals in the area of continuous development and testing of access and usage control systems. Method: The paper uses a Systematic Literature Review as a research method to define the research questions and answer them following a systematic approach. With the specified search string, 210 studies were retrieved. After applying the inclusion and exclusion criteria in two phases, a final set of 20 primary studies was selected for this review. Results: Results show that primary studies are mostly published in security venues followed by software engineering venues. Furthermore, most of the studies are based on the standard XACML access control language. In addition, a significant portion of the proposals for development and testing is automated with test assessment and generation the most targeted areas. Some general guidelines for leveraging continuous developing and testing of the usage and access control systems inside the DevOps process are also provided.DOI: 10.1145/3393822.3432330
Project(s): CyberSec4Europe
Metrics:
Daoudagh S, Lonetti F, Marchetti E
Context: Development and testing of access/usage control systems is a growing research area. With new trends in software development such as DevOps, the development of access/usage control also has to evolve. Objective: The main aim of this paper is to provide an overview of research proposals in the area of continuous development and testing of access and usage control systems. Method: The paper uses a Systematic Literature Review as a research method to define the research questions and answer them following a systematic approach. With the specified search string, 210 studies were retrieved. After applying the inclusion and exclusion criteria in two phases, a final set of 20 primary studies was selected for this review. Results: Results show that primary studies are mostly published in security venues followed by software engineering venues. Furthermore, most of the studies are based on the standard XACML access control language. In addition, a significant portion of the proposals for development and testing is automated with test assessment and generation the most targeted areas. Some general guidelines for leveraging continuous developing and testing of the usage and access control systems inside the DevOps process are also provided.DOI: 10.1145/3393822.3432330
Project(s): CyberSec4Europe
Metrics:
See at:
dl.acm.org | doi.org | CNR IRIS | CNR IRIS
2020
Journal article
Restricted
An automated framework for continuous development and testing of access control systems
Daoudagh S, Lonetti F, Marchetti E
Automated testing in DevOps represents a key factor for providing fast release of new software features assuring quality delivery. In this paper, we introduce DOXAT, an automated framework for continuous development and testing of access control mechanisms based on the XACML standard. It leverages mutation analysis for the selection and assessment of the test strategies and provides automated facilities for test oracle definition, test execution, and results analysis, in order to speedup and automate the Plan, Code, Build, and Test phases of DevOps process. We show the usage of the framework during the planning and testing phases of the software development cycle of a PDP example.Source: JOURNAL OF SOFTWARE (MALDEN, MASS. ONLINE)
DOI: 10.1002/smr.2306
Project(s): CyberSec4Europe
Metrics:
Daoudagh S, Lonetti F, Marchetti E
Automated testing in DevOps represents a key factor for providing fast release of new software features assuring quality delivery. In this paper, we introduce DOXAT, an automated framework for continuous development and testing of access control mechanisms based on the XACML standard. It leverages mutation analysis for the selection and assessment of the test strategies and provides automated facilities for test oracle definition, test execution, and results analysis, in order to speedup and automate the Plan, Code, Build, and Test phases of DevOps process. We show the usage of the framework during the planning and testing phases of the software development cycle of a PDP example.Source: JOURNAL OF SOFTWARE (MALDEN, MASS. ONLINE)
DOI: 10.1002/smr.2306
Project(s): CyberSec4Europe
Metrics:
See at:
Journal of Software Evolution and Process | CNR IRIS | CNR IRIS | onlinelibrary.wiley.com