2022
Journal article
Open Access
Designing and testing systems of systems: from variability models to test cases passing through desirability assessment
Lonetti F., De Oliveira Neves V., Bertolino A.
In the early stages of a system of systems (SoS) conception, several constituent systems could be available that provide similar functionalities. An SoS design methodology should provide adequate means to model variability in order to support the opportunistic selection of the most desirable SoS configuration. We propose the VANTESS approach that (i) supports SoS modeling taking into account the variation points implied by the considered constituent systems; (ii) includes a heuristics to weight benefits and costs of potential architectural choices (called as SoS variants) for the selection of the constituent systems; and finally (iii) also helps test planning for the selected SoS variant by deriving a simulation model on which test objectives and scenarios can be devised. We illustrate an application example of VANTESS to the "educational" SoS and discuss its pros and cons within a focus group.Source: Journal of software (Malden, Mass. Online) (2022). doi:10.1002/smr.2427
DOI: 10.1002/smr.2427
Metrics:
Lonetti F., De Oliveira Neves V., Bertolino A.
In the early stages of a system of systems (SoS) conception, several constituent systems could be available that provide similar functionalities. An SoS design methodology should provide adequate means to model variability in order to support the opportunistic selection of the most desirable SoS configuration. We propose the VANTESS approach that (i) supports SoS modeling taking into account the variation points implied by the considered constituent systems; (ii) includes a heuristics to weight benefits and costs of potential architectural choices (called as SoS variants) for the selection of the constituent systems; and finally (iii) also helps test planning for the selected SoS variant by deriving a simulation model on which test objectives and scenarios can be devised. We illustrate an application example of VANTESS to the "educational" SoS and discuss its pros and cons within a focus group.Source: Journal of software (Malden, Mass. Online) (2022). doi:10.1002/smr.2427
DOI: 10.1002/smr.2427
Metrics:
See at:
ISTI Repository 
| onlinelibrary.wiley.com 
| CNR ExploRA
2022
Journal article
Open Access
A formal validation approach for XACML 3.0 access control policy
Caserio C., Lonetti F., Marchetti E.
Access control systems represent a security mechanism to regulate the access to system resources, and XACML is the standard language for specifying, storing and deploying access control policies. The verbosity and complexity of XACML syntax as well as the natural language semantics provided by the standard make the verification and testing of these policies difficult and error-prone. In the literature, analysis techniques and access control languages formalizations are provided for verifiability and testability purposes. This paper provides three contributions: it provides a comprehensive formal specification of XACML 3.0 policy elements; it leverages the existing policy coverage criteria to be suitable for XACML 3.0; and it introduces a new set of coverage criteria to better focus the testing activities on the peculiarities of XACML 3.0. The application of the proposed coverage criteria to a policy example is described, and hints for future research directions are discussed.Source: Sensors (Basel) 22 (2022). doi:10.3390/s22082984
DOI: 10.3390/s22082984
Project(s): BIECO
, CyberSec4Europe
Metrics:
Caserio C., Lonetti F., Marchetti E.
Access control systems represent a security mechanism to regulate the access to system resources, and XACML is the standard language for specifying, storing and deploying access control policies. The verbosity and complexity of XACML syntax as well as the natural language semantics provided by the standard make the verification and testing of these policies difficult and error-prone. In the literature, analysis techniques and access control languages formalizations are provided for verifiability and testability purposes. This paper provides three contributions: it provides a comprehensive formal specification of XACML 3.0 policy elements; it leverages the existing policy coverage criteria to be suitable for XACML 3.0; and it introduces a new set of coverage criteria to better focus the testing activities on the peculiarities of XACML 3.0. The application of the proposed coverage criteria to a policy example is described, and hints for future research directions are discussed.Source: Sensors (Basel) 22 (2022). doi:10.3390/s22082984
DOI: 10.3390/s22082984
Project(s): BIECO
, CyberSec4Europe Metrics:
See at:
Sensors | ISTI Repository | CNR ExploRA | www.mdpi.com
2021
Conference article
Open Access
About the assessment of Grey Literature in Software Engineering
De Angelis G., Lonetti F.
There is an ongoing interest in the Software Engineering field for multivocal literature reviews including grey literature. However, at the same time, the role of the grey literature is still controversial, and the benefits of its inclusion in systematic reviews are object of discussion. Some of these arguments concern the quality assessment methods for grey literature entries, which is often considered a challenging and critical task. On the one hand, apart from a few proposals, there is a lack of an acknowledged methodological support for the inclusion of Software Engineering grey literature in systematic surveys. On the other hand, the unstructured shape of the grey literature contents could lead to bias in the evaluation process impacting on the quality of the surveys. This work leverages an approach on fuzzy Likert scales, and it proposes a methodology for managing the explicit uncertainties emerging during the assessment of entries from the grey literature. The methodology also strengthens the adoption of consensus policies that take into account the individual confidence level expressed for each of the collected scores.Source: PROPSER 2021 - International Workshop on Properties of Software Engineering Research, co-located with EASE 2021 - Evaluation and Assessment in Software Engineering, pp. 373–378, Trondheim, Norway and Online, 23/06/2021
DOI: 10.1145/3463274.3463362
Metrics:
De Angelis G., Lonetti F.
There is an ongoing interest in the Software Engineering field for multivocal literature reviews including grey literature. However, at the same time, the role of the grey literature is still controversial, and the benefits of its inclusion in systematic reviews are object of discussion. Some of these arguments concern the quality assessment methods for grey literature entries, which is often considered a challenging and critical task. On the one hand, apart from a few proposals, there is a lack of an acknowledged methodological support for the inclusion of Software Engineering grey literature in systematic surveys. On the other hand, the unstructured shape of the grey literature contents could lead to bias in the evaluation process impacting on the quality of the surveys. This work leverages an approach on fuzzy Likert scales, and it proposes a methodology for managing the explicit uncertainties emerging during the assessment of entries from the grey literature. The methodology also strengthens the adoption of consensus policies that take into account the individual confidence level expressed for each of the collected scores.Source: PROPSER 2021 - International Workshop on Properties of Software Engineering Research, co-located with EASE 2021 - Evaluation and Assessment in Software Engineering, pp. 373–378, Trondheim, Norway and Online, 23/06/2021
DOI: 10.1145/3463274.3463362
Metrics:
See at:
ISTI Repository | dl.acm.org | CNR ExploRA
2020
Conference article
Restricted
A Framework for the Validation of Access Control Systems
Daoudagh S., Lonetti F., Marchetti E.
In modern pervasive applications, it is important to validate Access Control (AC) mechanisms that are usually defined by means of the XACML standard. Mutation analysis has been applied on Access Control Policies (ACPs) for measuring the adequacy of a test suite. This paper provides an automatic framework for realizing mutations of the code of the Policy Decision Point (PDP) that is a critical component in AC systems. The proposed framework allows the test strategies assessment and the analysis of test data by leveraging mutation-based approaches. We show how to instantiate the proposed framework and provide also some examples of its application.Source: Emerging Technologies for Authorization and Authentication. ETAA 2019, pp. 35–51, Luxembourg City, Luxembourg, 27/09/2019
DOI: 10.1007/978-3-030-39749-4_3
Project(s): CyberSec4Europe
Metrics:
Daoudagh S., Lonetti F., Marchetti E.
In modern pervasive applications, it is important to validate Access Control (AC) mechanisms that are usually defined by means of the XACML standard. Mutation analysis has been applied on Access Control Policies (ACPs) for measuring the adequacy of a test suite. This paper provides an automatic framework for realizing mutations of the code of the Policy Decision Point (PDP) that is a critical component in AC systems. The proposed framework allows the test strategies assessment and the analysis of test data by leveraging mutation-based approaches. We show how to instantiate the proposed framework and provide also some examples of its application.Source: Emerging Technologies for Authorization and Authentication. ETAA 2019, pp. 35–51, Luxembourg City, Luxembourg, 27/09/2019
DOI: 10.1007/978-3-030-39749-4_3
Project(s): CyberSec4Europe
Metrics:
See at:
Lecture Notes in Computer Science | link.springer.com | CNR ExploRA
2020
Conference article
Open Access
Assessing testing strategies for access control systems: a controlled experiment
Daoudagh S., Lonetti F., Marchetti E.
This paper presents a Controlled Experiment (CE) for assessing testing strategies in the context of Access Control (AC); more precisely, the CE is performed by considering the AC Systems (ACSs) based on the XACML Standard. We formalized the goal of the CE, and we assessed two available test cases generation strategies in terms of three metrics: Effectiveness, Size and Average Percentage Faults Detected (APFD). The experiment operation is described and the main results are analyzed.Source: 6th International Conference on Information Systems Security and Privacy, pp. 107–118, Valletta, Malta, 25-27/02/2020
DOI: 10.5220/0008974201070118
Project(s): CyberSec4Europe
Metrics:
Daoudagh S., Lonetti F., Marchetti E.
This paper presents a Controlled Experiment (CE) for assessing testing strategies in the context of Access Control (AC); more precisely, the CE is performed by considering the AC Systems (ACSs) based on the XACML Standard. We formalized the goal of the CE, and we assessed two available test cases generation strategies in terms of three metrics: Effectiveness, Size and Average Percentage Faults Detected (APFD). The experiment operation is described and the main results are analyzed.Source: 6th International Conference on Information Systems Security and Privacy, pp. 107–118, Valletta, Malta, 25-27/02/2020
DOI: 10.5220/0008974201070118
Project(s): CyberSec4Europe
Metrics:
See at:
doi.org | ISTI Repository | CNR ExploRA | www.scitepress.org | www.scopus.com
2020
Journal article
Open Access
XACMET: XACML Testing & Modeling: An automated model-based testing solution for access control systems
Daoudagh S., Lonetti F., Marchetti E.
In the context of access control systems, testing activity is among the most adopted means to assure that sensible information or resources are correctly accessed. In XACML-based access control systems, incoming access requests are transmitted to the policy decision point (PDP) that grants or denies the access based on the defined XACML policies. The criticality of a PDP component requires an intensive testing activity consisting in probing such a component with a set of requests and checking whether its responses grant or deny the requested access as specified in the policy. Existing approaches for improving manual derivation of test requests such as combinatorial ones do not consider policy function semantics and do not provide a verdict oracle. In this paper, we introduce XACMET, a novel approach for systematic generation of XACML requests as well as automated model-based oracle derivation. The main features of XACMET are as follows: (i) it defines a typed graph, called the XAC-Graph, that models the XACML policy evaluation; (ii) it derives a set of test requests via full-path coverage of this graph; (iii) it derives automatically the expected verdict of a specific request execution by executing the corresponding path in such graph; (iv) it allows us to measure coverage assessment of a given test suite. Our validation of the XACMET prototype implementation confirms the effectiveness of the proposed approach.Source: Software quality journal 28 (2020): 249–282. doi:10.1007/s11219-019-09470-5
DOI: 10.1007/s11219-019-09470-5
Metrics:
Daoudagh S., Lonetti F., Marchetti E.
In the context of access control systems, testing activity is among the most adopted means to assure that sensible information or resources are correctly accessed. In XACML-based access control systems, incoming access requests are transmitted to the policy decision point (PDP) that grants or denies the access based on the defined XACML policies. The criticality of a PDP component requires an intensive testing activity consisting in probing such a component with a set of requests and checking whether its responses grant or deny the requested access as specified in the policy. Existing approaches for improving manual derivation of test requests such as combinatorial ones do not consider policy function semantics and do not provide a verdict oracle. In this paper, we introduce XACMET, a novel approach for systematic generation of XACML requests as well as automated model-based oracle derivation. The main features of XACMET are as follows: (i) it defines a typed graph, called the XAC-Graph, that models the XACML policy evaluation; (ii) it derives a set of test requests via full-path coverage of this graph; (iii) it derives automatically the expected verdict of a specific request execution by executing the corresponding path in such graph; (iv) it allows us to measure coverage assessment of a given test suite. Our validation of the XACMET prototype implementation confirms the effectiveness of the proposed approach.Source: Software quality journal 28 (2020): 249–282. doi:10.1007/s11219-019-09470-5
DOI: 10.1007/s11219-019-09470-5
Metrics:
See at:
ISTI Repository | Software Quality Journal | link.springer.com | CNR ExploRA
2020
Conference article
Open Access
EDUFYSoS: A Factory of Educational System of Systems Case Studies
Bertolino A., De Angelis G., Lonetti F., De Oliveira Neves V., Olivero M. A.
We propose a factory of educational System of Systems (SoS) case studies that can be used for evaluating SoS research results, in particular in SoS testing. The factory includes a first set of constituent systems that can collaborate within different SoS architectures to accomplish different missions. In the paper, we introduce three possible SoSs and outline their missions. For more detailed descriptions, diagrams and the source code, we refer to the online repository of EDUFYSoS. The factory is meant to provide an extensible playground, which we aim to grow to include more systems and other missions with the support of the community.Source: IEEE 15th Int. Conf. of System of Systems Engineering (SoSE), Budapest, Ungheria, 2-5/06/2020
DOI: 10.1109/sose50414.2020.9130551
Metrics:
Bertolino A., De Angelis G., Lonetti F., De Oliveira Neves V., Olivero M. A.
We propose a factory of educational System of Systems (SoS) case studies that can be used for evaluating SoS research results, in particular in SoS testing. The factory includes a first set of constituent systems that can collaborate within different SoS architectures to accomplish different missions. In the paper, we introduce three possible SoSs and outline their missions. For more detailed descriptions, diagrams and the source code, we refer to the online repository of EDUFYSoS. The factory is meant to provide an extensible playground, which we aim to grow to include more systems and other missions with the support of the community.Source: IEEE 15th Int. Conf. of System of Systems Engineering (SoSE), Budapest, Ungheria, 2-5/06/2020
DOI: 10.1109/sose50414.2020.9130551
Metrics:
See at:
idUS. Depósito de Investigación Universidad de Sevilla | ieeexplore.ieee.org | ISTI Repository | CNR ExploRA | doi.org
2020
Conference article
Open Access
Quality-of-Experience driven configuration of WebRTC services through automated testing
Bertolino A., Calabró A., De Angelis G., Gortázar F., Lonetti F., Maes M., Tuñón G.
Quality of Experience (QoE) refers to the end users level of satisfaction with a real-time service, in particular in relation to its audio and video quality. Advances in WebRTC technology have favored the spread of multimedia services through use of any browser. Provision of adequate QoE in such services is of paramount importance. The assessment of QoE is costly and can be done only late in the service lifecycle. In this work we propose a simple approach for QoE-driven non-functional testing of WebRTC services that relies on the ElasTest open-source platform for end-to-end testing of large complex systems. We describe the ElasTest platform, the proposed approach and an experimental study. In this study, we compared qualitatively and quantitatively the effort required in the ElasTest supported scenario with respect to a "traditional" solution, showing great savings in terms of effort and time.Source: IEEE 20th International Conference on Software Quality, Reliability, and Security (QRS), pp. 152–159, Macau, China, 11-14/12/2020
DOI: 10.1109/qrs51102.2020.00031
Project(s): ELASTEST
Metrics:
Bertolino A., Calabró A., De Angelis G., Gortázar F., Lonetti F., Maes M., Tuñón G.
Quality of Experience (QoE) refers to the end users level of satisfaction with a real-time service, in particular in relation to its audio and video quality. Advances in WebRTC technology have favored the spread of multimedia services through use of any browser. Provision of adequate QoE in such services is of paramount importance. The assessment of QoE is costly and can be done only late in the service lifecycle. In this work we propose a simple approach for QoE-driven non-functional testing of WebRTC services that relies on the ElasTest open-source platform for end-to-end testing of large complex systems. We describe the ElasTest platform, the proposed approach and an experimental study. In this study, we compared qualitatively and quantitatively the effort required in the ElasTest supported scenario with respect to a "traditional" solution, showing great savings in terms of effort and time.Source: IEEE 20th International Conference on Software Quality, Reliability, and Security (QRS), pp. 152–159, Macau, China, 11-14/12/2020
DOI: 10.1109/qrs51102.2020.00031
Project(s): ELASTEST
Metrics:
See at:
ISTI Repository | CNR ExploRA | qrs20.techconf.org | doi.org
2020
Conference article
Open Access
Standing on the Shoulders of Software Product Line Research for Testing Systems of Systems
Bertolino A., Lonetti F., De Oliveira Neves V.
The complex and dynamic nature of Systems of Systems (SoSs) poses many challenges on their validation and testing, but so far few effective test strategies exist to address them. On the other hand, extensive research has been conducted in the testing of Software Product Lines (SPLs), which present interesting convergence points with SoSs, as both disciplines aim at reducing development costs and time-to-market thanks to extensive reuse of existing artifacts. In this paper, we outline commonalities and differences between the SoS and SPL paradigms from the point of view of testing and investigate how existing methods and tools from SPL testing could be leveraged to address the challenges of SoS testing.Source: 2020 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), pp. 209–214, Coimbra, Portugal, 12/10/2020
DOI: 10.1109/issrew51248.2020.00074
Metrics:
Bertolino A., Lonetti F., De Oliveira Neves V.
The complex and dynamic nature of Systems of Systems (SoSs) poses many challenges on their validation and testing, but so far few effective test strategies exist to address them. On the other hand, extensive research has been conducted in the testing of Software Product Lines (SPLs), which present interesting convergence points with SoSs, as both disciplines aim at reducing development costs and time-to-market thanks to extensive reuse of existing artifacts. In this paper, we outline commonalities and differences between the SoS and SPL paradigms from the point of view of testing and investigate how existing methods and tools from SPL testing could be leveraged to address the challenges of SoS testing.Source: 2020 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), pp. 209–214, Coimbra, Portugal, 12/10/2020
DOI: 10.1109/issrew51248.2020.00074
Metrics:
See at:
ISTI Repository | doi.org | ieeexplore.ieee.org | CNR ExploRA
2020
Conference article
Closed Access
Continuous Development and Testing of Access and Usage Control: A Systematic Literature Review
Daoudagh S., Lonetti F., Marchetti E.
Context: Development and testing of access/usage control systems is a growing research area. With new trends in software development such as DevOps, the development of access/usage control also has to evolve. Objective: The main aim of this paper is to provide an overview of research proposals in the area of continuous development and testing of access and usage control systems. Method: The paper uses a Systematic Literature Review as a research method to define the research questions and answer them following a systematic approach. With the specified search string, 210 studies were retrieved. After applying the inclusion and exclusion criteria in two phases, a final set of 20 primary studies was selected for this review. Results: Results show that primary studies are mostly published in security venues followed by software engineering venues. Furthermore, most of the studies are based on the standard XACML access control language. In addition, a significant portion of the proposals for development and testing is automated with test assessment and generation the most targeted areas. Some general guidelines for leveraging continuous developing and testing of the usage and access control systems inside the DevOps process are also provided.Source: 2020 European Symposium on Software Engineering, pp. 51–59, Rome, Italy, 06-08/11/2020
DOI: 10.1145/3393822.3432330
Project(s): CyberSec4Europe
Metrics:
Daoudagh S., Lonetti F., Marchetti E.
Context: Development and testing of access/usage control systems is a growing research area. With new trends in software development such as DevOps, the development of access/usage control also has to evolve. Objective: The main aim of this paper is to provide an overview of research proposals in the area of continuous development and testing of access and usage control systems. Method: The paper uses a Systematic Literature Review as a research method to define the research questions and answer them following a systematic approach. With the specified search string, 210 studies were retrieved. After applying the inclusion and exclusion criteria in two phases, a final set of 20 primary studies was selected for this review. Results: Results show that primary studies are mostly published in security venues followed by software engineering venues. Furthermore, most of the studies are based on the standard XACML access control language. In addition, a significant portion of the proposals for development and testing is automated with test assessment and generation the most targeted areas. Some general guidelines for leveraging continuous developing and testing of the usage and access control systems inside the DevOps process are also provided.Source: 2020 European Symposium on Software Engineering, pp. 51–59, Rome, Italy, 06-08/11/2020
DOI: 10.1145/3393822.3432330
Project(s): CyberSec4Europe
Metrics:
See at:
dl.acm.org | doi.org | CNR ExploRA
2020
Journal article
Embargo
An automated framework for continuous development and testing of access control systems
Daoudagh S., Lonetti F., Marchetti E.
Automated testing in DevOps represents a key factor for providing fast release of new software features assuring quality delivery. In this paper, we introduce DOXAT, an automated framework for continuous development and testing of access control mechanisms based on the XACML standard. It leverages mutation analysis for the selection and assessment of the test strategies and provides automated facilities for test oracle definition, test execution, and results analysis, in order to speedup and automate the Plan, Code, Build, and Test phases of DevOps process. We show the usage of the framework during the planning and testing phases of the software development cycle of a PDP example.Source: Journal of software (Malden, Mass. Online) (2020). doi:10.1002/smr.2306
DOI: 10.1002/smr.2306
Project(s): CyberSec4Europe
Metrics:
Daoudagh S., Lonetti F., Marchetti E.
Automated testing in DevOps represents a key factor for providing fast release of new software features assuring quality delivery. In this paper, we introduce DOXAT, an automated framework for continuous development and testing of access control mechanisms based on the XACML standard. It leverages mutation analysis for the selection and assessment of the test strategies and provides automated facilities for test oracle definition, test execution, and results analysis, in order to speedup and automate the Plan, Code, Build, and Test phases of DevOps process. We show the usage of the framework during the planning and testing phases of the software development cycle of a PDP example.Source: Journal of software (Malden, Mass. Online) (2020). doi:10.1002/smr.2306
DOI: 10.1002/smr.2306
Project(s): CyberSec4Europe
Metrics:
See at:
Journal of Software Evolution and Process | onlinelibrary.wiley.com | CNR ExploRA
2019
Report
Open Access
ISTI Young Researcher Award "Matteo Dellepiane" - Edition 2019
Barsocchi P., Candela L., Crivello A., Esuli A., Ferrari A., Girardi M., Guidotti R., Lonetti F., Malomo L., Moroni D., Nardini F. M., Pappalardo L., Rinzivillo S., Rossetti G., Robol L.
The ISTI Young Researcher Award (YRA) selects yearly the best young staff members working at Institute of Information Science and Technologies (ISTI). This award focuses on quality and quantity of the scientific production. In particular, the award is granted to the best young staff members (less than 35 years old) by assessing their scientific production in the year preceding the award. This report documents the selection procedure and the results of the 2019 YRA edition. From the 2019 edition on the award is named as "Matteo Dellepiane", being dedicated to a bright ISTI researcher who prematurely left us and who contributed a lot to the YRA initiative from its early start.Source: ISTI Technical reports, 2019
Barsocchi P., Candela L., Crivello A., Esuli A., Ferrari A., Girardi M., Guidotti R., Lonetti F., Malomo L., Moroni D., Nardini F. M., Pappalardo L., Rinzivillo S., Rossetti G., Robol L.
The ISTI Young Researcher Award (YRA) selects yearly the best young staff members working at Institute of Information Science and Technologies (ISTI). This award focuses on quality and quantity of the scientific production. In particular, the award is granted to the best young staff members (less than 35 years old) by assessing their scientific production in the year preceding the award. This report documents the selection procedure and the results of the 2019 YRA edition. From the 2019 edition on the award is named as "Matteo Dellepiane", being dedicated to a bright ISTI researcher who prematurely left us and who contributed a lot to the YRA initiative from its early start.Source: ISTI Technical reports, 2019
See at:
ISTI Repository | CNR ExploRA
2019
Journal article
Open Access
A systematic review on cloud testing
Bertolino A., De Angelis G., Gallego M., García B., Gortázar F., Lonetti F., Marchetti E.
A systematic literature review is presented that surveyed the topic of cloud testing over the period 2012-2017. Cloud testing can refer either to testing cloud-based systems (testing of the cloud) or to leveraging the cloud for testing purposes (testing in the cloud): both approaches (and their combination into testing of the cloud in the cloud) have drawn research interest. An extensive paper search was conducted by both automated query of popular digital libraries and snowballing, which resulted in the final selection of 147 primary studies. Along the survey, a framework has been incrementally derived that classifies cloud testing research among six main areas and their topics. The article includes a detailed analysis of the selected primary studies to identify trends and gaps, as well as an extensive report of the state-of-the-art as it emerges by answering the identified Research Questions. We find that cloud testing is an active research field, although not all topics have received enough attention and conclude by presenting the most relevant open research challenges for each area of the classification framework.Source: ACM computing surveys 52 (2019). doi:10.1145/3331447
DOI: 10.1145/3331447
Project(s): ELASTEST
Metrics:
Bertolino A., De Angelis G., Gallego M., García B., Gortázar F., Lonetti F., Marchetti E.
A systematic literature review is presented that surveyed the topic of cloud testing over the period 2012-2017. Cloud testing can refer either to testing cloud-based systems (testing of the cloud) or to leveraging the cloud for testing purposes (testing in the cloud): both approaches (and their combination into testing of the cloud in the cloud) have drawn research interest. An extensive paper search was conducted by both automated query of popular digital libraries and snowballing, which resulted in the final selection of 147 primary studies. Along the survey, a framework has been incrementally derived that classifies cloud testing research among six main areas and their topics. The article includes a detailed analysis of the selected primary studies to identify trends and gaps, as well as an extensive report of the state-of-the-art as it emerges by answering the identified Research Questions. We find that cloud testing is an active research field, although not all topics have received enough attention and conclude by presenting the most relevant open research challenges for each area of the classification framework.Source: ACM computing surveys 52 (2019). doi:10.1145/3331447
DOI: 10.1145/3331447
Project(s): ELASTEST
Metrics:
See at:
Recolector de Ciencia Abierta, RECOLECTA | ISTI Repository | ZENODO | ACM Computing Surveys | dl.acm.org | ACM Computing Surveys | CNR ExploRA
2019
Conference article
Open Access
Governing Regression Testing in Systems of Systems
Bertolino A., De Angelis G., Lonetti F.
Great advances in network technology and software engineering have triggered the development and spread of Systems of Systems (SoSs). The dynamic and evolvable nature of SoSs poses important challenges on the validation of such systems and in particular on their regression testing, aiming at assessing that run-time changes and evolutions do not introduce regression in SoS behavior. This paper outlines issues and challenges of regression testing of SoSs, identifying the main kinds of evolution that can impact on their regression testing activity. Furthermore, it presents a conceptual framework for governing the regression testing of SoSs. The proposed framework leverages the concept of an orchestration graph that describes the flow of test cases and sketches a solution for deriving a regression test plan according to test cases dependencies.Source: 1st International Workshop on Governing Adaptive and Unplanned Systems of Systems, pp. 144–148, Berlin, Germany, 28/10/2019
DOI: 10.1109/issrew.2019.00064
Project(s): ELASTEST
Metrics:
Bertolino A., De Angelis G., Lonetti F.
Great advances in network technology and software engineering have triggered the development and spread of Systems of Systems (SoSs). The dynamic and evolvable nature of SoSs poses important challenges on the validation of such systems and in particular on their regression testing, aiming at assessing that run-time changes and evolutions do not introduce regression in SoS behavior. This paper outlines issues and challenges of regression testing of SoSs, identifying the main kinds of evolution that can impact on their regression testing activity. Furthermore, it presents a conceptual framework for governing the regression testing of SoSs. The proposed framework leverages the concept of an orchestration graph that describes the flow of test cases and sketches a solution for deriving a regression test plan according to test cases dependencies.Source: 1st International Workshop on Governing Adaptive and Unplanned Systems of Systems, pp. 144–148, Berlin, Germany, 28/10/2019
DOI: 10.1109/issrew.2019.00064
Project(s): ELASTEST
Metrics:
See at:
ISTI Repository | ZENODO | zenodo.org | doi.org | ieeexplore.ieee.org | CNR ExploRA
2019
Conference article
Open Access
A decentralized solution for combinatorial testing of access control engine
Daoudagh S., Lonetti F., Marchetti E.
In distributed environments, information security is a key factor and access control is an important means to guarantee confidentiality of sensitive and valuable data. In this paper, we introduce a new decentralized framework for testing of XACML-based access control engines. The proposed framework is composed of different web services and provides the following functionalities: I) generation of test cases based on combinatorial testing strategies; ii) decentralized oracle that associates the expected result to a given test case, i.e. an XACML request; and finally, iii) a GUI for interacting with the framework and providing some analysis about the expected results. A first validation confirms the efficiency of the proposed approach.Source: ICISSP 2019 - 5th International Conference on Information Systems Security and Privacy, pp. 126–135, Prague, Czech Republic, 23-25 February 2019
DOI: 10.5220/0007379401260135
Project(s): CyberSec4Europe
Metrics:
Daoudagh S., Lonetti F., Marchetti E.
In distributed environments, information security is a key factor and access control is an important means to guarantee confidentiality of sensitive and valuable data. In this paper, we introduce a new decentralized framework for testing of XACML-based access control engines. The proposed framework is composed of different web services and provides the following functionalities: I) generation of test cases based on combinatorial testing strategies; ii) decentralized oracle that associates the expected result to a given test case, i.e. an XACML request; and finally, iii) a GUI for interacting with the framework and providing some analysis about the expected results. A first validation confirms the efficiency of the proposed approach.Source: ICISSP 2019 - 5th International Conference on Information Systems Security and Privacy, pp. 126–135, Prague, Czech Republic, 23-25 February 2019
DOI: 10.5220/0007379401260135
Project(s): CyberSec4Europe
Metrics:
See at:
doi.org | ISTI Repository | CNR ExploRA | www.scitepress.org | www.scopus.com
2019
Conference article
Open Access
Towards Runtime Monitoring for malicious behaviors detection in Smart Ecosystems
Cioroaica E., Di Giandomenico F., Kuhn T., Lonetti F., Marchetti E., Jahic J., Schnicke F.
A Smart Ecosystem reflects in the control decisions of entities of different nature, especially of its software components. Particularly, the malicious behavior requires a more accurate attention. This paper discusses the challenges related to the evaluation of software smart agents and proposes a first solution leveraging the monitoring facilities for a) assuring conformity between the software agent and its digital twin in a real-time evaluation and b) validating decisions of the digital twins during runtime in a predictive simulation.Source: ISSREW 2019 - IEEE International Symposium on Software Reliability Engineering Workshops, pp. 200–203, Berlin, Germany, 27-30 October, 2019
DOI: 10.1109/issrew.2019.00072
Project(s): SECREDAS
Metrics:
Cioroaica E., Di Giandomenico F., Kuhn T., Lonetti F., Marchetti E., Jahic J., Schnicke F.
A Smart Ecosystem reflects in the control decisions of entities of different nature, especially of its software components. Particularly, the malicious behavior requires a more accurate attention. This paper discusses the challenges related to the evaluation of software smart agents and proposes a first solution leveraging the monitoring facilities for a) assuring conformity between the software agent and its digital twin in a real-time evaluation and b) validating decisions of the digital twins during runtime in a predictive simulation.Source: ISSREW 2019 - IEEE International Symposium on Software Reliability Engineering Workshops, pp. 200–203, Berlin, Germany, 27-30 October, 2019
DOI: 10.1109/issrew.2019.00072
Project(s): SECREDAS
Metrics:
See at:
ISTI Repository | doi.org | ieeexplore.ieee.org | CNR ExploRA
2019
Contribution to book
Closed Access
A General Framework for Decentralized Combinatorial Testing of Access Control Engine: Examples of Application
Daoudagh S., Lonetti F., Marchetti E.
Access control mechanisms aim to assure data protection in modern software systems. Testing of such mechanisms is a key activity to avoid security flaws and violations inside the systems or applications. In this paper, we introduce the general architecture of a new decentralized framework for testing of XACML-based access control engines. The proposed framework is composed of different web services and can be instantiated for different testing purposes: i) generation of test cases based on combinatorial testing strategies; ii) distributed test cases execution; iii) decentralized oracle derivation able to associate the expected authorization decision to a given XACML request. The effectiveness of the framework has been proven into two different experiments. The former addressed the evaluation of the distributed vs non distributed testing solution. The latter focused on the performance comparison of two distributed oracle approaches.Source: Information Systems Security and Privacy, edited by Paolo Mori, Steven Furnell, Olivier Camp, pp. 207–229, 2019
DOI: 10.1007/978-3-030-49443-8_10
Project(s): CyberSec4Europe
Metrics:
Daoudagh S., Lonetti F., Marchetti E.
Access control mechanisms aim to assure data protection in modern software systems. Testing of such mechanisms is a key activity to avoid security flaws and violations inside the systems or applications. In this paper, we introduce the general architecture of a new decentralized framework for testing of XACML-based access control engines. The proposed framework is composed of different web services and can be instantiated for different testing purposes: i) generation of test cases based on combinatorial testing strategies; ii) distributed test cases execution; iii) decentralized oracle derivation able to associate the expected authorization decision to a given XACML request. The effectiveness of the framework has been proven into two different experiments. The former addressed the evaluation of the distributed vs non distributed testing solution. The latter focused on the performance comparison of two distributed oracle approaches.Source: Information Systems Security and Privacy, edited by Paolo Mori, Steven Furnell, Olivier Camp, pp. 207–229, 2019
DOI: 10.1007/978-3-030-49443-8_10
Project(s): CyberSec4Europe
Metrics:
See at:
doi.org | link.springer.com | CNR ExploRA
2018
Conference article
Open Access
Leveraging Smart Environments for Runtime Resources Management
Barsocchi P., Calabrò A., Lonetti F., Marchetti E., Palumbo F.
Smart environments (SE) have gained widespread attention due to their flexible integration into everyday life. Applications leveraging the smart environments rely on regular exchange of critical information and need accurate models for monitoring and controlling the SE behavior. Different rules are usually specified and centralized for correlating sensor data, as well as managing the resources and regulating the access to them, thus avoiding security flaws. In this paper, we propose a dynamic and flexible infrastructure able to perform runtime resources' management by decoupling the different levels of SE control rules. This allows to simplify their continuous updating and improvement, thus reducing the maintenance effort. The proposed solution integrates low cost wireless technologies and can be easily extended to include other possible existing equipments. A first validation of the proposed infrastructure on a case study is also presented.Source: 10th International Conference on Software Quality: Methods and Tools for Better Software and Systems (SWQD 2018), pp. 171–190, Vienna, Austria, 16-19/01/2018
DOI: 10.1007/978-3-319-71440-0_10
Metrics:
Barsocchi P., Calabrò A., Lonetti F., Marchetti E., Palumbo F.
Smart environments (SE) have gained widespread attention due to their flexible integration into everyday life. Applications leveraging the smart environments rely on regular exchange of critical information and need accurate models for monitoring and controlling the SE behavior. Different rules are usually specified and centralized for correlating sensor data, as well as managing the resources and regulating the access to them, thus avoiding security flaws. In this paper, we propose a dynamic and flexible infrastructure able to perform runtime resources' management by decoupling the different levels of SE control rules. This allows to simplify their continuous updating and improvement, thus reducing the maintenance effort. The proposed solution integrates low cost wireless technologies and can be easily extended to include other possible existing equipments. A first validation of the proposed infrastructure on a case study is also presented.Source: 10th International Conference on Software Quality: Methods and Tools for Better Software and Systems (SWQD 2018), pp. 171–190, Vienna, Austria, 16-19/01/2018
DOI: 10.1007/978-3-319-71440-0_10
Metrics:
See at:
ISTI Repository | doi.org | link.springer.com | CNR ExploRA
2018
Journal article
Open Access
A categorization scheme for software engineering conference papers and its application
Bertolino A., Calabrò A., Lonetti F., Marchetti E., Miranda B.
Background In Software Engineering (SE), conference publications have high importance both in effective communication and in academic careers. Researchers actively discuss how a paper should be organized to be accepted in mainstream conferences. Aiming This work tackles the problem of generalizing and characterizing the type of papers accepted at SE conferences. Method The paper offers a new perspective in the analysis of SE literature: a categorization scheme for SE papers is obtained by merging, extending and revising related proposals from a few existing studies. The categorization scheme is used to classify the papers accepted at three top-tier SE conferences during five years (2012-2016). Results While a broader experience is certainly needed for validation and fine-tuning, preliminary outcomes can be observed relative to what problems and topics are addressed, what types of contributions are presented and how they are validated. Conclusions The results provide insights to paper writers, paper reviewers and conference organizers in focusing their future efforts, without any intent to provide judgments or authoritative guidelines.Source: The Journal of systems and software 137 (2018): 114–129. doi:10.1016/j.jss.2017.11.048
DOI: 10.1016/j.jss.2017.11.048
Metrics:
Bertolino A., Calabrò A., Lonetti F., Marchetti E., Miranda B.
Background In Software Engineering (SE), conference publications have high importance both in effective communication and in academic careers. Researchers actively discuss how a paper should be organized to be accepted in mainstream conferences. Aiming This work tackles the problem of generalizing and characterizing the type of papers accepted at SE conferences. Method The paper offers a new perspective in the analysis of SE literature: a categorization scheme for SE papers is obtained by merging, extending and revising related proposals from a few existing studies. The categorization scheme is used to classify the papers accepted at three top-tier SE conferences during five years (2012-2016). Results While a broader experience is certainly needed for validation and fine-tuning, preliminary outcomes can be observed relative to what problems and topics are addressed, what types of contributions are presented and how they are validated. Conclusions The results provide insights to paper writers, paper reviewers and conference organizers in focusing their future efforts, without any intent to provide judgments or authoritative guidelines.Source: The Journal of systems and software 137 (2018): 114–129. doi:10.1016/j.jss.2017.11.048
DOI: 10.1016/j.jss.2017.11.048
Metrics:
See at:
ISTI Repository | Journal of Systems and Software | CNR ExploRA | www.sciencedirect.com
2018
Contribution to book
Closed Access
Emerging Software Testing Technologies
Lonetti F., Marchetti E.
Software testing encompasses a variety of activities along the software development process and may consume a large part of the effort required for producing software. It represents a key aspect to assess the adequate functional and nonfunctional software behavior aiming to prevent and remedy malfunctions. The increasing complexity and heterogeneity of software poses many challenges to the development of testing strategies and tools. In this chapter, we provide a comprehensive overview of emerging software testing technologies. Beyond the basic concepts of software testing, we address prominent test case generation approaches and focus on more relevant challenges of testing activity as well as its role in recent development processes. An emphasis is also given to testing solutions tailored to the specific needs of emerging application domains.Source: Advances in Computers,, pp. 91–143. New York: Elsevier, 2018
DOI: 10.1016/bs.adcom.2017.11.003
Metrics:
Lonetti F., Marchetti E.
Software testing encompasses a variety of activities along the software development process and may consume a large part of the effort required for producing software. It represents a key aspect to assess the adequate functional and nonfunctional software behavior aiming to prevent and remedy malfunctions. The increasing complexity and heterogeneity of software poses many challenges to the development of testing strategies and tools. In this chapter, we provide a comprehensive overview of emerging software testing technologies. Beyond the basic concepts of software testing, we address prominent test case generation approaches and focus on more relevant challenges of testing activity as well as its role in recent development processes. An emphasis is also given to testing solutions tailored to the specific needs of emerging application domains.Source: Advances in Computers,, pp. 91–143. New York: Elsevier, 2018
DOI: 10.1016/bs.adcom.2017.11.003
Metrics:
See at:
doi.org | CNR ExploRA | www.sciencedirect.com