2017
Conference article  Open Access

Using NLP to Detect Requirements Defects: An Industrial Experience in the Railway Domain

Rosadini B., Ferrari A., Gori G., Fantechi A., Gnesi S., Trotta I., Bacherini S.

NLP  Quality  Requirements  Ambiguity  Defect detection 

Context and motivation: In the railway safety-critical domain requirements documents have to abide to strict quality criteria. Rule-based natural language processing (NLP) techniques have been developed to automatically identify quality defects in natural language requirements. However, the literature is lacking empirical studies on the application of these techniques in industrial settings. Question/problem: Our goal is to investigate to which extent NLP can be practically applied to detect defects in the requirements documents of a railway signalling manufacturer. Principal idea/results: To address this goal, we first identified a set of typical defects classes, and, for each class, an engineer of the company implemented a set of defect-detection patterns by means of the GATE tool for text processing. After a preliminary analysis, we applied the patterns to a large set of 1866 requirements previously annotated for defects. The output of the patterns was further inspected by two domain experts to check the false positive cases. Contribution: This is one of the first works in which defect detection NLP techniques are applied on a very large set of industrial requirements annotated by domain experts. We contribute with a comparison between traditional manual techniques used in industry for requirements analysis, and analysis performed with NLP. Our experience tells that several discrepancies can be observed between the two approaches. The analysis of the discrepancies offers hints to improve the capabilities of NLP techniques with company specific solutions, and suggests that also company practices need to be modified to effectively exploit NLP tools.

Source: Requirements Engineering: Foundation for Software Quality - 23rd International Working Conference, REFSQ 2017, pp. 344–360, Essen, Germany, February 27 - March 2, 2017


1. Alvarez, S.A.: An exact analytical relation among recall, precision, and classi cation accuracy in information retrieval. Tech. Rep. BCCS-02-01, Computer Science Department, Boston College (2002)
2. Arora, C., Sabetzadeh, M., Briand, L., Zimmer, F.: Automated checking of conformance to requirements templates using natural language processing. IEEE TSE 41(10), 944{968 (2015)
3. Berry, D., Gacitua, R., Sawyer, P., Tjong, S.F.: The case for dumb requirements engineering tools. In: REFSQ'12. pp. 211{217. Springer (2012)
4. Berry, D.M., Kamsties, E., Krieger, M.M.: From contract drafting to software speci cation: Linguistic sources of ambiguity (2003)
5. Bonin, F., Dell'Orletta, F., Montemagni, S., Venturi, G.: A contrastive approach to multi-word extraction from domain-speci c corpora. In: LREC'10 (2010)
6. CENELEC: EN 50128:2011: Railway applications - Communication, signalling and processing systems - Software for railway control and protection systems. Tech. rep. (2011)
7. Chantree, F., Nuseibeh, B., Roeck, A.N.D., Willis, A.: Identifying nocuous ambiguities in natural language requirements. In: RE'06. pp. 56{65 (2006)
8. Cunningham, H.: GATE, a general architecture for text engineering. Computers and the Humanities 36(2), 223{254 (2002)
9. Cutts, M.: The plain English guide. Oxford University Press (1996)
10. Falessi, D., Cantone, G., Canfora, G.: Empirical principles and an industrial case study in retrieving equivalent requirements via natural language processing techniques. IEEE Transactions on Software Engineering 39(1), 18{44 (2013)
11. Femmer, H., Fernndez, D.M., Wagner, S., Eder, S.: Rapid quality assurance with requirements smells. Journal of Systems and Software 123, 190 { 213 (2017)
12. Femmer, H., Kucera, J., Vetro, A.: On the impact of passive voice requirements on domain modelling. In: ESEM'14. p. 21. ACM (2014)
13. Ferrari, A., Gnesi, S.: Using collective intelligence to detect pragmatic ambiguities. In: RE'12. pp. 191{200 (2012)
14. Gleich, B., Creighton, O., Kof, L.: Ambiguity detection: Towards a tool explaining ambiguity sources. In: REFSQ'10. pp. 218{232. Springer (2010)
15. Gnesi, S., Lami, G., Trentanni, G.: An automatic tool for the analysis of natural language requirements. IJCSSE 20(1) (2005)
16. Kang, N., van Mulligen, E.M., Kors, J.A.: Comparing and combining chunkers of biomedical text. Journal of biomedical informatics 44(2), 354{360 (2011)
17. Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics pp. 159{174 (1977)
18. Manning, C.D.: Part-of-speech tagging from 97% to 100%: is it time for some linguistics? In: CICLing, pp. 171{189. Springer (2011)
19. Mich, L., Franch, M., Inverardi, P.N.: Market research for requirements analysis using linguistic tools. REJ 9(1), 40{56 (2004)
20. Tjong, S.F., Berry, D.M.: The design of SREE: A prototype potential ambiguity nder for requirements speci cations and lessons learned. In: REFSQ'13. pp. 80{ 95. Springer (2013)
21. Yang, H., Roeck, A.N.D., Gervasi, V., Willis, A., Nuseibeh, B.: Analysing anaphoric ambiguity in natural language requirements. REJ 16(3), 163{189 (2011)

Metrics



Back to previous page
BibTeX entry
@inproceedings{oai:it.cnr:prodotti:382479,
	title = {Using NLP to Detect Requirements Defects: An Industrial Experience in the Railway Domain},
	author = {Rosadini B. and Ferrari A. and Gori G. and Fantechi A. and Gnesi S. and Trotta I. and Bacherini S.},
	doi = {10.1007/978-3-319-54045-0_24},
	booktitle = {Requirements Engineering: Foundation for Software Quality - 23rd International Working Conference, REFSQ 2017, pp. 344–360, Essen, Germany, February 27 - March 2, 2017},
	year = {2017}
}