A zero-shot learning approach to classifying requirements: preliminary study

Alhoshan W., Zhao L., Ferrari A., Letsholo K. J.

NLP  Zero-shot learning  Requirements Engineering  Software Engineering 

Context and motivation: Natural Language Processing (NLP) techniques are constantly improving their capabilities, and deep learning approaches are now used in the daily practice of several application domains. Requirements engineering (RE) research has traditionally incorporated NLP solutions to ad-dress its fundamental tasks, such as classification, tracing, and defect detection. Question/problem: However, RE research often suffers from a lack of annotated datasets, and this makes it difficult to fully exploit supervised NLP techniques in general, and deep-learning ones in the specific, thereby losing the potential advantages offered by these techniques. Principal ideas/results: To address the problem of limited annotated datasets, we propose to use zero-shot classification, and apply this learning paradigm to RE tasks that can be treated as classification problems. We experimented with the task of distinguishing between two types of NFR requirements: usability and security requirement and obtained encouraging weighted F-scores over 80% and almost perfect recall rates from a number of the tested models, without any training data and fine-tuning. Contribution: This work paves the basis for further research in the application of zero-shot learning, and towards the solution of the long-standing problem of dataset annotation in RE.

Source: REFSQ 2022 - 28th International Working Conference on Requirement Engineering: Foundation for Software Quality, pp. 52–59, Birmingham, UK, 21-24/03/2022

Metrics