REFSQ 2023
Mon 17 - Thu 20 April 2023 Barcelona, Spain
Tue 18 Apr 2023 11:00 - 11:40 at Llívia - Session R2 - NLP and ML for RE I Chair(s): Sallam Abualhaija

[Context and motivation] Incompleteness in natural-language requirements is a challenging problem. [Question/Problem] A common technique for detecting incompleteness in requirements is checking the requirements against external sources. With the emergence of language models such as BERT, an interesting question is whether language models are useful external sources for finding potential incompleteness in requirements. [Principal ideas/results] We mask words in requirements and have BERT’s masked language model (MLM) generate contextualized predictions for filling the masked slots. We simulate incompleteness by withholding content from requirements and measure BERT’s ability to predict terminology that is present in the withheld content but absent in the content disclosed to BERT. [Contributions] BERT can be configured to generate multiple predictions per mask. Our first contribution is to determine how many predictions per mask is an optimal trade-off between effectively discovering omissions in requirements and the level of noise in the predictions. Our second contribution is devising a machine learning-based filter that post-processes predictions made by BERT to further reduce noise. We empirically evaluate our solution over 40 requirements specifications drawn from the PURE dataset [1]. Our results indicate that: (1) predictions made by BERT are highly effective at pinpointing terminology that is missing from requirements, and (2) our filter can substantially reduce noise from the predictions, thus making BERT a more compelling aid for improving completeness in requirements.

Tue 18 Apr

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

11:00 - 12:30
Session R2 - NLP and ML for RE IResearch Papers at Llívia
Chair(s): Sallam Abualhaija University of Luxembourg
11:00
40m
Technical design
Using Language Models for Enhancing the Completeness of Natural-language Requirements
Research Papers
P: Dipeeka Luitel University of Ottawa, A: Shabnam Hassani University of Ottawa, A: Mehrdad Sabetzadeh University of Ottawa, D: Sarmad Bashir RISE Research Institutes of Sweden
Pre-print
11:40
40m
Scientific evaluation
Requirement or not, that is the question: A case from the railway industry
Research Papers
P: Sarmad Bashir RISE Research Institutes of Sweden, A: Muhammad Abbas RISE Research Institutes of Sweden AB, A: Mehrdad Saadatmand RISE Research Institutes of Sweden, A: Eduard Paul Enoiu Mälardalen University, A: Markus Bohlin Mälardalen University, A: Pernilla Lindberg Alstom, D: Dipeeka Luitel University of Ottawa
DOI Pre-print