REFSQ 2023
Mon 17 - Thu 20 April 2023 Barcelona, Spain

Requirements Engineering (RE) is a critical factor in developing high-quality and successful software, systems, and services. The REFSQ working conference series is an established international forum for discussing current and state-of-the-art RE practices, celebrating its 29th edition.

Please check the CfP here: https://2023.refsq.org/track/refsq-2023-papers#Call-for-Papers

Supporters
Organized by
Organized by
Organized by
Sponsored by
Supported by
Supported by
Dates
Plenary
You're viewing the program in a time zone which is different from your device's time zone change time zone

Tue 18 Apr

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

09:00 - 09:15
Conference openingResearch Papers at Sitges
09:15 - 10:15
Keynote 1 - Barbara PaechResearch Papers at Sitges
09:15
60m
Keynote
Explicit and Implicit Values in and of Requirements Engineering Practice and Research.
Research Papers
Barbara Paech Heidelberg University
File Attached
10:30 - 11:00
Coffee BreakCatering at Garden
10:30
30m
Coffee break
Break
Catering

11:00 - 12:30
Session R2 - NLP and ML for RE IResearch Papers at Llívia
Chair(s): Sallam Abualhaija University of Luxembourg
11:00
40m
Technical design
Using Language Models for Enhancing the Completeness of Natural-language Requirements
Research Papers
P: Dipeeka Luitel University of Ottawa, A: Shabnam Hassani University of Ottawa, A: Mehrdad Sabetzadeh University of Ottawa, D: Sarmad Bashir RISE Research Institutes of Sweden
Pre-print
11:40
40m
Scientific evaluation
Requirement or not, that is the question: A case from the railway industry
Research Papers
P: Sarmad Bashir RISE Research Institutes of Sweden, A: Muhammad Abbas RISE Research Institutes of Sweden AB, A: Mehrdad Saadatmand RISE Research Institutes of Sweden, A: Eduard Paul Enoiu Mälardalen University, A: Markus Bohlin Mälardalen University, A: Pernilla Lindberg Alstom, D: Dipeeka Luitel University of Ottawa
DOI Pre-print
11:00 - 12:30
Session R1 - Requirements Communication and Conceptualization IResearch Papers at Sitges
Chair(s): Jennifer Horkoff Chalmers and the University of Gothenburg
11:00
40m
Scientific evaluation
Supporting Shared Understanding in Asynchronous Communication Contexts
Research Papers
P: Lukas Nagel Leibniz University Hannover, A: Oliver Karras TIB - Leibniz Information Centre for Science and Technology, A: Seyed Mahdi Amiri Leibniz University Hannover, A: Kurt Schneider Leibniz Universität Hannover, Software Engineering Group, D: Elisabeth Henkel University Freiburg
11:40
40m
Scientific evaluation
An Empirical Study of the Intuitive Understanding of a Formal Pattern Language
Research Papers
P: Elisabeth Henkel University Freiburg, A: Nico Hauff University Freiburg, A: Lukas Eber Albert-Ludwigs-Universitaet Freiburg, A: Vincent Langenfeld University of Freiburg, A: Andreas Podelski University of Freiburg, D: Lukas Nagel Leibniz University Hannover
File Attached
12:30 - 14:00
12:30
90m
Lunch
Lunch
Catering

14:00 - 15:30
Session R4 - Requirements and App Review ClassificationResearch Papers at Llívia
Chair(s): Maya Daneva University of Twente
14:00
40m
Technical design
Requirements classification using fastText and BETO in Spanish documents
Research Papers
P: Maria Isabel Limaylla Lunarejo Universidade da Coruña, A: Nelly Condori-Fernández Universidad de Santiago de Compostela, A: Miguel Rodríguez Luaces Universidade da Coruña, D: Michelle Binder University of Cologne, D: Annika Vogt University of Cologne
File Attached
14:40
40m
Technical design
Automatically Classifying Kano Model Factors in App Reviews
Research Papers
P: Michelle Binder University of Cologne, P: Annika Vogt University of Cologne, A: Adrian Bajraktari University of Cologne, A: Andreas Vogelsang University of Cologne, D: Maria Isabel Limaylla Lunarejo Universidade da Coruña
14:00 - 15:30
Session R3 - Requirements Communication and Conceptualization IIResearch Papers at Sitges
Chair(s): Eric Knauss Chalmers | University of Gothenburg
14:00
40m
Scientific evaluation
Requirements Engineering Issues Experienced by Software Practitioners: A Study on Stack Exchange
Research Papers
P: Rodrigo Spinola Virginia Commonwealth University, A: Sávio Freire Federal Institute of Ceará, A: Felipe Gomes Federal University of Bahia, A: Larissa Barbosa Federal University of Bahia, A: Thiago Souto Mendes Federal Institute of Bahia, A: Galdir Reges Salvador University, A: Rita S. P. Maciel Federal University of Bahia, A: Manoel Mendonça Federal University of Bahia, D: Victoria Sakhnini University of Waterloo
14:40
20m
Research preview
Bringing Stakeholders Along for the Ride: Towards Supporting Intentional Decisions in Software Evolution
Research Papers
P: Alicia M. Grubb Smith College, D: Paola Spoletini Kennesaw State University, D: Rodrigo Spinola Virginia Commonwealth University
15:00
20m
Research preview
Scope Determined (D) and Scope Determining (G) Requirements: A New Categorization of Functional Requirements
Research Papers
P: Victoria Sakhnini University of Waterloo, A: Dan Berry University of Waterloo, A: Marcia Lucena Universidade Federal do Rio Grande do Norte, A: Abhishek Dhakla University of Waterloo, D: Paola Spoletini Kennesaw State University
15:30 - 16:00
Coffee breakCatering at Garden
15:30
30m
Coffee break
Break
Catering

16:00 - 17:30
Session R6 - RE for Automotive and Mission-Critical SystemsResearch Papers at Llívia
Chair(s): Erik Kamsties FH Dortmund
16:00
40m
Scientific evaluation
Requirements Engineering for Automotive Perception Systems
Research Papers
P: Khan Mohammad Habibullah University of Gothenburg, A: Hans-Martin Heyn University of Gothenburg & Chalmers University of Technology, A: Gregory Gay Chalmers | University of Gothenburg, A: Jennifer Horkoff Chalmers and the University of Gothenburg, A: Eric Knauss Chalmers | University of Gothenburg, A: Markus Borg CodeScene, A: Alessia Knauss Zenseact AB, A: Hakan Sivencrona Zenseact AB, A: Polly Jing Li Kognic AB, D: Murat Erdogan Veoneer, Linköping
16:40
20m
Vision and Emerging Results
Out-of-Distribution detection as Support for Autonomous Driving Safety Lifecycle
Research Papers
P: Murat Erdogan Veoneer, Linköping, A: Jens Henriksson Semcon, dept. Software and Emerging Tech, Gothenburg, A: Stig Ursing Semcon, dept. Software and Emerging Tech, Gothenburg, A: Fredrik Warg RISE Research Institutes of Sweden, A: Anders Thorsén RISE Research Institutes of Sweden, A: Johan Jaxing Agreat, Gothenburg, A: Ola Örsmark Comentor, Gothenburg, A: Mathias Örtenberg Toftås Semcon, dept. Software and Emerging Tech, Gothenburg, D: Thomas Pressburger NASA ARC
Pre-print
17:00
20m
Experience report
Authoring, Analyzing, and Monitoring Requirements for a Lift-Plus-Cruise Aircraft
Research Papers
P: Thomas Pressburger NASA ARC, A: Andreas Katis KBR / NASA Ames Research Center, A: Aaron Dutle NASA Langley Research Center, A: Anastasia Mavridou KBR / NASA Ames Research Center, D: Khan Mohammad Habibullah University of Gothenburg, Sweden
16:00 - 17:30
Session R5 - RE for Artificial IntelligenceResearch Papers at Sitges
Chair(s): Andreas Vogelsang University of Cologne
16:00
40m
Scientific evaluation
An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applications
Research Papers
P: Hans-Martin Heyn University of Gothenburg & Chalmers University of Technology, A: Eric Knauss Chalmers | University of Gothenburg, A: Iswarya Malleswaran Chalmers University of Technology, A: Shruthi Dinakaran Chalmers University of Technology, D: Anastasia Mavridou KBR / NASA Ames Research Center
Pre-print
16:40
20m
Research preview
Exploring Requirements for Software that Learns: A Research Preview
Research Papers
P: Anastasia Mavridou KBR / NASA Ames Research Center, A: Marie Farrell The University of Manchester, A: Johann Schumann KBR / NASA Ames Research Center, D: Xavier Franch Universitat Politècnica de Catalunya
17:00
20m
Vision and Emerging Results
A Requirements Engineering Perspective to AI-based Systems Development: A Vision Paper
Research Papers
P: Xavier Franch Universitat Politècnica de Catalunya, A: Andreas Jedlitschka Fraunhofer, A: Silverio Martínez-Fernández UPC-BarcelonaTech, D: Hans-Martin Heyn University of Gothenburg & Chalmers University of Technology
17:30 - 18:00
Joint summary R1-R6Research Papers at Sitges

Thu 20 Apr

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

09:30 - 10:30
Keynote 3 - Klaas-Jan StolResearch Papers at Sitges
09:30
60m
Keynote
The ABC of Requirements Engineering Research
Research Papers
Klaas-Jan Stol Lero; University College Cork; SINTEF Digital
File Attached
10:30 - 11:00
Coffee breakCatering at Garden
10:30
30m
Coffee break
Break
Catering

11:00 - 12:30
Session R8 - RE in Practice & Goal ModellingResearch Papers at Llívia
Chair(s): Maya Daneva University of Twente
11:00
20m
Experience report
Knowns and Unknowns: An Experience Report on Discovering Tacit Knowledge of Maritime Surveyors
Research Papers
P: Tor Sporsem SINTEF, A: Morten Hatling SINTEF Technology and Society, A: Anastasiia Tkalich SINTEF, A: Klaas-Jan Stol Lero; University College Cork; SINTEF Digital , D: Diane Hassett University of Limerick
Pre-print
11:20
20m
Experience report
Feel It, Code It: Emotional Goal Modelling for Gender-Inclusive Design
Research Papers
P: Diane Hassett University of Limerick, A: Amel Bennaceur The Open University, A: Bashar Nuseibeh The Open University (UK) & Lero (Ireland), D: Lotte Mygind Mjølner Informatics
File Attached
11:40
20m
Experience report
A Product Owner's Navigation in Power Imbalance Between Business and IT: An Experience Report
Research Papers
P: Lotte Mygind Mjølner Informatics, A: Jens Bæk Jørgensen Mjølner Informatics, A: Lutz Prechelt Freie Universität Berlin, D: Tor Sporsem SINTEF
File Attached
12:00
20m
Journal Early-Feedback
Towards End-to-end Merging of Goal Models
Research Papers
P: Alicia M. Grubb Smith College, A: Anisha Jain Smith College, A: Xinran Bi Smith College, A: Kathleen R. Hablutzel Smith College, D: Paola Spoletini Kennesaw State University, D: Daniel Amyot University of Ottawa
11:00 - 12:30
Session R7 - NLP and ML for RE IIResearch Papers at Sitges
Chair(s): Nelly Condori-Fernández Universidad de Santiago de Compostela
11:00
40m
Technical design
Summarization of Elicitation Conversations to Locate Requirements-Relevant Information
Research Papers
P: Xavier de Bondt fizor., A: Tjerk Spijkman Utrecht University, A: Fabiano Dalpiaz Utrecht University, A: Sjaak Brinkkemper Utrecht University, D: David Mosquera Zurich University of Applied Sciences
Pre-print
11:40
40m
Technical design
Ontology-based Automatic Reasoning and NLP for Software Traceability with the OntoTrace Tool
Research Papers
P: David Mosquera Zurich University of Applied Sciences, A: Marcela Ruiz Zurich University of Applied Sciences (ZHAW), A: Oscar Pastor Universitat Politecnica de Valencia, A: Jürgen Spielberger ZHAW, D: Tjerk Spijkman
File Attached
12:30 - 14:00
12:30
90m
Lunch
Lunch
Catering

14:00 - 15:30
Session R10 - Security Requirements and Best Poster and ToolResearch Papers at Llívia
Chair(s): Sallam Abualhaija University of Luxembourg, Elda Paja IT University of Copenhagen

Elda will chair the part of the session dedicated to the scientific papers, and Sallam will chair the best poster and best tool

14:00
20m
Research preview
Understanding the Role of Human-Related Factors in Security Requirements Elicitation
Research Papers
P: Jason Jaskolka Carleton University, A: Sanaa Alwidian Ontario Tech University, D: Roman Trentinaglia Fraunhofer IEM
14:20
20m
Experience report
Eliciting Security Requirements - an Experience Report
Research Papers
P: Roman Trentinaglia Fraunhofer IEM, A: Sven Merschjohann Fraunhofer IEM, A: Markus Fockel Fraunhofer IEM, A: Hendrik Eikerling Fraunhofer IEM, D: Jason Jaskolka Carleton University
14:40
20m
Journal Early-Feedback
The Relationship between Team Climate and Implementation of Security in Software Development
Research Papers
A: Irit Hadar University of Haifa, P: Micha Prudjinski University of Haifa, A: Gil Luria University of Haifa, D: Paola Spoletini Kennesaw State University, D: Daniel Amyot University of Ottawa
15:00
10m
Best Poster
Research Papers

15:10
10m
Best Tool
Research Papers

14:00 - 15:10
Session R9 - Data-driven and Crowd REResearch Papers at Sitges
Chair(s): Eduard C. Groen Fraunhofer IESE
14:00
20m
Research preview
Data-driven Persona Creation, Validation, and Evolution
Research Papers
P: Nitish Patkar FHNW, A: Norbert Seyff University of Applied Sciences and Arts Northwestern Switzerland FHNW, D: Emitzá Guzmán University of Zurich
14:20
20m
Research preview
Towards a Cross-Country Analysis of Software-related Tweets
Research Papers
P: Emitzá Guzmán Vrije Universiteit Amsterdam, A: Ricarda Anna-Lena Fischer Vrije Universiteit Amsterdam, A: Saliha Tabbassum Vrije Universiteit Amsterdam, D: Leon Radeck Heidelberg University
14:40
20m
Research preview
Integrating Implicit Feedback into Crowd Requirements Engineering – a Research Preview
Research Papers
P: Leon Radeck Heidelberg University, A: Barbara Paech Heidelberg University, D: Nitish Patkar University of Bern
15:30 - 15:45
Coffee BreakCatering at Sitges
15:30
15m
Coffee break
Break
Catering

15:45 - 16:15
Session R11 - Most Influential PaperResearch Papers at Sitges
Chair(s): Martin Glinz University of Zurich
15:45
30m
Research paper
The Design of SREE — A Prototype Potential Ambiguity Finder for Requirements Specifications and Lessons Learned (2013)
Research Papers
A: Sri Fatimah Tjong , P: Dan Berry University of Waterloo
DOI File Attached
16:15 - 16:45
Joint summary R7-R10Research Papers at Sitges
16:45 - 17:10
Conference closingResearch Papers at Sitges

Accepted Papers

In response to the Call for Papers, we received 84 abstracts, which resulted in 78 full papers, which reviewed by three program committee members, extensively discussed among the reviewers, and then brought for additional discussion if needed and final decision at the plenary program committee meeting that was held (online) on January 17 and 18, 2023. Nine papers for which no consensus had been reached were discussed in special depth, with nine of them accepted on the condition that certain improvements be made (those underwent an additional check by a PC member before final acceptance).

Overall, 25 papers were finally accepted for publication. In particular, based on paper category, the acceptance ratios are as follows:

  • Scientific Evaluation (15 pages): 20 submissions, 7 accepted (35%)

  • Technical Design (15 pages): 22 submissions, 5 accepted (23%)

  • Experience report papers (12 pages): 14 submissions, 5 accepted (36%)

  • Vision (8 pages): 3 submissions, 2 accepted (67%)

  • Research Preview (8 pages): 14 submissions, 6 accepted (43%)

The acceptance rate of full contributions was thus 29% (12/42).

List of accepted papers

Title
An Empirical Study of the Intuitive Understanding of a Formal Pattern Language
Research Papers
File Attached
An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applications
Research Papers
Pre-print
A Product Owner's Navigation in Power Imbalance Between Business and IT: An Experience Report
Research Papers
File Attached
A Requirements Engineering Perspective to AI-based Systems Development: A Vision Paper
Research Papers
Authoring, Analyzing, and Monitoring Requirements for a Lift-Plus-Cruise Aircraft
Research Papers
Automatically Classifying Kano Model Factors in App Reviews
Research Papers
Bringing Stakeholders Along for the Ride: Towards Supporting Intentional Decisions in Software Evolution
Research Papers
Data-driven Persona Creation, Validation, and Evolution
Research Papers
Eliciting Security Requirements - an Experience Report
Research Papers
Exploring Requirements for Software that Learns: A Research Preview
Research Papers
Feel It, Code It: Emotional Goal Modelling for Gender-Inclusive Design
Research Papers
File Attached
Integrating Implicit Feedback into Crowd Requirements Engineering – a Research Preview
Research Papers
Knowns and Unknowns: An Experience Report on Discovering Tacit Knowledge of Maritime Surveyors
Research Papers
Pre-print
Ontology-based Automatic Reasoning and NLP for Software Traceability with the OntoTrace Tool
Research Papers
File Attached
Out-of-Distribution detection as Support for Autonomous Driving Safety Lifecycle
Research Papers
Pre-print
Requirement or not, that is the question: A case from the railway industry
Research Papers
DOI Pre-print
Requirements classification using fastText and BETO in Spanish documents
Research Papers
File Attached
Requirements Engineering for Automotive Perception Systems
Research Papers
Requirements Engineering Issues Experienced by Software Practitioners: A Study on Stack Exchange
Research Papers
Scope Determined (D) and Scope Determining (G) Requirements: A New Categorization of Functional Requirements
Research Papers
Summarization of Elicitation Conversations to Locate Requirements-Relevant Information
Research Papers
Pre-print
Supporting Shared Understanding in Asynchronous Communication Contexts
Research Papers
The Relationship between Team Climate and Implementation of Security in Software Development
Research Papers
Towards a Cross-Country Analysis of Software-related Tweets
Research Papers
Towards End-to-end Merging of Goal Models
Research Papers
Understanding the Role of Human-Related Factors in Security Requirements Elicitation
Research Papers
Using Language Models for Enhancing the Completeness of Natural-language Requirements
Research Papers
Pre-print

Call for Papers

IMPORTANT NEWS: Submission of new papers is allowed until Nov. 18th, 2022 also for authors who did not submit a preliminary abstract by the abstract deadline (Nov. 11th, 2022). Updates of papers submitted by the deadline of Nov. 18th, 2022 are allowed until Nov. 23rd, 2022

We invite original submissions in the following categories:

  • Technical design papers (15 pages incl. references) describe the design of new artifacts, i.e., novel solutions for requirements-related problems or significant improvements of existing solutions. A preliminary evaluation of the artifacts is also expected.
  • Scientific evaluation papers (15 pages incl. references) investigate existing real-world problems, evaluate existing real-world implemented artifacts, or validate newly designed artifacts, e.g., by means such as case studies, experiments, simulation, surveys, systematic literature reviews, mapping studies, or action research. Check the Empirical Standards for guidelines and review criteria for each research type: https://github.com/acmsigsoft/EmpiricalStandards
  • (NEW this year) Experience report papers (12 pages incl. references) describe retrospective reports on experiences in applying RE techniques in practice, or addressing RE problems in real-world contexts. These papers focus on reporting the experience in a narrative form, and give prominence to the lessons learned by the authors and/or by the participants. Experience reports include also studies in which academics interview practitioners about the application of specific RE techniques, or about RE problems in practice.
  • Vision papers (8 pages incl. references) state where research in the field should be heading.
  • Research previews (8 pages incl. references) describe well-defined research ideas at an early stage of investigation which may not be fully developed.

Each type of paper has its own review criteria, which are listed here: https://2023.refsq.org/track/refsq-2023-papers#Review-Criteria

Submission, Reviewing, and Publication

Contributions must be submitted to: https://easychair.org/conferences/?conf=refsq2023

Each submission in the scope of REFSQ will undergo a single-blind review process that will involve at least three members of the program committee.

The REFSQ 2023 proceedings will be published in Springer’s LNCS series.

The best papers will be invited to submit an extended version of their contribution to a Special Issue of the Requirements Engineering Journal

Formatting

All submissions must be formatted according to the Springer LNCS/LNBIP conference proceedings template (for LaTeX and Word): https://www.springer.com/gp/computer-science/lncs/conference-proceedings-guidelines. As per the guidelines, please remember to include keywords after your abstract.

Furthermore, to facilitate accurate bidding and a better understanding of the papers, each paper submitted to REFSQ 2023 is required to have a structured abstract. The imposed structure demands each abstract have exactly 4 paragraphs with the following content:

  • Context and motivation: Situate and motivate your research.
  • Question/problem: Formulate the specific question/problem addressed by the paper.
  • Principal ideas/results: Summarize the ideas and results described in your paper. State, where appropriate, your research approach and methodology.
  • Contribution: State the main contribution of your paper. What’s the value you add (to theory, to practice, or to whatever you think that the paper adds value). Also, state the limitations of your results.

Three examples of structured abstracts are given here.

Each paper category has its own review criteria. We invite authors and reviewers to check the criteria and consider their order of relevance.


Technical design papers (15 pages incl. references) describe the design of new artifacts, i.e., novel solutions for requirements-related problems or significant improvements of existing solutions. A preliminary evaluation of the artifacts is also expected.

Review Criteria (in order of relevance):

  • Novelty: to what extent is the proposed solution novel with respect to the state-of-the-art? To what extent is related literature considered? To what extent did the authors clarify their contribution?
  • Potential Impact: is the potential impact on research and practice clearly stated? Is the potential impact convincing? Has the proposed solution been preliminarily evaluated in a representative setting?
  • Soundness: has the novel solution been developed according to recognised research methods? Is the preliminary evaluation of the solution sound? Did the authors clearly state the research questions? Are the conclusions of the preliminary evaluation logically derived from the data? Did the authors discuss the limitations of the proposal?
  • Verifiability: did the authors share their software? Did the authors share their data? Did the authors provide guidelines on how to reuse their artfiacts and replicate their results? [NOTE: sharing data and software is NOT mandatory, but papers that make an effort in this direction should be adequately rewarded]
  • Presentation: is the paper clearly presented? To what extent can the content of the paper be understood by the general RE public? If highly technical content is presented, did the authors make an effort to also summarise their proposal in an intuitive way?

Scientific evaluation papers (15 pages incl. references) investigate existing real-world problems, evaluate existing real-world implemented artifacts, or validate newly designed artifacts, e.g., by means such as case studies, experiments, simulation, surveys, systematic literature reviews, mapping studies, or action research. Check the Empirical Standards for guidelines and review criteria for each research stretegy: https://github.com/acmsigsoft/EmpiricalStandards

Review Criteria (in order of relevance):

  • Soundness: has the novel solution been developed according to recognised research methods? Is the research method justified? Is the research method adequate for the problem at hand? Did the authors clearly state the research questions, data collection, and analysis? Are the conclusions of the evaluation logically derived from the data? Did the authors discuss the threats to validity?
  • Potential Impact: is the potential impact on research and practice clearly stated? Is the potential impact convincing? Was the study carried out in a representative setting?
  • Verifiability: did the authors share their software? Did the authors share their data? Did the authors provide guidelines on how to reuse their artfiacts and replicate their results? [NOTE: sharing data and software is NOT mandatory, but papers that make an effort in this direction should be adequately rewarded]
  • Novelty: to what extent is the study novel with respect to the related literature? To what extent is related literature considered? To what extent did the authors clarify their contribution? To what extent does the study contribute to extend the body of knowledge in requirements engineering?
  • Presentation: is the paper clearly presented? To what extent can the content of the paper be understood by the general RE public? If highly technical content is presented, did the authors make an effort to also summarise their study in an intuitive way?

Experience report papers (12 pages incl. references) describe retrospective reports on experiences in applying RE techniques in practice, or addressing RE problems in real-world contexts. These papers focus on reporting the experience in a narrative form, and give prominence to the lessons learned by the authors and/or by the participants. Experience reports include also studies in which academics interview practitioners about the application of specific RE techniques, or about RE problems in practice.

Review Criteria (in order of relevance):

  • Relevance of the Application: is the application context in which the experience is carried out interesting for the RE public? Is the application context sufficiently representative? To what extent is the paper reporting a real-world experience involving practitioners? Is the experience credible?
  • Relevance of Lessons Learned: are the lessons learned sufficiently insightful? Did the authors report convincing evidence, also anecdotal, to justify the lessons learned?
  • Potential for Discussion: will the presentation of the paper raise discussion at the REFSQ conference? To what extent can REFSQ participants take inspiration to develop novel solutions based on the reported experience? To what extent can REFSQ participants take inspiration to perform sound empirical evaluations based on the reported experience?
  • Novelty: is the context of the study in line with the current RE practice? Does the study report on a contemporary problem that RE practitioners and researchers typically face?
  • Presentation: is the application context clearly presented? Are the lessons learned clearly described? To what extent can the content of the paper be understood by the general RE public?

Vision papers (8 pages incl. references) state where research in the field should be heading.

Review Criteria (in order of relevance):

  • Potential Impact: will the vision impact the future research and practice in RE? Is a roadmap discussed? Is the vision sufficiently broad to affect different subfields of RE? Do the authors discuss both short-term and long-term impacts of their vision?
  • Potential for Discussion: will the presentation of the vision raise the interest of the REFSQ audience? Will the vision raise discussion? Can the vision raise controversial opinions in the audience?
  • Novelty: is the vision sufficiently novel with respect to existing reflections within the REFSQ community? Do the authors clarify the novelty of their vision?
  • Soundness of Arguments: is the vision supported by logical arguments? Are the implications convincing?
  • Presentation: is the vision presented in a compelling way? Is the vision presented in a way that can elicit reflections in the RE community?

Research previews (8 pages incl. references) describe well-defined research ideas at an early stage of investigation which may not be fully developed.

Review Criteria (in order of relevance):

  • Novelty: did the research preview make you say “I heard it first at REFSQ!”? Is the idea sufficiently novel with respect to the state-of-the-art? Do the authors discuss related work and the contribution of their study?
  • Soundness of the Research Plan: do the authors present a convincing research plan? Did the authors discuss the limitations and risks of their plan? Is the plan referring to sound research methods? Do the authors clarify their research questions, planned data collection, and data analysis? Did the authors perform a convincing proof-of-concept or preliminary research step?
  • Potential for Discussion: will the presentation of the preview raise the interest of the REFSQ audience? Will the preview raise discussion? Will the audience be able to provide useful feedback to the authors, given the typical background of the REFSQ audience? Can the preview raise controversial opinions in the audience?
  • Presentation: is the paper clearly presented? To what extent can the content of the paper be understood by the general RE public?