REFSQ 2023
Mon 17 - Thu 20 April 2023 Barcelona, Spain

[Context and Motivation] The automotive industry is moving towards increased automation, where features such as Automated Driving Systems (ADS) typically include machine learning (ML), e.g. in the perception system.

[Question/Problem] Ensuring safety for systems partly relying on ML is a challenge. Different approaches and frameworks have been proposed, typically where the developer must define quantitative and/or qualitative acceptance criteria, and ensure the criteria are fulfilled using different methods, often including the need for extensive testing. Further, during operation, methods need to be defined that enable the vehicle to judge that it is not leaving its operational design domain (ODD) as well as recognizing edge cases or distribution shifts as potential hazards. However, there is still a gap between many quality methods and metrics employed in the ML domain and their potential use for safety assurance.

[Principal Ideas/Results] In this paper, we argue the need for connecting available ML quality methods and metrics to the safety lifecycle and explicitly show their contribution to safety. In particular, we analyse Out-Of-Distribution (OOD) detection, e.g., the frequency of novelty detection, and show its potential for multiple safety-related purposes. I.e. as (a) an acceptance criterion to decide if the software is Ready-for-Release, e.g., if there only are X novelties per Y miles, (b) in ODD selection and expansion by including novelty samples into the training/development loop, and (c) as a run-time measure, e.g., if there is a sequence of novel samples, the vehicle should consider reaching a minimal risk condition.

[Contribution] This paper describes the possibility to use OOD detection as a safety measure, and is analysed in different stages of the safety lifecycle.

Tue 18 Apr

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

16:00 - 17:30
Session R6 - RE for Automotive and Mission-Critical SystemsResearch Papers at Llívia
Chair(s): Erik Kamsties FH Dortmund
16:00
40m
Scientific evaluation
Requirements Engineering for Automotive Perception Systems
Research Papers
P: Khan Mohammad Habibullah University of Gothenburg, A: Hans-Martin Heyn University of Gothenburg & Chalmers University of Technology, A: Gregory Gay Chalmers | University of Gothenburg, A: Jennifer Horkoff Chalmers and the University of Gothenburg, A: Eric Knauss Chalmers | University of Gothenburg, A: Markus Borg CodeScene, A: Alessia Knauss Zenseact AB, A: Hakan Sivencrona Zenseact AB, A: Polly Jing Li Kognic AB, D: Murat Erdogan Veoneer, Linköping
16:40
20m
Vision and Emerging Results
Out-of-Distribution detection as Support for Autonomous Driving Safety Lifecycle
Research Papers
P: Murat Erdogan Veoneer, Linköping, A: Jens Henriksson Semcon, dept. Software and Emerging Tech, Gothenburg, A: Stig Ursing Semcon, dept. Software and Emerging Tech, Gothenburg, A: Fredrik Warg RISE Research Institutes of Sweden, A: Anders Thorsén RISE Research Institutes of Sweden, A: Johan Jaxing Agreat, Gothenburg, A: Ola Örsmark Comentor, Gothenburg, A: Mathias Örtenberg Toftås Semcon, dept. Software and Emerging Tech, Gothenburg, D: Thomas Pressburger NASA ARC
Pre-print
17:00
20m
Experience report
Authoring, Analyzing, and Monitoring Requirements for a Lift-Plus-Cruise Aircraft
Research Papers
P: Thomas Pressburger NASA ARC, A: Andreas Katis KBR / NASA Ames Research Center, A: Aaron Dutle NASA Langley Research Center, A: Anastasia Mavridou KBR / NASA Ames Research Center, D: Khan Mohammad Habibullah University of Gothenburg, Sweden