Session Moderators

Moderator(s): 
Gil Amid, Chief Regulatory Affairs Officer, VP Operations and Co-Founder, Foretellix Ltd. 

Sagar Behere, VP of Safety, Foretellix Inc.

Session Organizers

Organizers:
Gil Amid, Foretellix Ltd.
Sagar Behere, Foretellix Inc.

TRB Sponsoring Committees and Partners

TRB Standing Committee on Vehicle-Highway Automation (ACP30)

Session Description and Agenda

Description

Various standards, methodologies, tools, and guidelines are being developed for defining ODDs and for describing test scenarios for Automated Driving Systems. However, a major challenge that remains for Safety V&V is: How to systematically create a set of scenarios that demonstrably, comprehensively, and efficiently covers the entire ODD and achieves the greatest risk discovery and qualification?

This session will explore recent perspectives and approaches for effective ODD definitions and using scenario-based testing to achieve, quantify, and evaluate adequate ODD coverage. It will showcase the views and needs of ADS solution developers as well as of various authorities.

Agenda

13:30-15:00 Opening presentations from various Industry solution providers and Regulatory authorities.

Presenters and topics:
Gil Amid, Chief Regulatory Affairs Officer, Co-Founder, Foretellix: Opening and Introduction
Xavier Delache, DMR/TUD, France: "A regulator enters the bar and asks for an innovative and sober cocktail of self-certification, national type-approval, and local scrutiny.“
Narayanan Kidambi NHTSA, USA: "Overview of NHTSA Research on ADS Driving Scenarios“
Chris Bartholomew, DENSO Intl., Chair of SAE 3259 (ODD Definition) and ORAD Chair: "ODD Standardization Activity“
Gil Amid, Foretellix Ltd "When ODD meets a scenario,
Prof. Siddartha Khastgir, WMG, UK: "Defining ODD: Use, Misuse, Disuse, and Abuse“
Dr. Jeff Wishart, VP Innovation, Science Foundation, Arizona, USA: "DDT Performance Assessment of an OEM and Research AV“
Sid Ram, Sr. Manager, AV Systems & Safety, Gatik, USA: "Anatomies of ODDs, scenarios, and routes for middle mile autonomy“
Sagar Behere, VP Customer Success and Safety, Foretellix, Israel/USA: Scenarios and metrics for High ODD Coverage

15:00-15:30 Break

15:30-16:30 Interactive work groups at the tables. Discussion on topic(s) related to safety-driven scenario-based V&V and achieving high ODD Coverage. Moderated round table discussions:
1. What is the meaning of ODD Coverage? What exactly are you measuring? (Sagar )
2. Where do you see the usage of ODD specification in qualification and certification ( self or by authority ) of an ADS ? How ( in what format ) should the ODD be specified ? (Gil)
3. How do you use an ODD definition (as part of the development and testing process)? (Chris B. )
4. What is ODD expansion? Many people are using this term and each one is using it differently. (Sid )
5. What should be the relation between scenarios and ODD ? coupled ? independent ? How to systematically generate a set of scenarios for a given ODD? How to make the claim that such a scenario set has high ODD Coverage? what is the usage model ?  (Xavier+Siddartha )
6. What is the distinction between: ODD as specification of an ADS, “minimum ODD requirement” toward an ADS, the current operating environment/conditions being sampled by an ADS? How should all these be used ? (NAR )
7. What aspect of safety validation are required in order to validate a correct behaviour of an ADS with respect to its ODD (e.g. no engagement while not inside ODD) ? (Alex Rudin )




16:30-17:00 Report out and conclusions.

Session Objectives

- Uncover challenges and approaches to effectively defining an ODD while considering implications for scenario-based testing and ODD Coverage.

- Develop a deeper understanding of the relation between ODD Coverage achieved by a V&V program and Safety of the ADS.

- Highlight how the needs of ADS solution developers and authorities overlap and influence each other.

- Discuss approaches for effective and efficient ODD-aware, scenario-based testing