< Back to Session View

Self-driving but guided by people - Developing Ethical Goal Functions to optimise automated vehicle behaviour

Date and Time: Tuesday, July 11, 2023: 5:30 PM - 7:00 PM

Nick Reed 

Founder, Reed Mobility
@reedmobility

Presentation Description

Background:
Self-driving vehicles (SDVs) have been promoted by automotive companies, technology developers and governments as a potential route to transport that is safer and more efficient. Their intuitive appeal is compelling. However, whilst a few small scale commercial services using SDVs have started to appear in defined locations, their emergence as a significant contributor to the transport system has been far slower than was anticipated. For example, in 2016, the ridehailing company, Lyft, envisaged 80% of all their trips would be delivered by SDVs by 2022. Clearly, this prediction was inaccurate.
Three interlinked factors seem to have caused the disparity between expected and actual operation of SDVs. The first factor is technological. Challenging though it is, creating a robotic vehicle that can perceive its environment, plan a route from origin to destination and then accelerate, brake and steer appropriately along its chosen route is the easy part. Building systems capable of doing that whilst also understanding the complexity of driving in mixed traffic, making valid predictions about the future behaviours of other road users and successfully driving day-after-day in a variety of light and weather conditions is a far greater challenge. The second factor is the regulations that govern SDVs. National and international rules that define roads, vehicles, certification, driving behaviours and insurance have all been established over more than a century of operation of motor vehicles. The emergence of SDVs has provoked reappraisal of regulatory frameworks across the board. The third factor is societal acceptance that SDVs will deliver the myriad benefits promised and
Exploring the interface between these three factors, I was a member of an expert group convened by the European Commission (including ethicists, philosophers, psychologists, sociologists, lawyers and engineers) to produce recommendations on ethics for the development and deployment of connected and automated vehicles (Bonnefon et al., 2020). One of our twenty recommendations related to how SDVs should behave with respect to the formal and informal rules of the road. In a follow-up paper (Reed, Leiman, Palade, Martens & Kester,, 2021), we discussed situations in which SDVs might be required to break such rules in the interests of road safety. With the algorithms responsible for operationalizing self-driving capability typically based on deep learning techniques (exposure the system to real or simulated driving situations and codification of appropriate responses), we recognised that it is not possible to provide an SDV with the infinite variety of situations that might be encountered in the real world and, even if it were possible to approach that, the system would not be deriving the ethical principles that underpin human decision-making in the same situation. This means that decision-making by SDVs in uncertain (and ethically challenging) situations could be determined by the vehicle manufacturer or technology provider or such decision-making may be left as an emergent property of the system - with the risk that the behaviours observed do not align with societal expectations and produce new risks.
Consequently, Reed et al. (2021) proposed that SDV behaviours should be guided by what we called ‘ethical goal functions’ (EGFs). This term originates in artificial intelligence research and suggests the creation of a mathematical description of the societal values that should underpin the behaviours of a complex system. It was proposed that such an EGF could be tailored geographically - it should suit the locations in which the system was designed to operate. The EGF should evolve - as societal expectations change, the EGF should be updated accordingly. The EGF should also permit variation in behaviour - for example, a manufacturer could design (or users might choose) for their vehicles to operate in a more sporty or more comfortable manner; this is acceptable provided the vehicles remained within the parameters of the EGF.
Importantly, Reed et al. (2021) suggested that the development of EGFs for SDVs should have ‘democratic legitimacy’ - the communities affected by the deployment of SDVs should have the opportunity to contribute to the function that governs their behaviour and that the process should be overseen by an appropriate government agency so that those unhappy with SDV behaviour and EGFs can express their dissatisfaction when casting their vote. However, beyond suggesting the concept, Reed at al. (2021) did not suggest a process by which the preferences of a community could be captured and translated into an EGF. This research represents a first attempt at doing so.

Approach:
Based on a pilot study, a survey was developed in which more than 2,000 participants were asked to rate their level of agreement with a number of statements relating to the behaviour of or interactions with self-driving vehicles, focusing on a use case of a city-based self-driving bus. An example statement was ‘I would not expect a self-driving vehicle to break the rules of the road in order to avoid holding up traffic’. Each statement of the survey was aligned with one or more of eight ethical values, either positively or negatively. This enabled an estimate of the values that participants prioritised in ethically challenging situations and highlighted particular circumstances that warranted deeper investigation in a workshop setting with members of the public, all of whom were ‘city dwellers’ and therefore frequently encountered public transport services either as users, other road users or drivers. Two workshops were therefore held in which specific scenarios were explored and the ethical basis of participants’ decision making discussed.

Results and conclusions:
The results of the survey and workshop revealed that the priority for the public was for behaviours and operation of SDVs that supported trust in their operation. This included a preference for vehicles to comply strictly with existing regulations and proof that, as a minimum, the operation of SDVs did not increase risk. There was an understanding that collisions involving SDVs may still occur but there was common agreement that costs should not be cut at the expense of safety and that SDV operators should be transparent with data, sharing relevant information with government investigators to enable lessons to be learned around future safety. Sharing of safety data between companies in the interests of safety was also raised as being potentially useful. Results indicated that self-driving vehicles should seek to preserve the safety of vehicle occupants and other road users equally and that the interior design of the vehicles should account for the possibility that the vehicle may need to swerve or brake sharply to prevent an incident with another road user. Compared to other values, participants were seemingly quite relaxed about a possible need to change the design of urban infrastructure to accommodate SDVs. However, for many topics there was significant diversity of opinion, highlighting that it will be difficult to define ethical goal functions vehicles that behave in line with the expectations of all citizens - and that the behaviours SDVs adopt in their early deployments are likely to encounter resistance from the public. To improve the sensitivity of the development of the ethical goal functions, it is suggested that public surveys and workshops are developed using simulated (rather than described) scenarios that enable participants to gain greater context on a variety of SDV behaviours and where those simulations can be subtly varied across the ethical dimensions to probe the limits of what participants consider to be acceptable. More broadly, the project has highlighted that it is possible to explore societal acceptance of SDV behaviours in ethically ambiguous scenarios but that the heterogeneity of opinions means significant oversight from government authorities will be required to specify ethical goal functions that can be applied by SDV developers. The results also highlight that the public expects dialogue between developers, regulators and society to help evolve SDV behaviours towards those that are most acceptable.

Speaker Biography

Dr Nick Reed has worked at the cutting edge of transportation research for more than fifteen years. From early studies using driving simulators to examine driver behaviour, he has since been instrumental in connected and automated vehicle projects in the UK to the value of more than £50m, including leadership of the GATEway project in Greenwich and the creation of London's Smart Mobility Living Lab. Nick was Academy Director at TRL (the UK’s Transport Research Laboratory) before becoming Head of Mobility R&D at Bosch, the world’s largest automotive supplier. He has since founded Reed Mobility – an independent expert consultancy on future mobility topics working across the public, private and academic sectors to deliver transport systems that are safe, clean, efficient, ethical and equitable and including projects for the European Commission, DfT, TfL, BSI and RSSB. In November 2021, he was appointed as the first ever Chief Road Safety Advisor to National Highways, providing review and challenge to the organisation in its aim to deliver zero harm on the strategic road network by 2040.

Presentation File