We are seeing them being used more and more – drones that deliver packages. According to Eline Bovy several factors of uncertainty play a role here. ‘Take the wind. Whether it’s very windy or there’s barely any breeze determines the need to adjust the drone’s route. Another factor is the weight, which can vary from package to package, affecting how much power the motors have to supply. These two types of uncertainty work differently: the weight remains constant during the drone’s flight, while the wind changes all the time. By incorporating the type of uncertainty in decision-making models, autonomous systems can make more optimal choices and provide more guarantees.’
How autonomous decision-making systems deal with uncertainty
Almost everyone finds it difficult to make decisions, especially when there are various forms of uncertainty at play. But when developing autonomous decision-making systems, to what extent is it possible to incorporate uncertainties so that those systems can reach more optimal decisions on their own? Eline Bovy, PhD candidate in Software Science at Radboud University, is exploring this using mathematical models. ‘By accounting for multiple possibilities, you can find more robust solutions.’
More robust solutions
Bovy explains that there is a standard mathematical model called the Markov decision process that can be used to model decision-making problems. ‘At each step, you make a choice, and each choice in turn can lead to multiple possibilities.’ Such models are used, for example, to determine how a drone can best react during a flight. Yet her research focuses not on those applications, but on the role of uncertainty within these models. ‘We are looking at questions like: can the probability of different outcomes change? If so, what does this tell us? And what does it mean if they can’t change?’
Mathematical decision-making models provide all sorts of uses. These models are already widely used in robotics for things like those delivery drones, but they can also play a role in the medical world and the financial sector, according to Bovy. ‘When you’re making investment decisions, you don’t know exactly what the market is going to do. You make estimates on which you base your choices, such as investing or selling. Decision-making models can help with that too.’
Uncertainty is inevitable, according to Bovy. ‘We rarely have all the information. In the medical world, for example, you rarely know for sure the probability of a treatment succeeding.’ It is therefore important to consider multiple scenarios. ‘If you don’t, then models are sensitive to small deviations. By accounting for multiple possibilities, you can find more robust solutions.’
Various sources of uncertainty
Bovy explains that there are various sources of uncertainty. ‘For one thing, there is uncertainty because we do not know the exact probabilities. And, for another, there’s uncertainty because we cannot always see all the information about a problem. For instance, a drone does not know the exact distance to an object or, in a medical setting, not all the health factors for a patient are known.’ What is innovative about her research is that it combines these two sources of uncertainty. ‘We already know a lot about sub-models that include one of these types of uncertainty, but less about what happens when they are merged into one model.’
In her research, Bovy is mainly looking at the properties of the underlying models. ‘We have a basic structure with states, choices, observations and possible transition models. There is also a function that indicates how good choices are with respect to a goal. We investigate how that function changes under different assumptions about uncertainty. Can you achieve the same result with simpler choices, or does that change the optimal solution?’
For her research, Bovy is already collaborating with research groups in Bochum, Antwerp, and Austin. She is also set to work with groups in Vienna and Brussels. She believes that the added value of decision-making models lies in the guarantees they provide. ‘We can mathematically determine the probability of a drone reaching its destination, or the likelihood of success for a course of medical treatment.’
The overall goal, according to Bovy, is clear. ‘We hope this research will help us calculate decisions that offer formal guarantees and are more robust in dealing with uncertainties for a wider range of problems. This will give us decisions that are safer because more possible probabilities are taken into account. This is important, for example, for autonomous systems that are actually going to be on the road with us.’
At the same time, Bovy nuances the concept of safety. ‘Safety does not just mean preventing an autonomous system from crashing. It is mainly about robustness in dealing with uncertainty. We want to use the knowledge we have to make the best-informed decisions possible.’
She emphasizes that not all problems are suitable for these models. ‘The models are especially beneficial for problems involving sequential choices with multiple outcomes, all of which have a chance of happening. Take, for example, solving a Sudoku puzzle. You have to choose a number for each empty cell, but each choice leads to a clear outcome. There are no probabilities you have to take into account.’
In her view, the greatest benefit lies in describing uncertainty more accurately. ‘If we model probabilities more realistically, we can make better decisions, which ultimately makes autonomous systems more reliable.’
Photo: GuerrillaBuzz via Unsplash
Contact information
- Organizational unit
- Faculty of Science, Institute for Computing and Information Sciences
