skip to primary navigationskip to content

AI Trust & Transparency with the Leverhulme Centre for the Future of Intelligence

This project is developing processes to ensure that AI systems are transparent, reliable and trustworthy.

As AI systems are widely deployed in real-world settings, it is critical for us to understand the mechanisms by which they take decisions, when they can be trusted to perform well, and when they may fail. This project addresses these goals in three strands.

Transparency 

We need to be able to understand the internal processes of AI systems. This is particularly challenging for approaches such as neural networks or genetic algorithms, which learn or evolve to carry out a task without clear mappings to chains of inference that are easy for a human to understand. This strand will study ways to make interpretable the reasons for an AI’s predictions or decisions.

Reliability

Real-world AIs need to perform reliably in settings that could be very different to their training environments, with associated risks of unpredictable and unwanted behaviours. We seek to develop new approaches that can guarantee good performance for scalable probabilistic reasoning, even in unforeseen settings. This may include notions of learning and inference which can supply proofs of accuracy, as used in formal verification systems. Another approach is to explore ways for an AI to monitor its situation dynamically to detect if its environment has changed beyond its reliability zone (allowing an alert and shift to a fallback mode).

Trustworthiness

Human studies indicate that a theory of mind may be essential to build empathetic trust, and for reliable initiation of acts of kindness. Equipping AIs to infer beliefs and goals of other agents (such as humans) may improve human-machine collaborations; yet such cognitive insight may prove a double-edged sword, allowing deception and even manipulation. We shall explore these themes with researchers on the Agents and Persons, and Kinds of Intelligence projects, and with leading experts from psychology.

 

Dr Adrian Weller
Trust & Technology Initiative; Department of Engineering, University of Cambridge

 


 

About us

The Trust & Technology Initiative brings together and drives forward interdisciplinary research from Cambridge and beyond to explore the dynamics of trust and distrust in relation to internet technologies, society and power; to better inform trustworthy design and governance of next generation tech at the research and development stage; and to promote informed, critical, and engaging voices supporting individuals, communities and institutions in light of technology’s increasing pervasiveness in societies.

Find out more > 

Mailing list

Sign up to the Trust & Technology mailing list to keep up-to-date with our news, events, and activities

Sign up to our mailing list >