In a significant development, researchers have successfully trained a Generative AI model to express uncertainty about its own answers in natural language! This breakthrough has far-reaching implications for the evolutionary process of AI overall
Imagine you are baking a cake, and you need to estimate the baking time. If you say, “I am 80% confident the cake will be ready in 30 minutes,” that’s like expressing calibrated uncertainty. It’s like a weather forecast saying, “There’s a 70% chance of rain tomorrow.” This helps you understand the level of certainty in a prediction. Just like a traffic light turning yellow indicates caution, calibrated uncertainty signals how much trust you can place in an Artificial Intelligence model’s answer. A breakthrough in AI models questioning themselves has far-reaching practical uses from autonomous vehicles to medical diagnosis.
In a significant development, researchers at the University of Oxford and OpenAI have successfully trained a Generative AI model to express uncertainty about its own answers in natural language. This breakthrough has far-reaching implications for the evolutionary process of AI, enabling the models to communicate their confidence levels in a more human-like manner.
The Problem of Uncertainty
Current state-of-the-art language models excel in various question-answering tasks but often struggle with expressing uncertainty about their answers. This lack of transparency can lead to users being misled by false statements or “hallucinations” generated by the models. To address this issue, the researchers introduced the concept of “verbalized probability,” where the model generates both an answer and a level of confidence in its answer.
Trained to Verbalize Probabilities
The team used a GPT-3 model and fine-tuned it to express epistemic uncertainty using natural language. The model was trained to produce verbalized probabilities, which are well-calibrated and map to probabilities that are well-calibrated under distribution shift. This means that the model’s confidence levels are accurate and consistent across different types of questions and contexts.
Questioning Itself
The study demonstrated that the GPT-3 model can learn to express calibrated uncertainty about its own answers in natural language without relying on model logits. The model’s ability to generalize calibration under distribution shift was tested using the CalibratedMath suite of tasks, which included elementary mathematics problems. The results showed that the model achieved reasonable calibration both in-and out-of-distribution, outperforming a strong baseline.
Towards Greater Transparency
This development has significant implications for the evolutionary process of Generative AI. By teaching models to express uncertainty, we can create more transparent and trustworthy AI systems. This is crucial for applications where AI models make statements about uncertain or unknown information, such as economic forecasts or open problems in science and mathematics.
The researchers highlighted several directions for future work, including exploring the use of verbalized probability in other AI applications and investigating the potential for meta-learning new features that generalize robustly to new tasks.
The ability of Generative AI models to express uncertainty about their own answers is a crucial step towards creating more transparent and trustworthy AI systems. Here are a few interesting use cases:
- Economic Forecasting: Expressing uncertainty is crucial for economic forecasts, as models cannot be certain about future events. Calibrated uncertainty allows economists to communicate the reliability of their predictions. For example, a model could say “there is a 70% chance that inflation will be between 3-5% next year.” This helps policymakers and businesses make more informed decisions.
- Open Problems in Science and Mathematics: In fields like physics, biology, and mathematics, models often make statements about open problems where there is no known ground truth. Calibrated uncertainty allows models to express their confidence in conjectures or hypotheses. For example, a model could say “there is a 40% chance that Goldbach’s conjecture is true.” This helps researchers prioritize which problems to focus on.
- Autonomous Vehicles: Self-driving cars need to be able to express uncertainty about their perception of the environment. For example, a car should be able to say, “I’m 90% sure there is a pedestrian in the crosswalk.” This allows the car to make safer decisions, like slowing down, when it is less certain about the situation. Calibrated uncertainty is crucial for the safe deployment of autonomous vehicles.
- Medical Diagnosis: In healthcare, AI models are increasingly being used to assist with medical diagnosis. However, doctors need to know how much to trust the model’s predictions. Calibrated uncertainty allows models to express their confidence in a diagnosis, like “there is an 80% chance this tumour is malignant.” This helps doctors make better treatment decisions and avoids overreliance on the model.
- Cybersecurity: AI models are used to detect cyber threats and vulnerabilities. Expressing calibrated uncertainty is important, as models may not always be certain about potential attacks. For example, a model could say “there is a 60% chance this network traffic is malicious.” This allows cybersecurity teams to prioritize threats and allocate resources effectively.
Towards Trustworthiness
By expressing their confidence levels, AI models can provide more transparent and trustworthy outputs to human users. This breakthrough has significant implications for the evolutionary process of Generative AI and could revolutionize the way we interact with AI models. The ability to express uncertainty in a human-like manner is crucial for building trust and transparency in AI systems. This breakthrough can be applied to any AI model that needs to communicate its confidence levels in its predictions or statements – ensuring that users understand the level of certainty involved.