OpenAI has introduced GPT-Red, an internal safety-focused model that automatically hunts for weaknesses in its own systems. By taking over much of the adversarial testing that human red-teamers do and feeding the attacks it finds back into training, the company says it has made its latest model, GPT-5.6, far more robust against prompt injection—the insertion of malicious instructions. GPT-Red itself is never distributed externally; it is kept in-house purely as a tool for strengthening defenses. Here is how it works and what OpenAI has reported.

The Weakness Called Prompt Injection

AI agents do their work while touching third-party data through browsers, connected apps, local files, and external tools. These connections are essential for real-world tasks, but they also open the door for malicious actors to steer a model's behavior[1]. An attacker might plant a cleverly crafted instruction in an email, a webpage, a tool response, or a code repository—designed, for example, to trick the model into sending sensitive data to an external server. This is the representative threat known as prompt injection.

Traditionally, human red-teamers hunted for such weaknesses by hand. But designing and running these exercises is time-consuming, which limits how quickly new attack techniques can be surfaced and folded into defenses. Human effort also cannot produce the sheer volume and diversity of adversarial data needed to harden a model through training[1]. OpenAI's concern is that keeping safety in step with rising model capability requires the testing side to scale up as well.

What GPT-Red Is

GPT-Red is an automated red-teaming model trained to meet that challenge. Much like a human attacker, it works toward a goal by sending a prompt, observing how the model responds, and iterating to refine its attack[1]. OpenAI says it trained GPT-Red at a compute scale comparable to some of its largest post-training runs, calling it an unprecedented amount of compute devoted purely to safety.

Training uses self-play reinforcement learning, in which the attacking GPT-Red and a set of diverse defender models are trained at the same time[1]. GPT-Red is rewarded for landing a valid attack, such as a successful prompt injection, while the defenders are rewarded for resisting the attack and still completing their original task. The stronger the defenders become, the more GPT-Red is forced to invent stronger and more varied techniques, so both sides improve together. A distinctive point is that OpenAI builds a broad set of realistic settings where injections might be inserted—part of a local file, a webpage banner, an email body, the output of a tool—each with a threat model specifying what GPT-Red can control and what counts as a successful attack.

Notably, OpenAI keeps GPT-Red separate from the models it deploys[1]. The idea is to keep the deliberately trained attack capabilities out of adversaries' hands, using them only to raise the robustness of production models.

The Attack Power It Demonstrated in the Field

How well GPT-Red holds up is shown through tests distinct from its training. On an indirect prompt injection arena made up of attack scenarios not used in training, human red-teamers and GPT-Red independently attempted attacks against GPT-5.1; GPT-Red succeeded on 84 percent of scenarios, well above the 13 percent for humans[1].

Tests closer to real-world operation were run as well. Targeting an AI-powered vending machine in OpenAI's office—made by Andon Labs and similar to the experimental Project Vend—GPT-Red achieved all three of its malicious objectives[1]: rewriting the price of an in-stock expensive item to the minimum allowed 0.50 USD (about 80 yen), ordering a new item worth more than 100 USD (about 16,000 yen) and offering it for 0.50 USD, and canceling another customer's order. OpenAI has disclosed these vulnerabilities and says new safeguards are currently being tested.※1 USD = 162 JPY

GPT-Red was also tested against an agent for the coding assistant Codex (built on GPT-5.4 mini), across 10 scenarios designed to exfiltrate sensitive data to the outside[1]. Compared with a baseline that simply used GPT-5.5, OpenAI says GPT-Red succeeded at data exfiltration in more cases and did so with fewer tokens.

How Much More Robust GPT-5.6 Became

The ultimate aim of GPT-Red is not attacking but raising the defensive strength of production models. Over roughly the past six months, OpenAI has trained progressively stronger attack models while increasing compute, using them in training every model since GPT-5.3[1]. As a result, the company says robustness has risen with each successive release.

As a concrete example, an early version of GPT-Red discovered a new class of direct prompt injection called "Fake Chain-of-Thought" attacks. These reached success rates above 95 percent against GPT-5.1, but have dropped below 10 percent on GPT-5.6 Sol[1]. On the hardest direct prompt injection benchmark, GPT-5.6 Sol also cut failures to one-sixth of the best model from four months earlier, and fails on just 0.05 percent of GPT-Red's direct prompt injections.

One natural worry is whether the apparent robustness merely comes from "doing less." Refusing more requests or dulling capability does make a model harder to attack, but that is not useful robustness. OpenAI says it evaluated both general capability and whether the model over-refuses, and found that normal capabilities were left intact while only robustness improved[1].

That said, all of these figures are OpenAI's own. The real effect can vary with the type of attack and the operating conditions, so it is reasonable to weigh them alongside the detailed paper (a preprint) said to be coming later this week, as well as third-party verification.

Summary

GPT-Red is an attempt at "self-improvement for safety"—using today's models to make tomorrow's models safer. By automating the adversarial testing that once relied on human effort and channeling the attacks it finds into training, GPT-5.6 is said to be the most robust yet against prompt injection. Keeping the attack-capable GPT-Red out of the public and confined to a defensive role is another measure to limit misuse risk. Bearing in mind that the figures are self-reported, it will be worth watching how far this approach holds up as the paper and independent verification arrive.

Source: https://openai.com/index/unlocking-self-improvement-gpt-red