🔍 Read the full analysis: What Does It Take To Combat Adversarial Distillation? on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI reports that updated safeguards in GPT-6.2 Astra reduced success in an internal test of extracting protected reasoning from 54.0% to 0.2%. A separate internal study found that student models trained on exact reasoning traces generally transferred cyber capabilities better than those trained on summaries, though the results lack published scores and independent replication.
OpenAI says updated safeguards in GPT-6.2 Astra lowered the success rate of an internal test for extracting protected reasoning from 54.0% to 0.2%. The company also reports that, in a separate study, student models trained on exact reasoning traces generally gained more cyber capability than students trained on summaries, highlighting a risk that model outputs could help transfer skills to systems with different safeguards.
The extraction result came from an internal adaptive evaluation using automated red-teaming. OpenAI compared GPT-6 Astra with its original safeguards against GPT-6.2 Astra with updated safeguards, testing whether an attacker could recover protected reasoning produced as the model answered an earlier prompt. The reported percentages describe success in that evaluation, not a general estimate of how often extraction attempts succeed in real-world use.
OpenAI says its response combines model training, classifiers and API-level protections, alongside investigations into coordinated attempts to extract outputs. The company describes defenses aimed at both individual requests and patterns across requests or accounts that may indicate a broader campaign. It says findings from investigations and automated red-teaming inform later training and tests.
In the capability-transfer study, researchers started each training run with the same pretrained student model. The supervised fine-tuning data mainly consisted of trajectories from Astra-level teacher models working on coding and defensive cybersecurity tasks, including task steps, tool interactions and final responses. Students trained on exact traces generally outperformed those trained on reasoning summaries after fine-tuning. After additional reinforcement learning, the exact-trace student scored higher at the largest inference budget tested on both reported benchmarks; the summary-trained student scored higher at lower budgets.
Combatting Adversarial Distillation
OpenAI reports stronger safeguards against attempts to extract protected reasoning, alongside evidence that exact reasoning traces can transfer cyber capability to student models.
How defenses work
OpenAI describes safeguards that look at individual requests and coordinated activity across requests and accounts.
Model training, classifiers and API-level protections aim to block reasoning extraction.
Patterns across requests and accounts can help identify organized extraction attempts.
Investigation findings and automated red-teaming feed back into training and testing.
What student models learn
OpenAI’s internal study compared training on exact reasoning traces with training on reasoning summaries.
- Students generally outperformed summary-trained students after supervised fine-tuning.
- After further reinforcement learning, the exact-trace student scored higher at the largest tested inference budget on both reported benchmarks.
- The summary-trained student scored higher at lower inference budgets after reinforcement learning.
- The study did not provide benchmark names or numerical scores.
Why it matters
Distillation can pass a teacher’s capabilities to a student model that may not have comparable safeguards.
Dual-use skills
Coding and defensive security work can also teach techniques applicable to attacks.
Sensitive research
Biological research can have both beneficial and harmful uses.
Why Reasoning Access Matters
Distillation can carry capabilities from a teacher model to a student through training data, not just copy individual answers. If exact reasoning traces convey more useful skills than summaries, access to those traces could affect what a student model learns, including in sensitive areas such as cybersecurity and biology. The concern is that a student may later be used in settings without protections comparable to those applied to the teacher.
The findings offer a practical safety signal, not proof that restricting traces will prevent misuse. The reported cyber tasks include defensive work, but coding and security skills can have both beneficial and harmful applications. OpenAI’s results suggest limiting access to exact traces may reduce one route for capability transfer; they do not establish how consistently that approach works across models, tasks or training methods.
The extraction test also matters to developers and API users because it frames misuse as a possible pattern of repeated activity, rather than only a single suspicious prompt. OpenAI says it looks for signals across requests and accounts. The source does not disclose specific investigation outcomes or enforcement actions, so the scale of attempted extraction and the impact of those measures cannot be judged from this report.
Products Worth Considering

DJI Mic 3 Wireless Microphone (2 Transmitters, 1 Receiver, Charging Case)

Apple AirPods Pro (1st Generation) with MagSafe Charging Case

Jackery Fridge Guard 1024Wh Backup Power Station with Mount
As an affiliate, we earn on qualifying purchases.
How Distillation Enters the Picture
In model distillation, a student learns from material generated by a teacher, which can include answers, judgments or reasoning. OpenAI uses the term adversarial distillation for unauthorized use of a frontier model’s outputs to train another model that may not have comparable safeguards. The report focuses on the risk that extracted material could serve as training data for another system.
For its cyber comparison, OpenAI says the teacher trajectories covered coding and defensive cybersecurity tasks. Researchers compared students given exact reasoning traces with students given summaries, then evaluated them after supervised fine-tuning and a further round of lightweight reinforcement learning in cyber and coding environments. The source does not name the benchmarks or publish numerical scores, limiting comparisons beyond the described ranking at different inference budgets.
These are company-reported internal findings, not results independently replicated in the material provided. The source also does not give the full evaluation methodology, attack sample size, or the calculation behind the extraction success rate. Those omissions matter when interpreting a sharp change in the reported result: the comparison is meaningful within the test as described, but its scope is not yet clear.
Limits of the Reported Tests
The source does not disclose the red-team sample size, full methodology or independent replication for the extraction evaluation. It also does not explain how success was calculated, when the updated safeguards were introduced, or whether they are available to all users. The 0.2% figure should therefore be read as the result of OpenAI’s reported internal test, not as a broad measure of extraction risk.
For the cyber study, benchmark names and numerical scores are absent. The report does not establish whether the same pattern would hold for other student models, training recipes, tasks or inference budgets. Nor does it quantify how often real-world adversarial distillation occurs, or describe investigation findings and enforcement actions.
There is also a trade-off that the supplied material does not resolve: restricting exact reasoning traces may limit some legitimate research or evaluation, while someone seeking training data could try other sources or methods. The reported tests do not show how these safeguards perform against approaches beyond the evaluation described.
Further Tests Across Models
OpenAI says it plans to study capability transfer across different student models and distillation methods. The company has not provided a timetable for more results. Additional reporting could clarify how the safeguards perform across attack approaches, whether they are broadly available and what the cyber benchmarks showed in numerical terms.
Independent replication and fuller methodological details would help establish how far the extraction result generalizes. Until then, the reported figures show what OpenAI observed in its own evaluation, while leaving the safeguards’ performance against other models and real-world attempts unsettled.
Key Questions
What is adversarial distillation?
It is the unauthorized use of a model’s outputs to train a second system. OpenAI says the risk is that the student may gain capabilities without safeguards comparable to those of the teacher.
What did OpenAI report about GPT-6.2 Astra?
OpenAI says updated safeguards lowered success in an internal test of extracting protected reasoning from 54.0% to 0.2%. The source does not provide enough detail to treat that figure as a general measure of extraction risk.
Did exact reasoning traces transfer more cyber capability?
In OpenAI’s internal study, students trained on exact traces generally outperformed summary-trained students after fine-tuning. After reinforcement learning, results depended on inference budget: the exact-trace student led at the largest budget tested on both benchmarks, while the summary-trained student led at lower budgets.
Have the results been independently verified?
The supplied source reports OpenAI’s internal evaluations and does not cite independent replication. It also omits the extraction test’s full methodology and the cyber study’s benchmark names and scores.
Primary source: OpenAI · via ThorstenMeyerAI.com
