The Encrypted Reasoning Problem: When AI Governance Meets Unauditable Models
Issue #3 | Wednesday, February 11, 2026 | 12-min read
TL;DR
Reasoning models (AI that “thinks” through problems step-by-step before answering) like OpenAI o1 hide their chain-of-thought (the step-by-step reasoning process) by design. You pay for reasoning tokens (the computational units that represent this thinking), but you can’t inspect them. This creates an impossible governance situation: you can’t document what you can’t see, and you can’t govern what you can’t document. With EU AI Act enforcement starting August 2026, encrypted reasoning isn’t just a transparency problem. It’s a compliance dead end. DeepSeek-R1’s full transparency is already forcing the market to choose between opacity and auditability.
Why This Topic, Why Now
A reader asked me this question on Substack Notes last week:
This question was already on my list of topics to explore in future issues. But when a reader articulates exactly the tension I’ve been watching unfold in real-time, it’s time to move it up the priority list. The encrypted reasoning problem isn’t just a theoretical debate anymore. It’s actively affecting how organizations evaluate, procure, and deploy AI systems. And with EU AI Act enforcement six months away, the clock is ticking.
So let’s talk about what happens when the newest class of AI models makes governance impossible by design.
The Hook
Imagine hiring a financial advisor who makes investment decisions for you, but when you ask how they reached those decisions, they say: “I thought about it, trust me, but I can’t show you my work.” You still get the bill for their thinking time.
That’s exactly what’s happening with the newest class of AI models called reasoning models or large reasoning models (LRMs): AI systems designed to think through problems step-by-step before generating answers.
OpenAI’s o1 model doesn’t just immediately generate responses like earlier LLMs (large language models). It uses a technique called chain-of-thought (CoT) reasoning: generating an internal sequence of reasoning steps, like showing your work on a math test. This process involves reasoning tokens (computational units representing each step of thinking), and you’re paying for every single one.
But here’s the catch: the chain-of-thought is encrypted. It’s completely hidden from you, and this hiding is deliberate.
OpenAI’s stated rationale includes protecting their competitive advantage (preventing competitors from copying their proprietary techniques) and enabling chain-of-thought monitoring (checking the AI’s internal reasoning for safety issues).
But here’s where this becomes a governance crisis:
If OpenAI needs to monitor the chain-of-thought for safety, why don’t companies deploying the AI need to monitor it for compliance?
When a bank uses AI to deny your loan, regulators require explainability: the ability to explain how a decision was made. When a hospital uses AI to suggest diagnoses, that reasoning needs to be auditable: reviewable by qualified personnel. When a company uses AI to screen resumes, employment law requires transparency.
How do you explain chain-of-thought reasoning that you’re architecturally prevented from accessing?
This isn’t a minor documentation gap. It’s a fundamental mismatch between how reasoning models operate and how governance frameworks (legal, financial, medical, ethical) are designed to operate.
Watchlist
FAIL: OpenAI o1. Hidden Chain-of-Thought as “Competitive Advantage”
What they did: OpenAI’s o1 model (launched September 2024) is a reasoning model that works differently from earlier LLMs. Before generating a response, it produces an internal chain-of-thought: a sequence of reasoning steps to solve complex problems. This involves generating reasoning tokens. You pay for these tokens (o1 costs approximately $60 per million tokens, versus $15 per million for GPT-4o), but the chain-of-thought is encrypted, so it’s completely hidden from users.
OpenAI’s explanation is that they hide the CoT (chain-of-thought) to protect their competitive advantage and to enable CoT monitoring for safety, specifically to check whether the AI is following rules.
Why this fails real governance:
This approach makes regulatory compliance impossible. Starting August 2026, the EU AI Act requires high-risk AI systems to be “sufficiently transparent to enable deployers to interpret a system’s output” (Article 13). This is the legal definition of explainability. If your model’s chain of thought is encrypted, you cannot comply. You’re trying to explain a reasoning process you don’t have access to.
This breaks industry-specific audit requirements. Banks must follow model risk management frameworks (OCC Bulletin 2011-12), requiring validation of all AI/ML models. Hospitals need traceability of clinical reasoning for medical AI. These aren’t optional. They’re regulatory requirements. When federal banking regulations say “validate all AI models with independent review,” how do you validate encrypted reasoning?
This prevents debugging and verification of model behavior. When the AI makes an error, developers need to inspect the chain of thought to identify where the reasoning failed. With encrypted CoT, this is impossible. Developer Simon Willison described it as “a big step backwards” for interpretability: the ability to understand what a model is doing.
The market forced a partial reversal. After competitor DeepSeek launched with full reasoning transparency in January 2025, OpenAI introduced reasoning summaries in o3-mini. These are processed, simplified versions of the chain-of-thought (though still not the raw reasoning tokens). This proved the encryption was always a business choice, not a technical necessity.
Grade: D (improved to C- with o3-mini reasoning summaries in February 2025)
Breaking it down:
Ethical Stakes: The grade is F because this actively prevents algorithmic transparency.
Accessibility: The grade is D because partial summaries don’t enable true auditability.
Positioning: The grade is C because they at least acknowledge the trade-off.
Execution: The grade is D because they threaten to revoke API access for users attempting CoT inspection.
WIN: DeepSeek-R1. Full Chain-of-Thought Transparency
What they did: DeepSeek-R1 (launched January 2025) takes the opposite approach by providing complete reasoning transparency. The model exposes its entire chain of thought in <think>...</think> tags. Every reasoning step is visible and inspectable. The CoT includes explicit reasoning phases: problem definition, initial analysis (called the “Bloom Cycle”), reconstruction cycles (self-verification and alternative exploration), and final decision.
It’s also open-source under an MIT license, allowing anyone to inspect the model architecture and training approach. It costs approximately $2-8 per million tokens (95% cheaper than o1) and achieves comparable benchmark performance on reasoning tasks.
Why this enables actual governance:
This provides true explainability. You can read through the complete chain-of-thought and verify the AI’s logic. When the model makes a recommendation, you can pinpoint exactly where in the reasoning process it arrived at each conclusion. This enables model behavior verification: confirming that the AI is reasoning correctly, not just producing outputs that appear correct.
This makes compliance achievable. The EU AI Act requirements for explainability are met. You can trace every decision through the visible CoT. Financial audit requirements are satisfied. Auditors can review timestamped reasoning chains. Clinical reasoning review is possible. Healthcare professionals can inspect the diagnostic logic step by step. This is what auditability actually looks like.
Competitive validation occurred through market forces. Within weeks of DeepSeek-R1’s launch, OpenAI introduced reasoning summaries in o3-mini. The reason was clear: enterprises (especially in regulated industries) were choosing the transparent reasoning model. When you’re making high-stakes decisions, black-box AI doesn’t meet procurement requirements.
What’s imperfect: The model occasionally exhibits “rumination“: generating excessively verbose reasoning that doesn’t improve accuracy, sometimes getting stuck in unproductive reflection loops. But critically, these failure modes are observable. You can measure rumination, quantify it, and design prompting strategies to minimize it. You can’t optimize what you can’t inspect.
Grade: A-
Breaking it down:
Ethical Stakes: The grade is A because this provides complete algorithmic transparency.
Accessibility: The grade is A because this offers full interpretability of reasoning processes.
Positioning: The grade is A because transparency was chosen as a core architectural decision.
Execution: The grade is B+ because the transparency is excellent despite minor verbosity issues.
What I’m Watching
The August 2026 EU AI Act enforcement deadline is approaching. The EU’s high-risk AI systems classification covers AI used in finance, healthcare, hiring, and legal applications. These systems must demonstrate explainability and maintain audit trails: documentation of how decisions were made. For companies deploying reasoning models with an encrypted chain of thought, this means they must withdraw from European markets, migrate to transparent reasoning models, or face enforcement. The first compliance cases will establish precedent on whether reasoning summaries (as provided by o3-mini) meet the legal bar or whether full CoT transparency (as in DeepSeek-R1) is required.
Enterprise adoption is hindered by hidden costs. Industry surveys show 63% of enterprises cite runaway token costs as their biggest barrier to LLM deployment. Only 5% achieve measurable ROI at scale. Encrypted reasoning exacerbates both problems: you’re paying for reasoning tokens you can’t inspect, and you can’t optimize inference costs for processes you can’t see. Organizations need cost transparency and visibility into model behavior to manage AI economics.
A two-tier market for reasoning models is emerging. Consumer AI applications (e.g., ChatGPT for personal use and content generation tools) may remain partially opaque. Users accept black-box models for convenience. But enterprise AI (especially in regulated industries) is moving toward full reasoning transparency. AI governance frameworks require auditability. Insurance underwriters won’t cover unauditable AI systems. External auditors won’t sign off on opaque models. Procurement teams are adding chain-of-thought transparency requirements to vendor contracts.
The Signal
If you can’t document it, you can’t govern it.
This is the core principle that encrypted chain-of-thought reasoning violates.
Think about how businesses actually work: every important process requires audit trails. Financial transactions get receipts. Medical procedures get clinical notes. Legal advice gets memos. Why does this matter? Because documentation creates accountability, enables review, and proves compliance.
AI reasoning is no different. When an AI denies a loan, recommends a medical treatment, or filters job candidates, that chain-of-thought needs to be documented. This is necessary not just for ethical reasons, but for legal and practical ones. This is what AI governance frameworks require.
Encrypted reasoning sends a clear message: “Our proprietary methods matter more than your ability to verify our work.” That stance might work for low-stakes consumer applications: recommendation algorithms, autocomplete, casual chatbot conversations. These are minimal-risk AI systems under most regulatory frameworks.
But for high-risk AI systems making consequential decisions about people’s lives, this approach is indefensible.
The DeepSeek moment proved that market forces move faster than regulation. When a competitor launched with full CoT transparency, the market (not regulators) forced change. OpenAI introduced reasoning summaries within weeks. The reason was clear: enterprises buying AI for serious applications stated their requirements clearly. They need auditability. They need explainability for regulators. They need interpretability for internal review.
The companies that solve transparent reasoning will own the enterprise market. The ones protecting “competitive advantage” through opacity will be locked out of the highest-value, most regulated use cases.
Let’s Talk: The Explainability Hierarchy
Not all AI transparency is created equal. Here’s how reasoning models map to governance requirements:
Level 0: Black Box
What you get: only the final output, with no explanation.
Real-world example: “Your loan application has been denied.” [End of message]
Technical term: This is an opaque model with zero interpretability.
Can you govern this? No. You can’t explain it, audit it, or improve it.
Appropriate for: Not appropriate for high-risk AI applications.
Level 1: Outcome Summary
What you get: A general category with no specifics.
Real-world example: “Your loan was denied due to credit risk factors.”
Technical term: This provides minimal explainability. It offers output categorization without reasoning.
Can you govern this? Barely. This satisfies basic disclosure requirements but fails to meet audit requirements. Which risk factors were considered? How were they weighted? You can’t perform model behavior verification.
Appropriate for: low-stakes, low-risk AI systems such as content recommendations.
Level 2: Processed Reasoning Summary
What you get: You receive high-level reasoning phases that have been filtered and simplified.
Real-world example: OpenAI o3-mini’s approach shows you that the AI “analyzed credit history,” “evaluated payment patterns,” and “compared to lending criteria,” but you can’t see the actual chain-of-thought. You only see these processed summaries.
Technical term: This is called summarized CoT or reasoning abstractions.
Can you govern this? Partially. This is better than black-box systems but insufficient for strict compliance. You can’t verify the actual reasoning logic. You only know what categories were considered.
Appropriate for: Medium-risk applications where full auditability isn’t legally mandated.
Level 3: Full Reasoning Transparency
What you get: A complete, inspectable chain of thought from start to finish.
Real-world example: DeepSeek-R1’s approach lets you see every reasoning token: “Analyzing credit history... found 2 late payments in 2024... each payment was 5-7 days late... industry standard threshold is 3+ late payments... conclusion: meets lending criteria... checking debt-to-income ratio... 28% vs. 43% threshold... passes income test... recommendation: approve with standard rate.”
Technical term: This is full CoT transparency with complete reasoning traces.
Can you govern this? Yes. This provides complete auditability. You can trace every decision, verify logic, catch errors (”those late payments were actually on-time per bank records”), and demonstrate compliance to regulators.
Appropriate for: High-risk AI systems, regulated industries, and enterprise applications where consequences matter.
Level 4: Encrypted/Hidden Reasoning
What you get: The reasoning model generated reasoning tokens, and you paid for them, but the chain-of-thought is deliberately encrypted.
Real-world example: OpenAI o1 (before February 2025) shows this approach. Your API bill shows charges for reasoning tokens, but you cannot access the CoT.
Technical term: This is called encrypted CoT or opaque reasoning.
Can you govern this? No. This is worse than black-box models because you know reasoning exists, but are architecturally prevented from accessing it. It provides zero interpretability despite paying for the reasoning process.
Appropriate for: There are no identified legitimate use cases in enterprise or high-risk AI applications.
The Governance Gap Assessment:
What explainability level is your AI at? Most organizations haven’t formally assessed this.
What level does your regulatory environment require? Healthcare, finance, and legal typically need Level 3, which is full CoT transparency.
Can you demonstrate compliance under audit? If you can’t produce reasoning traces showing how decisions were made, the answer is no.
The gap between what you have (often Level 1-2) and what you need (Level 3) is where compliance failures happen. This is the documentation gap that becomes an ethics gap.
P.S.
The conflict between encrypted reasoning and governance requirements is documented across multiple sources. Organizations must audit AI systems under federal banking regulations (OCC Bulletin 2011-12). The EU AI Act requires high-risk systems to be interpretable. Legal analyses from major law firms warn that black-box AI tools impede comprehensive security assessments.
Yet vendors are selling reasoning models where the reasoning itself is encrypted. Organizations are paying premium rates for reasoning tokens they cannot inspect. They’re deploying systems in high-stakes applications while the mechanism they need to audit is architecturally hidden from them. The documentation gap becomes a compliance gap: how do you build audit trails when the vendor has encrypted the reasoning you need to audit?
Three questions to assess your AI explainability this week:
Can you trace a decision back to the chain-of-thought that produced it?
If a regulator or auditor asked you to explain how your AI made a specific decision, could you provide reasoning traces?
Does your AI vendor contract guarantee access to CoT reasoning, or just final outputs?
Most companies answer “no” to at least two of these. That’s the compliance gap between what AI governance frameworks require and what encrypted reasoning provides.
Next week: How to audit your AI audit trail. A practical checklist for evaluating whether your AI documentation actually holds up under regulatory scrutiny. I’ll cover what constitutes sufficient auditability, how to test for interpretability, and where most organizations’ audit trails actually break down.
Want a deeper analysis on building AI governance infrastructure? Premium subscribers get monthly deep-dives on documentation frameworks that satisfy both regulators and your own risk teams. This includes templates for explainability documentation, audit-trail requirements, and model-behavior verification protocols that you can implement immediately.
The Documentation Engineer is a weekly newsletter that analyzes how AI companies explain—or fail to explain—their products to users. Every Wednesday.
Want to influence which companies I evaluate next? Let me know below.
If this issue helped you see AI documentation differently, consider BUYING ME A COFFEE. Your support makes it possible to keep publishing weekly, build the benchmark framework, and expand evaluations to more companies.
Sources & Further Reading:
Simon Willison, “Notes on OpenAI’s new o1 chain-of-thought models” (Sept 12, 2024)
EU AI Act, Article 13: Transparency and Provision of Information to Deployers
OCC Bulletin 2011-12: Sound Practices for Model Risk Management
TechCrunch, “OpenAI now reveals more of its o3-mini model’s thought process” (Feb 6, 2025)
Glossary: Building Your AI Vocabulary
Accountability: The assignment of responsibility for AI decisions and outcomes. In AI governance, this means identifying who is responsible when an AI system makes an error or causes harm.
Algorithmic Transparency: The degree to which the decision-making logic of an algorithm can be understood by humans. This is distinct from transparency about data sources or model architecture. This specifically refers to being able to see how decisions are made.
Audit Trail: A chronological record documenting the sequence of activities, decisions, or processes. In AI, this means logging how a model arrived at specific outputs, enabling post hoc review.
Auditability: The capability to review, verify, and validate how an AI system operates and makes decisions. An auditable system provides sufficient documentation and access for independent review.
Benchmark Performance: Standardized test scores are used to compare different AI models. Common benchmarks for reasoning models include AIME (mathematics), GPQA (science), and Codeforces (programming).
Black-Box Model: An AI system whose internal decision-making process is not visible or understandable to users. You can see inputs and outputs, but not the reasoning that connects them.
Chain-of-Thought (CoT): A reasoning technique where an AI model generates a sequence of intermediate reasoning steps before producing a final answer. Like “showing your work” on a math problem.
Chain-of-Thought Monitoring: The practice of reviewing an AI’s internal reasoning chain to check for safety issues, policy violations, or undesired behaviors. Currently, OpenAI performs this monitoring internally but doesn’t share the reasoning with users.
Compliance: Meeting the requirements of laws, regulations, and organizational policies. In AI, this often means demonstrating that systems meet explainability, fairness, and safety standards.
Competitive Advantage: In the AI context, proprietary techniques or approaches that give a company’s models superior performance. OpenAI cites this as a reason for hiding chain-of-thought reasoning.
Encrypted Reasoning: Chain-of-thought that is deliberately hidden from users, even though they pay for the computational cost of generating it.
EU AI Act: European Union legislation (entering enforcement August 2026) that regulates AI systems based on risk level, requiring transparency, accountability, and human oversight for high-risk applications.
Explainability: The ability to explain in human-understandable terms how an AI system reached a particular decision or output. A legal and ethical requirement for many AI applications.
Governance: The frameworks, policies, and processes used to ensure AI systems operate responsibly, ethically, and in compliance with regulations.
High-Risk AI Systems: Under the EU AI Act, AI applications that pose significant risks to health, safety, or fundamental rights. Includes AI used in hiring, credit decisions, medical diagnosis, and legal proceedings.
Inference: The process of using a trained AI model to make predictions or generate outputs. Also refers to the computational cost of model execution.
Inference Costs: The expenses associated with running AI models in production, typically measured in tokens processed or compute time used.
Interpretability: The degree to which a human can understand the cause of an AI’s decisions. This is more technical than explainability. Interpretability may involve understanding model internals, not just decision outcomes.
Large Language Model (LLM): AI models trained on massive text datasets to understand and generate human language. Examples include GPT-4, Claude, and Gemini.
Large Reasoning Model (LRM): A newer class of AI models specifically designed to perform multi-step reasoning, often using chain-of-thought techniques. Examples include OpenAI o1 and DeepSeek-R1.
Minimal-Risk AI Systems: Under the EU AI Act, AI applications with little to no risk to rights or safety. These are subject to minimal transparency requirements.
Model Architecture: The technical structure and design of an AI model. This describes how its components are organized and how data flows through it.
Model Behavior Verification: The process of confirming that an AI model is functioning as intended, following expected logic, and not exhibiting undesired behaviors.
Model Risk Management: Frameworks (especially in financial services) for identifying, measuring, and controlling risks associated with using quantitative models, including AI/ML models.
Opacity: The state of being difficult or impossible to understand or inspect. An opaque AI model is one whose decision-making process cannot be examined.
Open Source: Software whose source code is publicly available for inspection, modification, and distribution. DeepSeek-R1 is open source with an MIT license.
Reasoning Model: See Large Reasoning Model (LRM).
Reasoning Phases: Distinct stages in a chain-of-thought process. DeepSeek-R1 explicitly labels phases like problem definition, initial analysis (”Bloom Cycle”), self-verification, and final decision.
Reasoning Summaries: Processed, simplified descriptions of chain-of-thought reasoning. OpenAI’s o3-mini provides these instead of raw reasoning tokens.
Reasoning Tokens: The computational units representing individual steps in chain-of-thought reasoning. In encrypted reasoning systems, you pay for these tokens but cannot see them.
Reasoning Traces: The complete, detailed record of steps taken during chain-of-thought reasoning. Essential for auditability.
Rumination: In reasoning models, the tendency to generate excessively long or repetitive reasoning that doesn’t improve output quality. This is a known issue with some transparent reasoning models, such as DeepSeek-R1.
Token: The basic unit of text processing in language models. Roughly equivalent to a word or word fragment. Pricing is typically per million tokens.
Transparency: Openness about how an AI system works, what data it uses, and how it makes decisions. This can refer to model architecture, training data, or reasoning processes.
Transparent Reasoning Model: A reasoning model that exposes its chain-of-thought for inspection. DeepSeek-R1 is the primary example.