Provable inference: is your AI really your AI?
Recent technology disruptions and cybersecurity incidents around the world have highlighted how vulnerable modern digital infrastructure can be. From widespread service outages affecting major companies and financial institutions to cyberattacks involving AI-powered tools, organizations face growing challenges in protecting their systems and ensuring operational reliability.
As artificial intelligence becomes increasingly integrated into business processes and critical infrastructure, security requires more than protecting systems against unauthorized access. Organizations also need ways to verify that AI services operate as intended and deliver results using the models they claim to run.
Addressing this challenge, Anthropic has introduced FlashProve, an open-source system that uses cryptographic proofs to verify AI model inference without revealing proprietary model weights.
Users generally cannot independently confirm which model processes their requests. An AI provider could secretly substitute a smaller model to reduce costs, or an attacker could tamper with the inference process.
Anthropic’s approach, known as provable inference, uses cryptographic proofs to establish that a specific model generated a response based on the original input. This provides an additional layer of transparency for businesses and AI developers.
FlashProve relies on zero-knowledge succinct non-interactive arguments of knowledge (zkSNARKs), a cryptographic technique that allows computations to be verified without revealing confidential information.
The system operates in two stages:
- Model commitment: The model provider creates a cryptographic fingerprint of the model’s weights, identifying a specific version without disclosing its parameters.
- Inference verification: For each request, the provider generates a proof that the response was produced by the committed model using the original input.
Users can verify these proofs without accessing the model or reproducing its computations. This helps ensure that the claimed model was actually used while keeping its weights private.
Anthropic tested FlashProve on Llama 4 Maverick, a 400-billion-parameter mixture-of-experts (MoE) model, using 4-bit quantized weights.
Cryptographic verification could improve the security and accountability of AI services in several areas:
- AI security: Detect unauthorized model substitutions and certain forms of inference manipulation.
- Model provenance: Verify that an AI provider uses the model it claims to offer, rather than an undisclosed alternative.
- Regulatory compliance: Confirm that AI systems use approved models and meet specific requirements.
- AI safety monitoring: Support independent oversight of AI infrastructure and the deployment of advanced models.
These capabilities could be especially valuable in regulated industries such as healthcare and financial services, where model integrity and accountability are essential.
Anthropic has released FlashProve as an open-source research prototype under the Apache 2.0 license. The release includes executable code, a technical report, a protocol specification, and formal mathematical proofs written in Lean.
The current implementation has limitations: it supports inference contexts of up to 1,024 tokens, uses integer-based model execution with greedy decoding, and does not conceal the model’s architecture. Longer context windows and additional capabilities remain areas for future research.