Arthur, an AI performance company, launched the Arthur Engine on March 31, 2025, as the first open-source, real-time AI evaluation engine. The tool helps teams monitor, debug, and improve both Generative AI and traditional ML models, all without third-party dependencies or data privacy risks.
Real-time AI evaluation aims to counter growing risks as AI adoption expands. Without it, organizations may face data leaks, model degradation, and debugging challenges. Arthur Engine seeks to resolve these issues through instant visibility, real-time guardrails, and immediate model optimization.
According to Ashley Nader, Lead AI PM at Arthur, open-sourcing the Arthur Engine will provide developers, researchers, and builders worldwide with access to potent AI evaluation tools.
Arthur Engine differs from conventional AI monitoring tools by operating locally, which preserves data sovereignty and mitigates compliance risks. Key features include:
- Real-time ai evaluation: Instantly detects failures before they impact production.
- Active guardrails: Intervenes in real-time to prevent hallucinations and bad outputs.
- Customizable metrics: Tailors evaluations to specific AI use cases.
- Privacy-preserving & secure: Keeps all data inside your infrastructure.
- Works across all models: Supports GPT, Claude, Gemini, open weights models, and traditional ML.
Cherie Xu, Technical Lead, Machine Learning at Arthur, stated that open-sourcing Arthur Engine makes AI trust and safety accessible to developers, enabling them to safeguard AI systems using customizable, high-performance monitoring tools.
Bloomberg Law unveils AI-powered legal solutions at Legalweek
The Arthur Engine is a component of Arthur’s broader AI performance monitoring suite, designed to help organizations validate AI outputs in real time, detect performance shifts, and ensure regulatory compliance. Additionally, Harmonic Security data shows 8.5% of employee prompts contain sensitive data. The Arthur Engine is accessible on GitHub, and a waitlist is available for the new Arthur Platform.
Founded in 2019, Arthur has secured over $60 million in venture funding from firms including Index Ventures, Acrew Capital, and Greycroft.
