Dataconomy
  • News
    • Artificial Intelligence
    • Cybersecurity
    • DeFi & Blockchain
    • Finance
    • Gaming
    • Startups
    • Tech
  • Industry
  • Research
  • Resources
    • Articles
    • Guides
    • Case Studies
    • Glossary
    • Whitepapers
  • Newsletter
  • + More
    • Conversations
    • Events
    • About
      • About
      • Contact
      • Imprint
      • Legal & Privacy
      • Partner With Us
Subscribe
No Result
View All Result
  • AI
  • Tech
  • Cybersecurity
  • Finance
  • DeFi & Blockchain
  • Startups
  • Gaming
Dataconomy
  • News
    • Artificial Intelligence
    • Cybersecurity
    • DeFi & Blockchain
    • Finance
    • Gaming
    • Startups
    • Tech
  • Industry
  • Research
  • Resources
    • Articles
    • Guides
    • Case Studies
    • Glossary
    • Whitepapers
  • Newsletter
  • + More
    • Conversations
    • Events
    • About
      • About
      • Contact
      • Imprint
      • Legal & Privacy
      • Partner With Us
Subscribe
No Result
View All Result
Dataconomy
No Result
View All Result

CrowdStrike and Meta launch open-source CyberSOCEval benchmark to test AI cybersecurity models

The tool allows organizations to measure large language model performance in incident response, threat analysis, and malware detection.

byAytun Çelebi
September 16, 2025
in Cybersecurity

Cybersecurity firm CrowdStrike and Meta have released CyberSOCEval, an open-source benchmark suite designed to evaluate the performance of AI models in security operations centers (SOCs). The tool aims to help businesses select the right AI-powered cybersecurity solutions by providing a standardized way to test their capabilities in key security tasks.

The challenge of choosing the right AI security tool

As AI becomes integrated into a growing number of cybersecurity products, security professionals face the challenge of choosing from a wide array of options with varying costs and capabilities. CyberSOCEval addresses this by offering a structured method for testing large language models (LLMs) on core SOC functions, including incident response, threat analysis, and malware detection.

Without clear benchmarks, it’s difficult to know which systems, use cases, and performance standards deliver a true AI advantage against real-world attacks.

By standardizing these evaluations, the benchmark allows organizations to objectively measure how different AI models perform in realistic scenarios, helping them identify the tools that best fit their operational needs.

Stay Ahead of the Curve!

Don't miss out on the latest insights, trends, and analysis in the world of data, technology, and startups. Subscribe to our newsletter and get exclusive content delivered straight to your inbox.

How CyberSOCEval benefits both businesses and developers

For businesses, the benchmark provides clear, comparable data on model performance. For AI developers, it offers valuable insights into how enterprise clients use their models for cybersecurity. This feedback can guide future improvements, helping creators refine their models to better handle specific industry jargon or complex threat intelligence. The framework is designed to be adaptable, allowing for the inclusion of new tests as threats like zero-day exploits emerge.

The release of CyberSOCEval comes amid a digital arms race where both attackers and defenders are leveraging AI. A survey by Mastercard and the Financial Times Longitude found that financial services firms have saved millions of dollars by using AI-powered tools to combat AI-enabled fraud, demonstrating the tangible benefits of effective defensive AI.

An open-source approach to improving security

Meta’s involvement in the project aligns with its history of supporting open-source AI development, such as its Llama models. By making CyberSOCEval an open-source tool, the companies encourage community collaboration to improve and expand the benchmarks over time. This approach aims to accelerate industry-wide progress in defending against advanced, AI-based threats.

With these benchmarks in place, and open for the security and AI community to further improve, we can more quickly work as an industry to unlock the potential of AI in protecting against advanced attacks, including AI-based threats.

CyberSOCEval is available now on GitHub, where users can download the suite to run evaluations on their preferred LLMs. The repository includes documentation, sample datasets, and instructions for integrating the tests into existing security platforms.


Featured image credit

Tags: CrowdstrikeCyberSOCEvalMeta

Related Posts

183M Gmail passwords exposed via infostealer malware

183M Gmail passwords exposed via infostealer malware

October 28, 2025
Google’s Live Threat Detection is reportedly coming to more Android phones

Google’s Live Threat Detection is reportedly coming to more Android phones

October 23, 2025
Meta’s latest update focuses on protecting older users from scams

Meta’s latest update focuses on protecting older users from scams

October 22, 2025
US judge bans NSO Group from targeting WhatsApp users with Pegasus spyware

US judge bans NSO Group from targeting WhatsApp users with Pegasus spyware

October 21, 2025
AWS outage: A complete list of every site and app that went down

AWS outage: A complete list of every site and app that went down

October 20, 2025
Windows 11’s new security patch is a system-breaking disaster, fix it now

Windows 11’s new security patch is a system-breaking disaster, fix it now

October 20, 2025

LATEST NEWS

183M Gmail passwords exposed via infostealer malware

Google’s AI health coach debuts for Android Fitbit users

Grokipedia’s “AI-verified” pages show little change from Wikipedia

OpenAI data reveals 0.15% of ChatGPT users express suicidal thoughts

OpenAI makes ChatGPT Go free across India

Goodbye: Pixel Watch gets its final update

Dataconomy

COPYRIGHT © DATACONOMY MEDIA GMBH, ALL RIGHTS RESERVED.

  • About
  • Imprint
  • Contact
  • Legal & Privacy

Follow Us

  • News
    • Artificial Intelligence
    • Cybersecurity
    • DeFi & Blockchain
    • Finance
    • Gaming
    • Startups
    • Tech
  • Industry
  • Research
  • Resources
    • Articles
    • Guides
    • Case Studies
    • Glossary
    • Whitepapers
  • Newsletter
  • + More
    • Conversations
    • Events
    • About
      • About
      • Contact
      • Imprint
      • Legal & Privacy
      • Partner With Us
No Result
View All Result
Subscribe

This website uses cookies. By continuing to use this website you are giving consent to cookies being used. Visit our Privacy Policy.