OpenAI’s newly introduced internal scale aims to evaluate its AI systems’ progress and capabilities systematically:
| Level | Capabilities |
| Level 1 | Engages in simple conversational tasks, similar to current chatbots like ChatGPT |
| Level 2 | Solves basic problems at the level of a PhD holder |
| Level 3 | Takes actions on behalf of users, demonstrating practical utility |
| Level 4 | Creates novel solutions and innovations, exhibiting creativity and adaptability |
| Level 5 | AGI – Performs tasks equivalent to entire organizations, surpassing human-level performance across various tasks |
This scale, ranging from Level 1 to Level 5, seeks to track the progression towards achieving Artificial General Intelligence (AGI) — the holy grail of AI development where machines exhibit human-like cognitive abilities.
Understanding OpenAI’s five levels of AI development
Here’s a detailed breakdown of how each level is defined and the criteria used to assess the power of AI systems:
Level 1: Basic conversational AI
AI systems at this level can engage in simple conversational tasks, akin to current chatbots like ChatGPT.
Assessment criteria:
- Natural Language Processing (NLP) Skills: Ability to understand and generate human-like text responses.
- Basic task performance: Execution of simple tasks such as answering questions, providing information, and engaging in basic dialogue.
- Contextual understanding: Limited capability to maintain context over short interaction
Level 2: Advanced problem-solving AI
AI systems at this level are capable of solving basic problems at the level of a person with a PhD.
Assessment criteria:
- Complex problem solving: Ability to tackle academic and theoretical problems in specific domains.
- Specialized knowledge: Depth of understanding in particular fields, demonstrating expertise similar to a doctoral level.
- Analytical skills: Proficiency in performing detailed analysis and generating well-founded conclusions.
Level 3: Autonomous action AI
AI agents at this level can take autonomous actions on behalf of users.
Assessment criteria:
- Decision-making: Capability to make informed decisions based on given data and predefined goals.
- Task automation: Execution of tasks without human intervention, showing autonomy in various applications.
- User interaction: Effectiveness in interacting with users to gather necessary information and perform actions accordingly.
Level 4: Innovative AI
AI systems at this level can create new innovations and exhibit creativity and adaptability.
Assessment criteria:
- Innovation generation: Ability to develop novel solutions and ideas that are original and valuable.
- Adaptive learning: Capacity to learn and adapt from new information and experiences, improving over time.
- Creative problem solving: Demonstrating ingenuity in approaching and resolving complex issues.
Level 5: AGI (Artificial General Intelligence)
The final level represents AI that can perform the work of entire organizations, surpassing human-level performance in most economically valuable tasks.
Assessment criteria:
- Broad Skillset: Mastery across a wide range of tasks and domains, demonstrating versatility and comprehensive knowledge.
- Economic Value: Capability to generate significant economic value by performing complex tasks more efficiently than human teams.
- Autonomous Operation: High degree of autonomy, managing and executing large-scale operations without human oversight.
- Generalization: Proficiency in applying knowledge and skills to unfamiliar problems and contexts, showcasing true general intelligence.
How is OpenAI so confident about these levels?
To ensure the accuracy and reliability of its AI power scale, OpenAI plans to conduct rigorous internal evaluations of its AI systems through several key methods.
Benchmark testing involves standardized tests designed to measure specific capabilities and performance metrics aligned with each level’s criteria. These tests provide a consistent framework for evaluating AI systems, ensuring objective assessments and identifying areas for improvement.
Expert review engages domain experts to assess the AI systems’ performance in specialized fields. These experts ensure thorough and accurate evaluations, validating that the AI meets high standards required for each level.
Real-world scenarios test AI systems in practical applications to validate their effectiveness and reliability. This approach allows OpenAI to observe how systems perform in dynamic environments, ensuring robustness and practical utility.
User feedback involves collecting and analyzing feedback from users interacting with AI systems. This feedback provides insights into practical utility and user satisfaction, highlighting strengths and areas for improvement.
By combining these methods, OpenAI aims to thoroughly evaluate and verify its AI systems, ensuring they meet the criteria for each level of the power scale and driving progress towards achieving Artificial General Intelligence (AGI).
All images are generated by Eray Eliaçık/Bing
