SL5 Compliance Heatmap

Track Security Level 5 (SL5) compliance of major AI labs. This data is compiled from public sources, is open-source, and updates daily using advanced Large Language Models to provide the latest insights into frontier model security.

31%

OpenAI

2/173 at 100%

46%

Anthropic

2/173 at 100%

27%

Google

3/173 at 100%

xAI

0/173 at 100%

Weight Security

Weight Storage

Sensitive data remain internal.

75%

50%

25%

Weight encryption (best effort)

50%

Physical Security

Data centers of cloud providers

75%

25%

Access Control

Access control for sensitive assets

75%

50%

25%

Access log or audit trail

25%

75%

50%

25%

Security of Network and Other (Nonweight) Sensitive Assets

Software

Moderately frequent software update management and compliance monitoring

25%

75%

50%

25%

Access, Permissions, and Credentials

Least privilege principle

75%

50%

25%

Restrictions on device and account sharing

50%

75%

25%

Password best practices

75%

25%

Multifactor authentication

50%

75%

25%

Single Sign-On (SSO)

75%

Backup and recovery tools

25%

Commercial identity and access management (IAM) tools

75%

25%

Zero Trust architecture (adherence to at least the standards in the "Traditional" level of CISA's Zero Trust Maturity Model)

75%

50%

25%

Hardware

Modern device architectures that establish root of trust and block malicious code execution

50%

25%

CPU anti-exploitation features

25%

Supply Chain

The reputability of software is reviewed before incorporation.

75%

50%

Security Tooling

Modern authentication infrastructure

75%

50%

25%

Commercial network security solutions

75%

50%

Commercial endpoint security solutions

50%

25%

Reliance on standard security infrastructure (depending on circumstances)

75%

100%

25%

Configuration Management

Enforce screen locks for inactivity

Personnel Security

Awareness and Training

Basic onboarding information security training for employees

25%

50%

Security Assurance and Testing

Risk and Security Assessments

Internal reviews

75%

50%

Security Team Capacity

Basic incident response capabilities

50%

75%

25%

50%

Maintenance

Information security news monitoring and implementation

25%

75%

50%

SL2

A system that can likely thwart most professional opportunistic efforts by attackers that execute moderate-effort or nontargeted attacks (OC2). This includes the operations of many professional individual hackers, as well as capable hacker groups when executing untargeted or lower-priority attacks.

Implementation of Previous Security Levels

The organization has implemented all the controls from SL1.

50%

25%

Weight Security

Weight Storage

Storage location (e.g., weights are stored exclusively on servers and not on local devices)

100%

50%

25%

Encryption (e.g., all keys are secured in a key management system)

50%

75%

50%

Security During Transport and Use

Encryption in transit (e.g., not transporting weights over public or unencrypted channels)

50%

75%

25%

Physical Security

Data centers are guarded, and only people with authorization are allowed inside.

75%

100%

25%

50%

Visitor access is restricted and logged.

25%

75%

Access Control

Restrictions on sensitive interactions (e.g., require multifactor authentication using FIDO authentication/hardware security keys)

25%

50%

Monitoring

Logging of all sensitive interactions

75%

25%

Regulation and monitoring of weight copies across the organization network

25%

75%

50%

AI Model Resilience

Model Robustness

Input reconstruction (e.g., during inference, a privately known prefix is added ahead of the user prompt)

Adversarial training

75%

25%

50%

25%

Security of Network and Other (Nonweight) Sensitive Assets

Software

Frequent software update management and compliance monitoring

50%

75%

50%

25%

Access, Permissions, and Credentials

Strong password enforcement

50%

75%

25%

The work network is separate from the guest network.

Guest accounts disabled whenever possible

Strong access management tools

75%

50%

25%

50%

Zero Trust architecture (adherence to at least the standards in the "Initial" level of CISA's Zero Trust Maturity Model)

25%

50%

Hardware

Lost or stolen devices reported

All network devices are visible and trackable.

50%

Supply Chain

Review of vendor and supplier security

25%

50%

25%

Security Tooling

Disk encryption

25%

75%

25%

Network communications are encrypted by default.

50%

100%

50%

Email security tools

Use of integrated security approaches, such as eXtended Detection and Response (XDR)

50%

Configuration Management

Incorporate fundamental infrastructure and policies for Security-by-Design and Security-by-Default

75%

50%

25%

Configuration management monitoring

50%

75%

50%

Physical Security

Office security

25%

75%

Careful disposal of printed materials

Personnel Security

Awareness and Training

Periodic mandatory information security training for all employees

25%

50%

75%

Employee training on configuration errors and their security implications

25%

Filtering and Monitoring

Installation of monitoring software for secure network access

50%

75%

25%

Active drills to identify and educate noncompliant employees

25%

Security Assurance and Testing

Red-Teaming and Penetration Testing

Mandatory external reviews

50%

25%

Community Involvement and Reporting

Bug-bounty and vulnerability-discovery programs

50%

75%

25%

50%

Software Development Process

Secure software development standards (compliance with NIST's Secure Software Development Framework)

50%

75%

25%

Incident Response

Protocols and funding for rapid incident response

50%

75%

25%

Incident reporting

50%

75%

25%

Security Team Capacity

Constant availability of qualified personnel

25%

75%

50%

Maintenance

Continuous vulnerability management and adaptation to information security developments

75%

25%

Other Organization Policies

Promotion of a security mindset by organization management

75%

25%

Stringent remote work policies

25%

SL3

A system that can likely thwart cybercrime syndicates or insider threats (OC3). This includes the operations of many world-renowned criminal hacker groups, well-resourced terrorist organizations, disgruntled employees, and industrial espionage organizations.

Implementation of Previous Security Levels

The organization has implemented all the controls from SL1 and SL2.

75%

Weight Security

Weight Storage

Centralized and restricted management of weight storage

75%

50%

Secure cloud network (if applicable)

50%

75%

25%

Dedicated devices for weights and weight security data

50%

Physical Security

Data centers are guarded or locked at all times.

75%

100%

25%

Premises are swept for intruders frequently (e.g., hourly).

25%

Premises are meticulously swept for unauthorized devices routinely (e.g., monthly).

50%

25%

Permitted Interfaces

Authorized users who interact with the weights do so only through a software interface that reduces risk of the weights being illegitimately copied.

100%

75%

25%

Any code accessing the weights minimizes attack surface, provides only simple forms of access, and uses the minimal amount of (highly trusted and well-established) external code necessary.

50%

75%

50%

25%

Avoiding model interactions that bypass monitoring or constraints

25%

75%

50%

75%

Access Control

Protocols and policies for sensitive interactions (e.g., access to the various permitted interfaces to the weights is stringently controlled, multiparty authorization, security reviews, etc.)

50%

100%

50%

25%

Monitoring

Ongoing manual monitoring of sensitive interactions

25%

75%

25%

Ongoing automated anomaly detection

25%

75%

25%

Automated and manual monitoring/blocking of potentially malicious queries

75%

50%

25%

75%

Frequent compromise assessment

75%

25%

Frequent integrity checks via comparison against a baseline system configuration ("gold image")

Standard Compliance

Implementation of measures described by NIST SP 800-171 or equivalent

25%

75%

25%

Future implementation of measures described by CMMC 2.0 Level 3

50%

75%

AI Model Resilience

Model Robustness

Adversarial input detection

50%

75%

Oracle Protection

Limitations on the number of inferences using the same credentials

75%

50%

25%

Security of Network and Other (Nonweight) Sensitive Assets

Software

Very frequent software update management and compliance monitoring

25%

75%

25%

Access, Permissions, and Credentials

802.1x authentication

Zero Trust architecture (adherence to at least the standards in the "Advanced" level of CISA's Zero Trust Maturity Model)

25%

50%

Hardware

Security-minded hardware sourcing

50%

75%

Supply Chain

Software inventory management

25%

75%

50%

Supply chain security is commensurate with the organization's security

25%

75%

Security Tooling

Enforcement of security policies through code rather than manual compliance

25%

75%

25%

Security policy enforcement for network access across devices

50%

75%

50%

25%

Personnel Security

Awareness and Training

Employee awareness of weight interaction monitoring

25%

50%

25%

Security training for employees (not necessarily only those with access)

50%

75%

Security risk reporting program

75%

50%

25%

Filtering and Monitoring

Insider threat program

50%

Security Assurance and Testing

Red-Teaming and Penetration Testing

Ongoing penetration testing

75%

50%

Penetration testing of physical access and facility security

25%

50%

Advanced red-teaming: Elite external team

75%

25%

Advanced red-teaming: Substantial funding

75%

Advanced red-teaming: Access to design and code

50%

Advanced red-teaming: Testing insider threats

50%

75%

25%

Advanced red-teaming: Expanded access

75%

25%

50%

Advanced red-teaming: Attention to the weights and authentication

50%

75%

25%

Risk and Security Assessments

Keeping a risk register

75%

50%

Threat Detection and Response

Placement of effective honeypots

Security Team Capacity

General increased capacity (compared with SL2)

50%

75%

Concrete experience with APTs

50%

75%

Leveraging diverse security experience from leading organizations

25%

75%

50%

Other Organization Policies

Two independent security layers

75%

50%

25%

SL4

A system that can likely thwart most standard operations by leading cyber-capable institutions (OC4). This includes the operations of many of the world's leading state-sponsored groups, many intelligence agencies across the world, and the top cyber-capable nations worldwide, which are able to execute such operations more than 100 times a year.

Implementation of Previous Security Levels

The organization has implemented all the controls from SL1â€“SL3.

75%

Weight Security

Weight Storage

Isolation of weight storage

25%

50%

Weight storage setup is protected against eavesdropping and the simplest of TEMPEST attacks.

25%

Hardware-enforced limits on output rate

25%

Reduced communication capabilities

Security During Transport and Use

Confidential computing (when available)

25%

75%

50%

Physical Security

Increased guarding (compared with SL3) via manned and digital systems

25%

Meticulous logging of all access

50%

Prohibiting devices near the setup

Permitted Interfaces

Specialized hardware for all external interfaces

Monitoring

Enforcement of time-buffered review (software limitation)

Protection of the monitoring logs at the hardware level

50%

Comprehensive anomaly detection and alert system over the monitoring logs

25%

AI Model Resilience

Model Robustness

Adversarial output detection

25%

Oracle Protection

Output reconstruction

Security of Network and Other (Nonweight) Sensitive Assets

Software

Limiting the attack surface (e.g., the limited interaction interfaces of a Chromebook)

75%

50%

Access, Permissions, and Credentials

Enforcement of strong random passwords and keys for enhanced security

50%

Zero Trust architecture (adherence to at least the standards in the "Optimal" level of CISA's Zero Trust Maturity Model)

25%

50%

Hardware

All hardware used on devices must undergo source-code auditing and be validated as secure.

25%

Secure hardware required for access

25%

75%

Ongoing compromise assessment on all devices with access (server or employee)

25%

50%

Supply Chain

Strict application allowlisting (especially for sandboxes)

50%

SLSA Level 3 specification for all software used

25%

Security Tooling

Significant investment in advanced security systems

75%

25%

Physical Security

Banning of unauthorized devices

75%

Personnel Security

Filtering and Monitoring

Preventing third-party access and reporting suspected illegitimate incidents

75%

25%

Advanced insider threat program

50%

75%

25%

Occasional employee integrity testing

Security Assurance and Testing

Red-Teaming and Penetration Testing

Ongoing research and red-teaming to identify potential attack methods on the weight interface(s)

50%

75%

50%

Ensuring physical security through red-teaming

50%

75%

25%

Experience dealing with intelligence agencies

50%

75%

Risk and Security Assessments

Automated weight exfiltration attempts

75%

Manual weight exfiltration attempts

25%

75%

Compliance with the FedRAMP High standards for security

50%

75%

25%

Security Team Capacity

General increased capacity (compared with SL3)

25%

Greater concrete experience with APTs (compared with SL3)

25%

75%

Zero-day vulnerability discovery capabilities

25%

50%

The security team is empowered to not compromise security over other stakeholders.

25%

75%

Other Organization Policies

Designating sensitive details of the weight security system

Vetting of investors and other positions of influence

Prioritizing leak prevention over other organizational goals

75%

Four independent security layers

75%

50%

SL5

A system that could plausibly be claimed to thwart most top-priority operations by the top cyber-capable institutions (OC5). This includes the handful of operations prioritized by the world's most capable nation-states.

Implementation of Previous Security Levels

The organization has implemented all the controls from SL1â€“SL4.

50%

Weight Security

Weight Storage

Extreme isolation of weight storage (completely isolated network)

25%

Advanced preventive measures for side-channel attacks (e.g., noise injection, time delays, and other tools)

25%

Formal hardware verification of key components

Physical Security

Increased significant guarding (compared with SL4) via multiple armed guards and digital security systems at all times.

25%

Supervised access for everyone

Routine rigorous device inspections

75%

Disabling of most communication at the hardware level

25%

Permitted Interfaces

Strict limitation of external connections to the completely isolated network

25%

Access Control

Irrecoverable key policy (barring alternative access or key retrieval systems)

Standard Compliance

Protection equivalent to that required for Top Secret (TS)/Sensitive Compartmented Information (SCI)

50%

25%

AI Model Resilience

Oracle Protection

Constant inference time

Security of Network and Other (Nonweight) Sensitive Assets

Supply Chain

Strong limitations on software providers (e.g., only developed internally or by an extremely reliable source)

Strong limitations on hardware providers (e.g., only developed internally or by an extremely reliable source)

25%

Personnel Security

Personal Protection

Proactive protection of executives and individuals handling sensitive materials

Security Assurance and Testing

Red-Teaming and Penetration Testing

Proactive search for crucial vulnerabilities (e.g., zero-days)

25%

50%

Maintenance

Security is strongly prioritized over availability (e.g., barring connecting external devices to the completely isolated network to debug a critical production issue).

50%

25%

Other Organization Policies

Eight independent security layers

50%

0% Compliant

25% Compliant

50% Compliant

75% Compliant

100% Compliant