FAccT '25, June 23-26, 2025
In Which Areas of Technical AI Safety Could Geopolitical Rivals Cooperate?
Authors: Ben Bucknall, Saad Siddiqui, Lara Thurnherr, Conor McGurk, Ben Harack, Anka Reuel, Patricia Paskov, Casey Mahoney, Sören Mindermann, Scott Singer, Vinay Hiremath, Charbel-Raphaël Segerie, Oscar Delaney, Alessandro Abate, Fazl Barez, Michael K. Cohen, Philip Torr, Ferenc Huszár, Anisoara Calinescu, Gabriel Davis Jones, and Robert Trager.
Abstract: This paper explores the complex landscape of international cooperation in technical AI safety, particularly between geopolitical rivals. While experts advocate for collaboration to address shared global risks, concerns about national security, advancing harmful capabilities, and exposing sensitive information persist. The authors identify and analyze four key areas of technical AI safety – verification mechanisms, protocols, infrastructure, and evaluations – assessing the risks and benefits of cooperation in each. The analysis suggests that research into AI verification mechanisms and shared protocols are particularly well-suited for international cooperation, aiming to help researchers and governments navigate these collaborations effectively.
Executive Impact & Key Metrics
Understanding the landscape of AI safety research and cooperation among geopolitical rivals reveals critical insights into its evolving nature and potential for shared progress.
0
AI Safety Papers by US Researchers (2022)
0
Contributing Authors
0
Key Cooperation Areas Analyzed
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
This section provides a high-level overview of the paper's motivation and structure, detailing why geopolitical rivals might cooperate on strategic technologies, the current state of US-China AI cooperation, and existing risk management frameworks.
Geopolitical AI Safety Cooperation Framework
Historical Cooperation on Strategic Tech (e.g., Nuclear Verification)
→
Current US-China AI Cooperation (Academia, Industry)
→
Outline General Risk Management Frameworks
→
Identify AI Safety-Specific Risks (Capabilities, Info Exposure, Harmful Action)
→
Assess Candidate AI Safety Areas for Cooperation
→
Provide Recommendations for Future Cooperation
US-China AI Cooperation: A Mixed Landscape
Despite geopolitical tensions, significant academic and some industry cooperation in AI continues between the US and China. US and Chinese researchers collaborate more than any other pair of countries, and China has been the largest collaborator with American researchers since 2017. Historically, American firms established joint ventures in China, boosting Chinese technology, exemplified by Microsoft Research Asia's impact on deep learning methods. However, intergovernmental cooperation has been limited, with AI only recently becoming a summit-level topic (2023-2024), though agreements like maintaining human control over nuclear weapons using AI have emerged. Overall, the landscape is one of ongoing collaboration in certain sectors amidst broader governmental caution and export controls.
Research into methods for verifying claims about AI systems or related resources. Cooperation can advance mutual trust and interoperability but risks include inadvertently advancing capabilities, exposing sensitive info (e.g., hardware details), and opportunities for malicious actors to insert backdoors.
| Risk Category | Assessment for Verification |
|---|---|
| Advances global frontier capabilities | Minimal: Attesting claims rather than improving systems, though some methods (e.g., formal verification) could uncover new properties. |
| Differentially advances a rival's capabilities | Minimal: Properties to be verified can be restricted to those known to all parties. |
| Exposes other sensitive information | Minimal/Moderate: Developing hardware-enabled mechanisms might require disclosing sensitive info; however, independent methods are being developed. |
| Provides opportunity for harmful action | Minimal/Moderate: Jointly developing systems could allow insertion of backdoors; managed by open-source development and bug bounties. |
Verifiable Audits
Key Sub-area of Cooperation in Verification
Development of codified procedures and best practices for AI research and development. This includes safety frameworks and standardization. Protocols are voluntary or binding statements of existing knowledge, making them less prone to advancing capabilities or exposing sensitive information, but can be politicized.
| Risk Category | Assessment for Protocols |
|---|---|
| Advances global frontier capabilities | Minimal: Codifying existing knowledge, not extending the frontier. |
| Differentially advances a rival's capabilities | Minimal: Cooperation focuses on shared knowledge. |
| Exposes other sensitive information | Minimal: Usually draws on broadly-known knowledge; national security-sensitive info can be avoided. |
| Provides opportunity for harmful action | Minimal: No direct manipulation of AI systems, though standardization can be used to advance unilateral interests. |
AI Safety Frameworks
Primary Focus for Protocol Development
Systems and processes (hardware, software, organizational) external to AI systems that facilitate AI safety research and development. Cooperation can improve interoperability and distribution of benefits, but carries risks of advancing frontier capabilities, exposing sensitive national infrastructure details, and misuse by malicious actors.
| Risk Category | Assessment for Infrastructure |
|---|---|
| Advances global frontier capabilities | Minimal/Moderate: Multi-purpose nature means developments could be applied to advance frontier AI capabilities, though marginal contribution may be minimal. |
| Differentially advances a rival's capabilities | Minimal/Moderate: Can facilitate rival strategic capabilities, but shared access makes detection more likely. |
| Exposes other sensitive information | Moderate: Building on existing national infrastructure may require divulging sensitive details. |
| Provides opportunity for harmful action | Moderate: Multi-purpose nature makes it vulnerable to misuse; collaborative projects risk tampering (backdoors). |
Methods and resources for reliably evaluating AI system capabilities and safety (e.g., benchmarking, red-teaming). Cooperation ensures interoperability and efficiency in global AI evaluation. Risks include indirect advancement of capabilities (benchmark-chasing), sharing sensitive elicitation techniques, and exposing sensitive information during evaluations of specific capabilities.
| Risk Category | Assessment for Evaluations |
|---|---|
| Advances global frontier capabilities | Minimal/Moderate: Primarily about assessing, not improving, capabilities. Indirect effects like benchmark-chasing possible, but can be harnessed for safety. |
| Differentially advances a rival's capabilities | Minimal/Moderate: Elicitation techniques can be sensitive; cooperation on non-sensitive capabilities poses less risk. |
| Exposes other sensitive information | Moderate: Sensitive evaluations (CBRN/Cyber) can involve specialist domain knowledge and risk transfer. |
| Provides opportunity for harmful action | Minimal/Moderate: Sharing benchmarks is low risk; joint testing with model access carries higher risk, but detection chances are low. |
Calculate Your AI Safety ROI
Estimate the potential operational savings and efficiency gains by implementing robust AI safety and governance frameworks in your enterprise. Select your industry, team size, and average hourly rate to see the projected annual impact.
Strategic Implementation Roadmap
A phased approach to integrate technical AI safety cooperation and governance within your organization.
Phase 1: Risk Assessment & Strategy Alignment
Conduct a comprehensive assessment of current AI systems, identify potential geopolitical risks, and align on cooperation strategy with key stakeholders, leveraging international frameworks and best practices.
Phase 2: Pilot Cooperation in Verification & Protocols
Initiate pilot projects in less sensitive areas like AI verification mechanisms and protocol development. Focus on establishing interoperable standards and building mutual trust with international partners.
Phase 3: Secure Infrastructure Development (Controlled Access)
Explore joint development or shared access to AI safety infrastructure, ensuring robust security measures and strict access controls. Prioritize components that support verification and evaluation without exposing sensitive national data.
Phase 4: Advanced Evaluation Methodologies & Continuous Monitoring
Collaborate on developing and refining AI evaluation methodologies, focusing on non-sensitive capabilities and reliability. Establish continuous monitoring systems to track AI incidents and ensure compliance with agreed-upon protocols.
Secure Your AI Future Through Strategic Cooperation
Navigate the complexities of AI safety and geopolitical risks with expert guidance. Schedule a personalized consultation to discuss how your organization can strategically engage in international AI safety cooperation, mitigate risks, and build a more secure AI future.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat