AI RESEARCH ANALYSIS
PETA: Privacy Enabled Testing for AI using PETs
Outsourcing testing of AI/ML models can have many privacy implications such as 1. leakage of training data used to train the model 2. leakage of model IP which can be proprietary. Similarly methods to evaluate AI/ML model cannot be shared with model owner as it can be proprietary as well. There are many research works that strongly argue about the need to evaluate ML models for functional and non functional requirements. However to our knowledge there isn't any work that provide theoretical and empirical analysis of these methods in privacy setting. To this end, we propose PETA, Privacy Enabled Testing for AI/ML. PETA, which is first-of-its-kind framework that, enables us to evaluate ML models in privacy setting using Privacy Enhancing Technologies (PETs) like FHE and MPC. Fully Homomorphic Encryption and Multi-Party Computation have their own advantages and limitations and hence evaluating ML models using these technologies requires designing efficient circuits, which is non trivial and requires considerable design effort. To demonstrate efficacy of our framework, we evaluate three testing methods of ML models, namely Calibration, Resilience and Fairness in privacy setting. We use the state of the art libraries for FHE and MPC to demonstrate feasibility of these techniques and analyze challenges ahead to make them practical.
Executive Impact Brief
The rapid advancement of AI/ML necessitates robust evaluation for trustworthiness, especially given privacy concerns. Traditional plaintext testing risks exposing sensitive data and model IP. This paper introduces PETA, a first-of-its-kind framework for Privacy-Enabled Testing for AI/ML, utilizing Fully Homomorphic Encryption (FHE) for Calibration and Secure Multi-Party Computation (sMPC) for Resilience and Fairness. PETA addresses critical challenges in secure AI model testing, balancing computational efficiency with stringent privacy requirements.
0
Privacy Preserved
0
Enhanced Reliability
0
Efficiency Gains (Masking)
0
Scalability for AI
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Calibration
Resilience
Fairness
Calibration ensures that an AI model's predicted probabilities accurately reflect the likelihood of outcomes, improving its reliability in critical applications like medical diagnosis or financial forecasting. PETA employs Fully Homomorphic Encryption (FHE) for secure calibration, preserving data and model IP. The BFGS algorithm, essential for temperature scaling, is adapted for FHE, requiring careful approximation of complex operations like logarithm, exponentiation, and division.
| Feature | FHE | MPC |
|---|---|---|
| Operations | Add, Mult | Add, Mult, Comparison |
| Complex operations | Limited through approximation | Fairly complex operations supported |
| Computation model | Outsourcing computations to a powerful server | Client computing complex functions along with server |
| Accuracy | Depends on quality of approximations | Minimal loss in terms of accuracy |
| Primary Bottleneck | Computation | Communication |
| Security Guarantees | Post Quantum secure | Post Quantum secure |
| Preferred computation type | Offloading one-off computations | Iterative computations can be done |
242.1s
Optimized BFGS Minimization Time (without bootstrapping)
FHE performance for complex operations like BFGS minimization is significantly impacted by bootstrapping. Minimizing bootstrapping reduces execution time, but may require additional client-server communication rounds for re-encryption, highlighting a crucial trade-off between computational efficiency and data security.
FHE Outsourcing Computation Flow
User Encrypts Data (X)
→
Server Computes on Encrypted Data (F(Enc(X)))
→
User Decrypts Result (F(X))
Resilience in AI models is the ability of a system to withstand and operate effectively despite perturbations or noise in the data. PETA addresses this using the Hop-Skip-Jump (HSJ) attack implemented via Secure Multi-Party Computation (sMPC). HSJ generates adversarial examples by adding minimal, carefully crafted noise to input images. MPC ensures the attack execution remains secure, preserving data privacy while enabling effective adversarial training and evaluation, particularly for complex iterative comparison operations where FHE is less suitable.
| Model Name | Time (hrs) | CIFAR_88 | |
|---|---|---|---|
| VGG | Without mask | 2.5 hrs | 0.36 hrs |
| VGG | With mask | 1.28 hrs | 0.12 hrs |
Crypten for Secure Resilience Testing
For resilience testing, we utilized Facebook's Crypten library, built on PyTorch, within an MPC setup. Crypten's Pythonic, high-level language significantly simplified porting complex HSJ attack algorithms from plaintext to the MPC domain. This choice allowed us to execute the HSJ attack securely, preserving data privacy while enabling effective adversarial training and evaluation on large, complex models like CIFAR and VGG-16.
Furthermore, applying perturbations only on significant image regions (via mask information) led to 50-66% reduction in processing time, demonstrating efficiency gains for practical privacy-preserving resilience testing.
Fairness in AI models involves addressing and mitigating bias to ensure equitable outcomes across different demographic groups. PETA evaluates fairness using a three-phase approach: Pre-processing (assigning weights to training examples), In-processing (training model with weights), and Post-processing (computing fairness metrics). All phases, except In-processing, are conducted in a Secure Multi-Party Computation (sMPC) environment using MP-SPDZ to ensure data privacy and prevent IP exposure of fairness metrics.
| Phase | Time (mins) | Communication per party (GB) |
|---|---|---|
| Pre-processing | 284.8 | 1.94 |
| In-processing | 19.5 (secs) | NA |
| Post-processing | 38.6 | 0.179 |
1.94 GB
Communication Cost for Pre-processing (per party)
Fairness evaluation using MPC incurs significant communication costs, particularly in the pre-processing phase (1.94 GB/party) due to complex comparison operations and statistical calculations. This highlights the overhead associated with ensuring fairness in a privacy-preserving setting, emphasizing the trade-off between privacy and communication efficiency.
Quantify Your AI ROI
Use our calculator to estimate the potential time savings and cost efficiencies your organization could achieve by implementing secure, trustworthy AI.
Strategic Implementation Roadmap
Our phased approach ensures a smooth, secure, and effective integration of privacy-enabled AI testing into your enterprise.
PETs Integration Assessment
Evaluate existing AI/ML models and data pipelines for PETs compatibility, identifying sensitive data points and potential integration challenges with FHE/MPC.
Custom Circuit Design & Optimization
Develop and optimize efficient approximated circuits for complex operations (e.g., log, exp, gradients) within FHE, or design secure multi-party protocols for iterative comparisons in MPC, balancing accuracy and performance.
Secure Testing Framework Deployment
Implement PETA's privacy-enabled modules (Calibration, Resilience, Fairness) using selected PETs. Establish secure client-server communication channels and conduct initial secure evaluations.
Continuous Trustworthiness Monitoring
Integrate secure testing into MLOps pipelines for ongoing evaluation. Refine PETs parameters, explore domain-specific optimizations (e.g., mask information), and expand to advanced models like LLMs.
Ready to Build Trustworthy AI?
Partner with us to secure your AI/ML models, ensure compliance, and achieve unparalleled trustworthiness with our privacy-enabled testing framework.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat