当前位置：首页 > news >正文

AI应用的安全工程：从威胁建模到防护

news 2026/6/10 16:58:43

AI应用的安全工程：从威胁建模到防护

前言

我们的产品上线后，遇到了各种安全问题：Prompt 注入、数据泄露、API 滥用...

今天，分享我们是如何建立安全工程体系的。

一、威胁建模

1.1 威胁识别

class ThreatModeling: def identify(self) -> list: """识别威胁""" return [ {"type": "prompt_injection", "severity": "high"}, {"type": "data_leakage", "severity": "high"}, {"type": "api_abuse", "severity": "medium"}, {"type": "model_manipulation", "severity": "medium"} ]

1.2 风险评估

class RiskAssessment: def assess(self, threats: list) -> dict: """评估风险""" prioritized = sorted(threats, key=lambda x: x["severity"], reverse=True) return { "critical": [t for t in prioritized if t["severity"] == "high"], "medium": [t for t in prioritized if t["severity"] == "medium"] }

二、安全防护

2.1 输入验证

class InputValidation: def validate(self, input_data: str) -> dict: """验证输入""" checks = [ {"name": "length", "passed": len(input_data) < 5000}, {"name": "patterns", "passed": not self._contains_suspicious(input_data)}, {"name": "sanitization", "passed": True} ] return {"valid": all(c["passed"] for c in checks), "checks": checks}

2.2 输出过滤

class OutputFiltering: def filter(self, output: str) -> str: """过滤输出""" # 移除敏感信息 output = self._remove_personal_info(output) # 检查内容安全性 if self._contains_harmful(output): return "抱歉，无法生成相关内容" return output

三、API 安全

3.1 认证授权

class Authentication: def authenticate(self, token: str) -> dict: """认证用户""" return { "user_id": "123", "role": "user", "valid": True } def authorize(self, user_id: str, resource: str) -> bool: """授权检查""" permissions = {"user": ["read"], "admin": ["read", "write"]} return resource in permissions.get(self._get_role(user_id), [])

3.2 限流保护

class RateLimiting: def __init__(self): self.limits = {"free": 100, "pro": 1000} def check(self, user_id: str, plan: str) -> bool: """检查限流""" current = self._get_request_count(user_id) return current < self.limits.get(plan, 100)

四、数据安全

4.1 数据加密

class DataEncryption: def encrypt(self, data: str) -> str: """加密数据""" from cryptography.fernet import Fernet key = Fernet.generate_key() f = Fernet(key) return f.encrypt(data.encode()).decode() def decrypt(self, encrypted: str) -> str: """解密数据""" from cryptography.fernet import Fernet key = self._get_key() f = Fernet(key) return f.decrypt(encrypted.encode()).decode()

4.2 访问控制

class AccessControl: def check(self, user_id: str, resource: str) -> bool: """检查访问权限""" return resource == "own_data" or self._is_admin(user_id)

五、安全监控

5.1 异常检测

class AnomalyDetection: def detect(self, activity: dict) -> dict: """检测异常""" anomalies = [] if activity["requests_per_minute"] > 100: anomalies.append("请求频率异常") if activity["unusual_pattern"]: anomalies.append("行为模式异常") return {"anomalies": anomalies, "risk_level": "high" if anomalies else "low"}

5.2 告警机制

class Alerting: def trigger(self, alert: dict): """触发告警""" return { "type": alert["type"], "severity": alert["severity"], "message": alert["message"], "actions": ["通知安全团队", "临时限流"] }