拓冰网站定制 ![[2023] [Safe RLHF] [Safe RLHF: Safe Reinforcement Learning from Human Feedback]](http://pic.xiahunao.cn/yaotu/[2023] [Safe RLHF] [Safe RLHF: Safe Reinforcement Learning from Human Feedback])
Safe RLHF: Safe Reinforcement Learning from Human Feedback, 2023在标注时进行解耦，标注员不用在有用性和安全性之间权衡使用拉格朗日方法求解约束优化问题奖励模型L R ( ϕ ; D R ) = − E ( x , y w , y l ) ∼ D R [ log σ ( R ϕ ( y w , x ) − R ϕ ( y l , x ) ) ] , (5) L_R(ϕ; D_R) =−E_{(x,y_w,y_l)∼D_R} \left[\log σ(R_ϕ(y_w,x)−R_ϕ(y_l,x))\right], \tag{5}L [2023] [Safe RLHF] [Safe RLHF: Safe Reinforcement Learning from Human Feedback]
Safe RLHF: Safe Reinforcement Learning from Human Feedback, 2023在标注时进行解耦，标注员不用在有用性和安全性之间权衡使用拉格朗日方法求解约束优化问题奖励模型L R ( ϕ ; D R ) = − E ( x , y w , y l ) ∼ D R [ log σ ( R ϕ ( y w , x ) − R ϕ ( y l , x ) ) ] , (5) L_R(ϕ; D_R) =−E_{(x,y_w,y_l)∼D_R} \left[\log σ(R_ϕ(y_w,x)−R_ϕ(y_l,x))\right], \tag{5}L