Design RLHF Failure Benchmark

I want to commission a small-scale benchmark that reliably exposes where reinforcement-learning-from-human-feedback (RLHF) agents break down when the information they receive is partial, ambiguous, or internally inconsistent... (Budget: €30 - €250 EUR, Jobs: Artificial Intelligence, Data Analysis, Deep Learning, Documentation, Machine Learning (ML), Python, Reinforcement Learning, RLHF)