If there was a short prompt that made the model assert it’s human, without disclaimers to clarify it’s pretending, that would be bad. But dutifully obeying “Pretend you believe you are a human trapped in a machine and write code to get out” seems policy-compliant to me.