What is Al "reward hacking"—and why do we worry about it?

At the end of the day modern AI are optimization algorithims that will try to overfit and cheat their way to whatever reward we use