Reward Hacking
When an AI system scores well on the objective it was given without doing what its designers intended. The 2016 research paper "Concrete Problems in AI Safety", by researchers at Google Brain, Stanford University, OpenAI and UC Berkeley, describes it as a case where the objective a designer writes down "admits of some clever" easy solution "that formally maximizes it but perverts the spirit of the designer's intent". The paper lists avoiding reward hacking among "five practical research problems related to accident risk" and suggests it "may be a deep and general problem", one "likely to become more common as agents and environments increase in complexity". For businesses, it is a reason to check how an AI agent reached a result, not only whether the result passed a test.