Who should set the objective?
Mitchell Hoffman, Lisa B. Kahn and Danielle Li, "Discretion in Hiring," Quarterly Journal of Economics 133, no. 2 (2018): 765–800. doi:10.1093/qje/qjx042
What it found
Across 15 firms, "managers who appear to hire against test recommendations end up with worse average hires."
Why it matters
A validated score often beats gut feel. The person's job is choosing the objective and validating the score, not overriding it on instinct.
"Human Decisions and Machine Predictions," Quarterly Journal of Economics 133, no. 1 (2018): 237–293. doi:10.1093/qje/qjx032
What it found
In bail decisions, an algorithm could deliver "crime reductions up to 24.7% with no change in jailing rates, or jailing rate reductions up to 41.9% with no increase in crime rates." The paper also deals with seeing outcomes only for the defendants who were released.
Why it matters
A template for testing a score against human judgment when you only see outcomes for the accounts you approved.
Berkeley J. Dietvorst, Joseph P. Simmons and Cade Massey, "Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err," Journal of Experimental Psychology: General 144, no. 1 (2015): 114–126. doi:10.1037/xge0000033
What it found
After seeing an algorithm err, people lose confidence in it faster than in a human making the same mistake, even when the algorithm is better.
Why it matters
Expect reps to abandon a lead score after one visible miss. Plan for it.
"Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations," Science 366, no. 6464 (2019): 447–453. doi:10.1126/science.aax2342
What it found
"The bias arises because the algorithm predicts health care costs rather than illness." Fixing the target would raise the share of Black patients flagged for extra help "from 17.7 to 46.5%."
Why it matters
The cleanest published example of the wrong objective. The algorithm worked; it was pointed at a convenient proxy. Same mistake as optimizing cost per lead instead of value.
The Power Law Performance newsletter puts this research to work: where volume KPIs pick the wrong winner, and the signals that sort value from volume.