Notes from Weapons of Math Destruction
This is the convention being followed for all reading notes exported after January 31, 2023 (and some previous exports):
| KOReader/Exported | Kindle | Meaning |
|---|---|---|
| Lighten/Normal | Yellow | Quotables, concepts, and general ideas. |
| Underline | Orange | Further thought is required on this for clarity. |
| Highlighted/Bold | Blue | Something strikingly novel/Deeply moving/Highly thought-provoking. |
| Pink | In discord with this opinion. |
Weapons of Math Destruction
By Cathy O'Neil
Chapter 2: Shell Shocked: My Journey of Disillusionment
Page 55 @ November 9, 2019
I started out building models to anticipate the behavior of visitors to various travel websites. The key question was whether someone showing up at the Expedia site was just browsing or looking to spend money. Those who weren’t planning to buy were worth very little in potential revenue. So we would show them comparison ads for competing services such as Travelocity or Orbitz. If they clicked on the ad, it brought in a few pennies, which was better than nothing. However, we didn’t want to feed these ads to serious shoppers. In the worst case, we’d gain a dime of ad revenue while sending potential customers to rivals, where perhaps they’d spend thousands of dollars on hotel rooms in London or Tokyo. It would take thousands of ad views to make up for even a few hundred dollars in lost fees. So it was crucial to keep those people in house.
Wah!
Chapter 3: Arms Race: Going to College
Page 71 @ November 9, 2019
The response to this crackdown on cheating was volcanic. Some two thousand stone-throwing protesters gathered in the street outside the school. They chanted, “We want fairness. There is no fairness if you don’t let us cheat.”
Chapter 5: Civilian Casualties: Justice in the Age of Big Data
Page 104 @ November 12, 2019
So fairness isn’t calculated into WMDs. And the result is massive, industrial production of unfairness . If you think of a WMD as a factory, unfairness is the black stuff belching out of the smoke stacks. It’s an emission, a toxic one.
Conclusion
Page 210 @ November 12, 2019
Like doctors, data scientists should pledge a Hippocratic Oath, one that focuses on the possible misuses and misinterpretations of their models. Following the market crash of 2008, two financial engineers, Emanuel Derman and Paul Wilmott, drew up such an oath. It reads:
- I will remember that I didn’t make the world, and it doesn’t satisfy my equations.
- Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.
- I will never sacrifice reality for elegance without explaining why I have done so.
- Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.
- I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.