Kevin Drum raises some interesting questions about the change in meaning of some legal terms when computers are used in the decision:
The nice thing about probable cause and reasonable suspicion and other similar phrases is that they have a long history behind them. There are hundreds of years of statutory definition and case law that define what they mean, and human judges interpret them in ways that most of us understand….
But the NSA’s domestic spying program doesn’t rely on the ordinary human understanding of these phrases. Instead, it appears to rely primarily on software algorithms that determine whether or not a person is acting in a way that merits eavesdropping.
Can’t say as I agree. The terms are understandable by humans, and the computer algorithms were written by humans. Human cops use an algorithm to determine whether they have probable cause to seach a car at a traffic stop.
In fact, isn’t it easier to determine the factors that go into a computer algorithm than the factors that go into the algorithm used by a human cop’s brain?
If a cop is racist, for instance, you need to prove that in court by a preponderance of the evidence. If the algorithm is racist, you could probably prove it via code review, rather than focusing on the results…
Not so much. It’s not uncommon for a policy to be found racist in court because of its effects rather than its stated intent. To use a ridiculous example, profiling people under 5’6″ would unfairly target women and older Asians.
If anything, it’s much easier to hide this sort of thing in code, IMO. I know all sorts of tricks to make my code do stuff that most people can’t find unless they track it line-by-line.
More to the point, the courts haven’t ruled on a algorithmic definition of “reasonable”, and since human judgment is part of that legal definition, one could argue that that should never be applied to an expert system.