Yeah. I love HN and many of the thoughts expressed on it, but if there's one thing that always jumps out at me as being an incredibly widely held cognitive bias in the tech community, it's a tendency to view certain things in a very black and white, binary fashion, and to treat certain very complicated, messy problems with lots of social nuances as if they were simple problems, if only everyone could be rational about it. Grades and hiring are obvious examples, but you can also see plenty of it in discussions about open source business models and codes of conduct and copyright and so on.
Throughout my life and career I've seen so many statements of this nature from my peers: "the status quo will only ever be a 90% solution at best, therefore we should throw all of that away and just do everything on a case-by-case basis in isolation from one another, then we'd all live in an ideal meritocracy (or something)".
See also any discussion about flat management. "Explicit management hierarchies obviously aren't perfect, therefore let's get rid of them altogether and assume that everyone is awesome and will magically self-organize and do the right thing."
I guess the common theme is, someone is greatly overestimating their own rationality, and projects it onto other people, and ends up with an almost religious faith in the self-organizing competencies of unmanaged people, or the natural inclination of unregulated schools or industries to always do the right thing even when it's not in their own self-interest.
IME there are lots of fundamentally messy or imperfect things in life, where any choice you make has negative consequences. There's also a personality type that is repelled by this sort of messiness and will always be drawn towards throwing everything away and starting from scratch, often with a childlike belief that things they don't understand aren't important, and why can't everyone see how obvious and elegant their solution is? IME they're usually wrong about those situations, but I guess I'm still glad people less pessimistic than I am are tilting at those windmills.
The problem isn't hyperrationality per se; it's a lack of intellectual humility and self-awareness. The behavior you're talking about arises when someone looks at a problem, immediately wants to fix it and makes snap judgements to figure out how to do so. Instead, they should dispassionately consider 1) whether the problem is fixable in any meaningful sense, and 2) whether they have the requisite experience, skill and insight to solve it in five minutes of thought.
You could further distill this idea with the observation that a lot of people consider things too systematically, rather than holistically. "First principles" thinking is a powerful drug, which appears to crack open the world and solve every problem. But no problem-solving paradigm can deconstruct away unknown unknowns or inexperience.
Exercising power over other people is a serious thing. It's extraordinarily presumptuous to believe you have designed a rule that will do more good than harm. I'm much more worried about the cognitive bias that says that because someone is currently the decision-maker, they must be competent at it.
Sorry, it seems that you are jumping into conclusions that were never warranted. Chesterton's fence is a principle that I learned a while ago and I am not defending that we need to throw everything away.
What I am saying is that what you regard as a "90% solution at best" is actually not a solution at all. Standardized tests are terrible predictors of performance on actual jobs. If standardized tests (or IQ tests or anything that is supposed to be a proxy for measuring performance) shows time and again that it only manages to measure people that are good to "study for the test", then why do we keep trying to use them to measure performance? Is it because it "the only thing we have"? This seems like saying "I am lost in Rome but all I have is a map of Paris, let me use that anyway."
And no, I am not proposing some other solution to replace it.
I am not asking for any utopia. What I am asking is "Instead of setting ourselves for failure every time coming up with some scheme for testing performance, why can't we just use actual performance on the activity that we want to evaluate candidates"?
It works for people in sports. It works for trade professions. It works for traders. It works for performers, entertainers, even entrepreneurs.
Why can't we have an educational system that does not put people on tracks to mold them and instead just coach them to follow whatever path they seem fit?
Because proper evaluation takes time and a bad choice carries a huge cost. In the absence of any objective measures, an employer gets around this by just hiring the son of the guy they golf with. Connections.
Sports works, but only because sports are extremely popular, with tens of millions of people enthusiastically participating on a regular basis, and accessible to anybody no matter their age and income. This is not true for, say, working at a law firm. You can’t take the best clerk from a pool of a thousand kids who have been clerking since elementary school. It’s either who you know, or who got good grades and test scores.
How objective are, e.g, IQ tests when I can take two different tests in a span of two hours and get from 128 to 146?
How my grades in Computer Engineering can be used to predict my performance as an Engineer in a country where 85% of the graduates would work in non-technical and more profitable fields?
If a company looking for a Quant gets someone with a Master in Math and a good tracking record knocking on their door, would they stop and ask to see if they can get someone with a PhD instead?
You are right that "proper evaluation takes time". What I am saying is that standardized tests should not count as proper evaluation.
Again, the alternative to standardized tests and credentials is hiring based on connections, ethnicity, pedigree, physical appearance, and superficial confidence. These problems are exactly why meritocracy was invented in the first place. The alternative is not everybody getting the jobs they want, it's you being turned away without any consideration at all because the boss already hired their cousin.
> How objective are, e.g, IQ tests when I can take two different tests in a span of two hours and get from 128 to 146?
You can't take two proper WAIS-IV tests in two hours. In fact, unless you have been part of an unusual psychology study, you've probably never taken a real IQ test in your life. There are a lot of gimmicky free things on the internet that call themselves IQ tests but aren't predictive of anything.
You are moving the goal posts. I was talking only about standardized tests, and now you are also including credentials on the mix.
As for the IQ test - it wasn't online and it wasn't a study, as far as I know - but I can totally concede that it was not a "real" IQ test.
Anyway, the story happened over 20 years ago. Part of the driving exam in Brazil included a "neuro-psychological evaluation" and the doctor flagged me for some extra tests.
From what I remember I went to the clinic the next week, did one test, did the "neuro-technical" exam and was told to wait for an IQ test. When I got the form, it was the same as I got before originally (different order of questions) Long story short I asked why I was re-taking the test and I thought I "failed" somehow. They actually said they were not aware of the first one and later on found it.
Now that I think about it, perhaps it was some weird psychology test? :)
