I am not certain on any of it, but I imagine it was a combination of the student input (we all got to rank them on various things and any complaints were noted) and also what the well-respected teachers who had been there a long time thought of them.
Almost all of the admin were teachers who retired from teaching, so I imagine they took input from the fellow teachers (who were still teaching) very seriously.
Additionally, it was seen as pretty important that the teachers engage the students in non-classroom ways. For instance a new teacher started a formal chess club at the high school and got us to play against other schools, and I would imagine that things like that are considered a huge plus.
It probably also helped that the school was small (880 students) so it wasn't hard to notice quality and for the admin to know every teacher on a personal level.
From what I'm aware of a lost of private schools work in a similar way with respect to extra curricular activities, to the point where they expect you to take on extra duties like coaching sporting teams and so on.
My problem with this is that people feel they have to do it to have a job, and it then takes time away from their actual job which is teaching students. I have no problem with the performance review aspect of it though, as long as it used correctly (administrators are people too, and just as capable of doing a bad job, pushing a personal agenda, etc).