Wikipedia is one of those services about whose reliability I have never thought, simply because it always seems to work. Very impressive especially considering that their tech stack doesn't appear to be the most modern or is (in the case of php) especially known for its robustness.
They do have a pretty complex infra with redundancy, extensive caching, the whole works. But I think they use a relatively boring, reliable stack throughout. It doesn't look like they run a lot of GARTNER MAGIC QUADRANT Cloud Solutions to Supercharge™ their web-scale data blazingly.
On the plus side solid stable software can run for decades without incidents, as long as the hardware is willing. The main negative is that it's not as Supercharged™ as the latest cloud product with a 99.9% SLA, which itself relies on 5 other internal microservices with 99.9% SLA each.
I doubt anyone at HN takes Gartner seriously, as James Plamondon (Microsoft) wrote:
> Analysts sell out - that's their business model. But they are very concerned that they never look like they are selling out, so that makes them very prickly to work with.
The question is, who is gullible enough to still believe Gartner has any value? You'd think by now even the most clueless of MBAs would have caught on.
Thought the same thing. Finding the root cause is easier when you don't have to debug dozens of abstraction layers (KVM, Containerd/Docker, Kubernetes, etc).
> Seriously though, not a fan of php at all but the js tooling is rocket science in comparison.
laughs in left-pad, Webpack, Grunt, ...
JavaScript tooling absolutely sucks - even for a moderate sized project, `npm install` can take many minutes, often enough because some native code has to be compiled. Webpack builds can take even longer.
In contrast, with PHP you run `composer install` and it works, no bundling or whatever required.
The Foundation have invested a lot over the last decade in reliability. Nowadays it is in a great place, but about 10 years ago it was up and down all the time.
Wikipedia is maybe the most important open source project ever (competing only with linux, but that is operating at a more hidden technical level)
This (non)episode of "wikipedia outage" is as good prompt as any to get our collective human chatGPT going :-). Here is my take:
Wikipedia's world-changing impact has been stifled in the era of centralized walled gardens regurgitating memes and ads and isn't really future-proof. Yes you can run you own mediawiki but 1) it not very easy or pleasant at all and 2) knowledge base federation [0] is still a distant dream and 3) its whole stack is nowhere near being integrated with other open source tools that people have at their disposals (heck, even the wiki markup is now a lone exception in the markdown monoculture)
Somehow the next decade of wikipedia should look much more decentralized, resilient and integrated and that could be a sublime next level.
there is vastly more (and ever exploding) knowledge than what the current wikipedia model can integrate (talking always about public knowledge).
Unless there is a breakthrough in how the open source ecosystem integrates, validates, federates public knowledge somebody else will claim that they can better "organize the world's information" (using, e.g "AI") and we will probably not like the new state of the world.
I think there is substantial value in preventing that outcome from happening. NB: I use the term decentralization not just a technical database/server architecture but also as a means of engaging far more resources.
> there is vastly more (and ever exploding) knowledge than what the current wikipedia model can integrate (talking always about public knowledge).
Sure, but there is two vastly different "distributed" architecture at stake here:
- the social one, as in everyone can contribute easily in positive constructive manner, with optimized user path for each user level and interest, and no worry to have about any form of online harassment/bullying or outdoor threats like the-angry-party-staring-at you that might make you vanish for contributing the-wrong-thing-under-the-bad-perspective
- the technical one, which is mainly about "no single point of failure"
Sure there are things that largely overlap, but still two vastly independent set.
> Sure there are things that largely overlap, but still two vastly independent set.
Yes they are by no means the same aspect but the technical (the design part) and "social" domain (how human actors interact using the technology, what incentives they have to contribute, how easy it is to contribute, how to form consensus etc) are never very far apart.
My argument is a general one. If, say, ten years from now wikipedia is to have the same quantum leap impact and positive role it had twenty years ago (when it first got going) it will have to be much more ambitious. But that ambition is unlikely to be served well by a forever centralized service because most knowledge is produced, stored and consumed in distributed ways.
Just for conreteness, think of all those other sources of public knowledge: from openstreetmap, to research paper archives, musea collections and many other public institutions of all kinds (laws and regulations, public statistics etc). Right now everything must come to a centralized store but the size of wikipedia is already flattening [0] and a lot of that data never gets referenced or is a very cumbersome manual job.
In any case, as per link in my other comment some federation is part of the plan (wikibases) and there is the broader "linked data" movement. Its just that at some point those ideas have to get real legs and start walking :-).
That would be one of the many issues to resolve in a federated context. But achieving "consensus" is never trivial and compromises and choices underlie its current model too.
thats a strange assertion when the internet was famously invented to eliminate complex failure modes. Quoting from "DARPA and the Internet Revolution" By Mitch Waldrop [0]
"Finally, Roberts decided to make the network completely decentralized, with no one master computer responsible for sorting the packets and routing them to their destination. Such a Grand Central Station approach would have been much simpler to implement, Roberts knew. But one blown transistor could have taken the whole network down.
So instead, he decreed that the ARPA sites would be linked in a complex pattern rather like the Interstate highway map, and that each site would share the routing responsibilities equally. That is, the computer there would read the digital address on each packet as it came in, accept that packet if the address was local, or else send it off again on the next stage of its journey.
This “packet-switching” approach would mean more complexity and more programming during the set-up phase. But the final system would be far more robust: No one failure could bring it down."
Seems like they have fixed it, but it doesn't hurt to mention Kiwix. Download a full copy of Wikipedia as a .zim file, and you're good to go in even low connectivity scenarios.
The instance of Wikipedia (and other sites) that I'm hosting can be tried out here if you wish: https://kiwix.ounapuu.ee
You can even set up a local kiwix-server as a web server to serve up the ZIM for those who don't have Kiwix installed (or space on their devices, the full English-language Wikipedia with images is about 82GB).
Not sure about the edits, but the title of the requests graph including "Varnish frontend CDN" and the query ("job_method_status:varnish_requests...") imply to me that those are the requests from a frontend cache (Varnish) or series of caches to the backend, rather than all requests.
(I'm not 100% sure you intended to suggest Wikipedia only gets 100k requests/s, I just thought it was an interesting discussion point.)
I'm not sure that the person who submitted this could have considered the fact that the service has now been restored, regardless of how imminent the restoration was.
It's an interesting event (albeit it relatively low impact one) which we would not have known about if it wasn't posted here. Also, check out the wikimedia service status page, pretty cool, huh?
These kinds of posts are interesting to learn more about the architectures and failure modes at play.
I agree that HN should not be a down detector but for a high profile service with infrequent outages like Wikipedia in this instance, a post with discussion is acceptable in my eyes.
Tangential: what is the current funding status of the Wikimedia Foundation related to the donation banners that have started popping up on Wikipedia a day ago or so?
It switched from a charity-like operation, with barebones staffing and incredible benefits for users per dollar spent, to a 'splash the cash and hope' strategy.
I personally don't want my money spent like that, so I don't donate.
