Wikimedia Brings The Wikipedia Approach To How It Runs Its Servers

Wikipedia and all of its related projects are obviously driven by the work of thousands of volunteers. Interestingly, though, while Wikipedia runs on top of an open-source stack, it’s been traditionally very hard for volunteers to help the organization run its sites. Most of that work is currently done by paid employees. For about a year and a half, though, the Wikimedia Foundation – the organization behind Wikipedia and its sister projects – has been quietly working on Wikimedia Labs, a new project based on OpenStack that will allow volunteers to help the Wikimedia team develop, test and deploy changes to the organization’s back-end infrastructure. Wikimedia Labs launched as a closed beta back in October 2011 and is still in closed beta today.

To be clear, Wikimedia Labs isn’t so much about developing the Wikipedia software, as about running and improving the infrastructure that keeps massive projects like Wikipedia up and running smoothly. Some of the projects the current community of volunteers is working on include the infrastructure that hosts the numerous bots that automatically edit Wikipedia and a collaborative project with OpenStreetMap to improve that organization’s infrastructure and to add OpenStreetMap support to Wikimedia’s projects.

As Ryan Lane, an operations engineer for the Wikimedia Foundation, noted today, in the early days of Wikipedia, volunteers often had root access to the project’s infrastructure. As Wikipedia and its sister sites grew, though, and as downtime became less acceptable, the foundation slowly reduced the degree to which volunteers had access to its infrastructure. According to Lane, Wikimedia hasn’t “had a new volunteer root in years. We haven’t even had a new volunteer with shell access.”

The reasons for this are understandable. While open-source software development tends to scale pretty well, after all, operations doesn’t exactly lend itself to crowdsourcing.

What Wikimedia is doing now then, is setting up an infrastructure that allows volunteers to test and document their contributions. If accepted, that work can then be deployed live to production. This, writes Lane, allows the operations team to regain some of the flexibility it had in the early days of the organization without risking downtime and other issues.