Meet The Team Helping Bring Pinterest’s Code To The Rest Of The World

Pinterest has built plenty of its technology on open source projects, so of course it wants to open its projects up to the developer community, as well. So the company has built a committee at Pinterest dedicated to making its tools available to the open source community when it thinks they are ready.

Enter Pinterest’s open source committee, a team of engineers that got together two years ago: Pawel Garbacki, Ludo Antonov and Dannie Chu. There wasn’t a formal open source initiative at Pinterest — in fact, engineers at Pinterest didn’t even know they were allowed to open source their technology — which brought Chu, Antonov and Garbacki together.

Over time, the committee developed an ambassador program for teams. Each team has a representative that joins the committee once a month to discuss current open-source projects, those being considered for open sourcing, or what new projects can be open sourced. It’s not seen as a gating process, but more of a governing body that helps make sure open sourced projects have a higher chance of success, Antonov said.

“I don’t think we’ve seen any spectacular conflicts where someone wanted to open source a piece of crap, and the manager said no or something,” Garbacki said. “It’s been working really smoothly, if you have the right people in the room you can trust that it’s going to work.”

Engineers at the company are usually looking for more exposure, and it’s great for developing engineers within the company, Chu said. It gets them excited to work on things outside of day-to-day activities to help them grow as an engineer. And of course, all of this brings an opportunity here to identify and attract engineering talent to companies like Pinterest when they contribute to open source projects.

“We know open source is a great opportunity to build our engineering brand,” Chu said. “Culture-wise there are a lot of benefits to open-sourcing, the way in which you build software to be open-sourceable, opens up the system to be more maintainable, extensible, because you know other people are looking.”

That program extends across the company, Antonov said, and the ambassadors encourage engineers to think carefully about what should be considered for open source. That also encourages engineers to generally improve their code because they know it’ll be very public-facing. That means reducing dependencies, and it also forced the company to think about isolation of various code bases so they work separately, while still being an integral part of what the company does, Antonov said.

Pin peek“A lot of open source projects, because we’ve taken the time to look at whether this is a good open source initiative, has actually allowed these projects to develop in this componentized way in a way you have clear dependencies and it’s a lot easier for other teams to adopt other peoples’ code and maintain that code independently,” Chu said.

Still, there isn’t exactly dedicated team that open sources technology at Pinterest. When someone thinks a  project is a good candidate to open source, the committee provides them with guidance of what good open source projects look like. The three drew from experience and as part of a three to six month project to codify all the process in order to assist project owners that want to open source their technology.

A lot of it came down to bucketing what the process looked like in terms of a coherent framework that project owners can draw from. “They need a crisp path so they know how to get from point A to point B,” Antonov said.

There are other benefits to open sourcing technology. For example, developers may end up discovering use cases for technology that Pinterest developers may not have thought about. Those discoveries can then be fed back into Pinterest’s projects.

One recent example is PinLater, a tool that helps companies update activities between two users asynchronously. When someone hits a pin button, it’ll show up immediately for that user, but it doesn’t have to show up for followers right away. That can be useful for users who have millions of followers, for example, which might cause a strain on Pinterest’s service. Another potential application is sending out push notifications.

“Whenever we build, since we rely on open source, that engagement helps us work better with the community,” Raghavendra Prabhu, Pinterest’s head of infrastructure and the lead behind PinLater, said. “We also get the benefits. The community can build capabilities we can’t imagine.”

There are a few things that define a good open source project, Antonov said: It has to be externally useful; it’s hardened in production (basically, it works and is high quality); and it has to be a work in progress — as in, the company has to continue contributing.

The committee coaches future maintainers to continue contributing and maintaining the code once it enters the general engineering community. In terms of quality, the project has to actually work in production internally, Antonov said. It also has to be easily adoptable and easy to understand in the on-boarding process, he says.

PinLater Code (1)

“We don’t want to ship out an improvement to a data storage system, and know that other companies when they adopt it that it has issues, that makes our brand look bad,” Chu said. “In terms of code style and quality, people don’t worry too much about that as long as it works.”

Managers have to get through a couple of steps before the project actually gets open sourced. The project owners first fill out a “pre-flight checklist” that make them think about whether a project is a good candidate for open source, and what requirements it should meet to qualify for it. That also has a section for legal approval, where the committee works with a legal team to approve it and make sure there are no dependencies on incompatible licenses.

Once managers clear the requirements, if the project’s source code is served with other projects that aren’t open sourcing, the code goes into final checks to ensure there are no internal passwords or the like. At that point, a member of the committee gets involved to make sure it meets quality and legal requirements — and then they flip the switch and make the repository public.

There’s always a chance that an open source project can be forked into a new one, and Chu said the company looks at it as a red flag that it isn’t doing a good job of contributing to the community or it isn’t evolving as it should be. Sometimes there are legitimate cases of when the company hasn’t thought of an alternative direction of the project, Antonov said, and if that’s the case the team isn’t opposed to it.

Every project is still a work in process. The company nezxeds a bigger body of maintainers, Garbacki said. Like other open source projects, it also needs dedicated members from the community that will help maintain and make decisions as to where engineering projects go.

“The internal pressure is there,” Chu said. “If you know your code is gonna be maintained and potentially extended by someone else, it makes your ears perk up a bit and makes you want to pay a little more attention to how other people will be perceiving your code and will be able to add to it.”