Optimizely helps businesses test out different ways to make their websites or apps more effective — but, as vice president of Product Management Claire Vo put it, how do you go from “five experiments per week to hundreds and thousands of experiments per week”?
Vo said companies face real challenges as they try to increase their experiment “velocity” — the issue isn’t in the “core technology of experimentation,” but rather in how teams prioritize and collaborate on these tests. That’s why Optimizely is launching a new suite of tools called Program Management.
This is basically the Optimizely version of products built by Experiment Engine, a startup where Vo was founder and CEO, and which Optimizely acquired last year. Vo said that since then, a team has been working to rebuild Experiment Engine with deeper integration into Optimizely’s core technology. Program Management is the result.
So the Program Management dashboard allows users to bring up a list of all experiments and experiment ideas — each experiment should include the documentation needed for different team members to understand what’s being tested and why.
It also allows everyone to weigh in on which tests should be prioritized, because they vote on each experiment’s likely impact and required effort (managers, of course, will decide for themselves whether to follow the voting).
Once an experiment actually starts, the Program Management listing will be updated with the results. Optimizely also is building integration with other project management tools, starting with Asana.
And Program Management isn’t just for keeping track of individual tests. It also can help managers see how their experimentation programs are going overall, because it aggregates data about things like how many experiments are being conducted, plus how many of them are “winning” and resulting in real product changes.
Dan Siroker, Optimizely’s co-founder and executive chairman, said that without these tools, companies find themselves relying on Post-It notes and Excel spreadsheets to manage the process, so this should dramatically “accelerate [their] current pace.”
Vo added that companies don’t need to be running hundreds of experiments to use these new tools.
“When you’re starting, you want to have a repeatable process, a framework for managing experimentation,” she said. “All of our customers may not be at a 100- or 1,000-experiment velocity, but they tend to have ambition to be there.”
The BBC is one of the customers that’s already using Program Management, and Optimizely shared this statement from BBC Experimentation Lead Olivier Tatard:
The high-level view that we see through Optimizely’s new Program Management dashboard provides us with invaluable insights into experimentation at the BBC and across our products. Furthermore, Program Management allows us to consolidate what were previously multiple tools into a single, slick environment, drastically improving the efficiency of our end-to-end workflow.