Google’s brain team is open sourcing Tensor2Tensor, a new deep learning library designed to help researchers replicate results from recent papers in the field and push the boundaries of what’s possible by trying new combinations of models, datasets and other parameters. The sheer number of variables in AI research combined with the fast pace of new developments makes it difficult for experiments run in two distinct settings to match. This is a pain for researchers and a drag on research progress.
The Tensor2Tensor library makes it easier to maintain best practices while conducting AI research. It comes equipped with key ingredients including hyperparameters, data-sets, model architectures and learning rate decay schemes.
The best part is that any of these components can be swapped in and out in a modular fashion without completely destroying everything. From a training perspective, this means that with Tensor2Tensor you can bring in new models and data sets at any time — a much simpler process than would ordinarily be possible.
Google isn’t alone in its pursuits to help make research more reproducible outside the lab. Facebook recently open sourced ParlAI, its tool to facilitate dialog research that comes prepackaged with commonly used datasets.
Similarly, Google’s Tensor2Tensor comes with models from recent Google research projects like “Attention Is All You Need” and “One Model to Learn Them All.” Everything is available now on Github so you can start training your own deep learning-powered tools.