After Aaron, Reputation Metrics Startups Aim To Disrupt The Scientific Journal Industry

Next Story

In Europe Etsy Sees Opportunity, Says Caroline Drucker [TCTV]

Editor’s note: Richard Price is founder and CEO of Academia.edu, a platform for academics to share research papers. You can follow Richard on Twitter @richardprice100.

Aaron Swartz was determined to free up access to academic articles. He perceived an injustice in which scientific research lies behind expensive paywalls despite being funded by the taxpayer. The taxpayer ends up paying twice for the same research: once to fund it and a second time to read it.

The heart of the problem lies in the reputation system, which encourages scientists to put their work behind paywalls. The way out of this mess is to build new reputation metrics. The changes to reputation metrics in science that are underway are reflective of how reputation is measured online: Twitter has followers and retweets; GitHub has followers and forks; StackOverflow has reputation; Facebook has likes and comments; YouTube has view counts. An ecosystem of startups is working on building these new reputation metrics in science, including my startup Academia.edu, as well as Mendeley and ResearchGate (other important players in the space are PLoS and Google Scholar). All three platforms have passed 2 million users and are growing fast. In three to four years, all the world’s scientists will be on one or all of these platforms.

The Tragedy Of The Commons In Science

Scientists need to build their reputations, and the primary reputation metric in science is being published in prestigious journals, such as Nature, Science, and The Lancet. When scientists apply for a grant or a job, they know that there are 200 other people applying for the same grant, and that the grant committee scans resumes looking for such journal titles.

Journal publishers use their ownership of the reputation system to their advantage. When a scientist is looking to be published, they require a scientist to transfer the copyright of their paper. In this transaction, the scientists who wrote the paper are not paid and receive no royalties from the revenues from the paywalls. The peer reviewers who review the paper for the journal are not paid, nor are the taxpayers who have provided between $20K and $160K for the funding of the research behind the paper.

Because of its ownership of the reputation system in science, the journal industry is able to acquire the copyright to the world’s peer-reviewed scientific output for free. It then charges the public who funded the research — and the scientific community who authored and peer-reviewed it — $8 billion a year to access it. Effectively, the scientific community provides the product to the journal industry (the papers and the peer reviews), and then has to pay, along with the public, to get it back.

The tragedy of the commons is that individually rational decisions, namely scientists handing over the copyright of their papers to collect reputation metrics, lead to an outcome that is bad for the public at large: Because of paywalls, the majority of the world ends up being unable to access the scientific literature that it has funded.

New Reputation Metrics

To break out of the tragedy of the commons, new reputation metrics, developed by a number of startups, have been developed that incentivize scientists to share their research openly, rather than incentivizing them to put their research behind a paywall. Scientists are adopting them to better stand out from the crowd when applying for jobs. Examples of these new reputation metrics include inbound citation counts, readership metrics and follower counts.

Inbound citation metrics. A few years ago, Google Scholar started displaying inbound citation counts for papers – counts of how often a given paper was cited by other papers. Scientists have started to see these inbound citation counts as a way to demonstrate the impact of their work, and are increasingly including them in their job and grant applications. In some fields, such as physics, scientists are more proud of their inbound citation counts than they are of the journal titles on their resume.

Readership metrics. Academia.edu, Mendeley and ResearchGate are helping scientists to understand readership metrics around their research. These sites tell academics how many people are reading their work, as well as some demographic data about those readers. Increasingly these readership metrics are helping to influence hiring decisions by tenure committees.

Follower counts. Scientists are increasingly wanting direct, unmediated relationships with their audiences. Twitter, Facebook and other sites have put content creators directly in touch with their audiences. Scientists are saying ‘I want that direct relationship with my audience too!’ The personal brands of scientists are starting to eclipse those of journals, and follower counts help a scientist understand the growth of their personal brand.

In the pre-web era, scientists used to print out papers and read them in their labs in non-trackable ways. Increasingly scientists are reading and sharing papers online. The reputation metrics described above are derived from this online activity; two others that will emerge include:

  1. Commenting metrics: As scientists increasingly comment on papers online, metrics will emerge to reflect the most discussed papers.
  2. Recommendation metrics: As scientists increasingly share paper recommendations online, metrics will emerge to reflect the most shared/recommended papers.

To distinguish between mere popularity and genuine impact, these metrics will take into account the reputation of the scientists doing the commenting/recommending. The metrics will be recursive in the way that Google’s PageRank algorithm looks at the quality of the linking site and not just the quantity of them.

Scientific Journals Will Disappear

As I mentioned, the journal title has historically accounted for close to 100 percent of a scientist’s public reputation. That figure is probably now at 90 percent, with 10 percent for the new reputation metrics mentioned above. As new reputation metrics emerge, the journal title will decline in relative significance. Soon we will get to a point where the journal title contributes less than 10 percent of a scientist’s reputation, and the bulk of the scientist’s reputation metrics are coming from other sources.

At this point scientists will see that the costs of publishing a paper via a journal outweigh the benefits, and they will stop publishing papers in journals. The costs of publishing a paper via a journal are significant, both in impact and money. Journals take a long time to publish research. There is an average time lag of 12 months between submitting a paper to a journal, and the journal publishing it. This is 12 months of lost impact for the scientist.

Journals mostly put papers behind paywalls, which further limits the audience and impact of the paper. Some journals now make the paper accessible to readers for free, but the author typically has to pay $1,000-$3,000 to remove the paywall around their research.

Increasingly it will be seen as perverse to submit a paper to a journal and wait 12 months for comments from two scientists, instead of sharing it on a platform like Academia.edu and getting comments from hundreds of scientists in two weeks.

The first journals to disappear will be the ones whose titles offer the least reputation boost – the second- and third-tier journals. Shortly afterwards, Nature, Science and the top-tier journals will disappear. Scientists will be sharing their work on multiple platforms, and their reputations will be based on a constellation of metrics. And as journals lose their significance, the dream of open access will be realized: a villager in India will have the same access to the world’s scientific literature as a professor at Harvard.

How Reputation Metrics Will Change Science

In addition to incentivizing scientists to share their work openly, new reputation metrics will also play a role in changing science in a number of ways:

Better peer review. Right now the peer-review system takes 12 months to complete, and surfaces the opinions of only two scientists – scientists who may be biased, uninformed about the subject matter, or just in a bad mood when writing the review. Reputation metrics will bring about a system where opinions are surfaced from the entire scientific community, and in real time. A mathematician who sees an incorrect theorem in a paper they are reading will be racing to get their refutation out by 6 p.m. in order to collect the glory and the reputation metrics that will follow from that insight.

Instant distribution. Reputation metrics will incentivize scientists to share their work instantly, rather than let their work be held back in 12-month publication time lags.

Data sets and other content formats. Historically, papers are shared because the journal title has been the only reputation metric, and journals only publish papers. Journals don’t publish data sets, code, videos, and other aspects of a scientist’s output. Seventy-five percent of the world’s scientific data isn’t shared because the incentives aren’t there for scientists to share it. New reputation metrics will provide those incentives.

Business Models

Platforms like Facebook, Twitter, YouTube, and others don’t charge users to share or consume content. The costs of the platforms are low enough for them to be able to monetize via ancillary services such as advertising.

We are moving towards a science where scientists and the general public will not be paying to share and consume research. The business models that will emerge in science will be as diverse as the ones on the web at large. There will be advertising businesses; freemium models; and enterprise sales models.

$1 trillion a year is spent on R&D, and as scientific activity moves online and becomes trackable, it is going to be possible to build tools that help that R&D capital be better spent more efficiently.

Every innovation in medicine and technology in the world has its roots in a science paper, and speeding up science will change the rate of innovation. The startups looking to help facilitate this, such as those mentioned above and Science Exchange, Figshare, Microryza, Quartzy, Altmetric and ImpactStory, are engineering-driven and need engineers and designers to aid in the effort. If you are interested in joining, there is a list of startups looking to accelerate science here.

[Richard recently appeared on “In The Studio” with TechCrunch’s Semil Shah. Watch him discuss Academia.edu and his plan to help scientists break out of the tragedy of the commons.]