Posting about a research paper on social media platform X (formerly known as Twitter) doesn’t translate into a bump in citations, according to a study that looked at 550 papers.
The finding comes as scientists are moving away from the platform in the wake of changes after its 2022 purchase by entrepreneur Elon Musk.
An international group of 11 researchers, who by the end of the experiment between them had nearly 230,000 followers on X, examined whether there was evidence that posting about a paper would increase its citation rate.
“There certainly is a correlation, and that’s been found in a lot of papers. But very few people have ever looked to see whether there’s any experimental causation,” says Trevor Branch, a marine ecologist at the University of Washington in Seattle and lead author on the paper, published in PLoS ONE last week1.
Every month for ten months, each researcher was allocated a randomly selected primary research article or review from a journal of their choice to post about on their personal account. Four randomly chosen articles from the same edition of the journal served as controls, which the researchers did not post about. They conducted the experiment in the period before Elon Musk took ownership of what was then known as Twitter and complaints of its declining quality increased.
‘Nail in the coffin’
Three years after the initial posts, the team compared the citation rates for the 110 posted articles with those of the 440 control articles, and found no significant difference. The researchers did acknowledge that their followers may not have have been numerous enough to detect a statistically significant effect on citations.
The rate of daily downloads for the posted papers was nearly fourfold higher on the day that they were shared, compared with controls. Shared papers also had significantly higher accumulated Altmetric scores both 30 days and three years after the initial post. Calculated by London-based technology company Digital Science, an Altmetric score, says Branch, is a measure of how many people have looked at a paper and are talking about it, but it’s not a reliable indicator of a paper’s scientific worth. “It’s thoroughly biased by how many people with large followings tweet about it,” he says.
The findings echo those of information scientist Stefanie Haustein at the University of Ottawa, whose 2013 study2 found a low correlation between posts and citations.
Haustein says the problem with using posts as a metric is that, even a decade ago, there was a lot of noise in the signal.
“We actually showed that a lot of the counts on Twitter you would get were bots, it wasn’t even humans,” says Haustein, who wasn’t involved in the new study.
She says the more recent departure of scientists from the platform has been the final nail in the coffin of the idea that posting could increase citations.