It was hailed as a potentially transformative technique for measuring brain activity in animals: direct imaging of neuronal activity (DIANA), held the promise of mapping neuronal activity so fast that neurons could be tracked as they fired. But nearly two years on from the 2022 Science paper1, no one outside the original research group and their collaborators have been able to reproduce the results.
Now, two teams have published a record of their replication attempts — and failures. The studies, published on 27 March in Science Advances2,3, suggest that the original results were due to experimental error or data cherry-picking, not neuronal activity after all.
But the lead researcher behind the original technique stands by the results. “I’m also very curious as to why other groups fail in reproducing DIANA,” says Jang-Yeon Park, a magnetic resonance imaging (MRI) physicist at Sungkyunkwan University in Suwon, South Korea.
Science said in an e-mail to Nature that, although it’s important to report the negative results, the Science Advances studies “do not allow a definitive conclusion” to be drawn about the original work, “because there were methodological differences between the papers”.
‘Extraordinary claim’
In conventional functional MRI (fMRI), researchers monitor changes in blood flow to different brain regions to estimate activity. But this response lags by at least one second behind the activity of neurons, which send messages in milliseconds.
Park and his co-authors said that DIANA could measure neuronal activity directly, which is an “extraordinary claim”, says Ben Inglis, a physicist at the University of California, Berkeley.
The DIANA technique works by applying minor electric shocks every 200 milliseconds to an anaesthetized animal. Between shocks, an MRI scanner collects data from one tiny piece of the brain every 5 milliseconds. After the next shock, another spot is scanned. The software stitches together data from all the spots, to visualize changes in an entire slice of brain over a 200-millisecond period. The process is similar to filming an action pixel by pixel, where the action would need to be repeated to record every pixel, and those recordings stitched together, to create a full video.
Park and his colleagues claimed that this approach suppressed the slower-paced signal produced by changes in blood flow, which is what conventional fMRI tracks, and could measure the faster-paced signals produced when several neurons change their voltage.
Missing slices
But Park says that, as far as he knows, researchers outside his collaborative spheres have not been able to reproduce the results.
One published attempt2 was led by Seong-Gi Kim, an MRI researcher at the Institute for Basic Science in Suwon, who has previously worked with Park but did not contribute to DIANA. Kim and his colleagues copied the original paper’s protocol, with some enhancements. They found a DIANA-like signal resembling brain activity when they averaged data from 50 brain slices per mouse, but only if they removed data that didn’t fit with the desired response. And the signal vanished when data from more than 1,000 brain slices from six mice were averaged.
In fMRI, averaging more brain slices should strengthen, not weaken, the brain-activity signal, says Kim. Without enough data, he adds, background noise can look like brain activity.
In the original Science paper, the team collected 48–98 brain slices per mouse, but examined only 40 for each animal, Park reports. The researchers say they excluded slices so that they could compare a consistent number across all animals, and removed those with the most background noise. But Park did not mention this until his team shared information with other laboratories hoping to use DIANA. He says that not including that step in the methods was an oversight.
Park adds that if the team non-selectively averaged data from just the first 40 brain slices per mouse, and from all animals for up to about 700 brain slices, the DIANA response was weaker but still statistically significant.
Last August, Science added an editorial expression of concern to the original paper, stating that “the methods described in the paper are inadequate to allow reproduction of the results” and “the results may have been biased by subjective data selection”. The statement says that Science has asked Park to provide more methods and data, and Park says he will submit the additional information by August. He says it takes time to re-analyse the relevant data and prepare detailed methods for reproducing DIANA.
Sequence of events
Valerie Phi Van, a radiologist and bioengineer at the Massachusetts Institute of Technology (MIT) in Cambridge and a co-author of the other Science Advances paper3, initially thought she had recreated the DIANA brain responses in a rat study.
But she also saw those signals when the electrical-stimulation tool was disconnected, and even when dead rats were being scanned.
Looking more closely at the sequence of events, she noticed a 12-microsecond delay between when the electric shock was triggered and when the animal was actually shocked. When Phi Van removed the time gap, the supposed DIANA signal disappeared.
Co-author Alan Jasanoff, a bioengineer and neuroscientist at MIT, says the delay caused “a little fluctuation in the [baseline] MRI signal” that looked like a DIANA response.
Park disagrees that the observed response in the original paper was due to mistimed electrical stimulation, because he says he had previously corrected for a similar aberration in the MRI baseline.
Park has continued to refine the DIANA method and says he has reproduced it in ongoing animal and human studies. He encourages researchers who have had difficulties to contact him, and says he has already shared data with scientists at nearly a dozen institutions.
However, the latest Science Advances papers have cast doubt on the original findings. It’s clear that the signals DIANA detects are “not necessarily related to neural signal”, says Shella Keilholz, an MRI physicist and neuroscientist at Emory University in Atlanta, Georgia. Although, she says, it’s possible that brain activity contributed to the detected signals.
Neuroscientists will continue to explore the cause of the conflicting results. And that could have an upside, says Noam Shemesh, an MRI researcher at the Champalimaud Foundation in Lisbon. The original paper and attempts to replicate or rebut it could lead researchers towards developing and finessing more-direct ways to measure neural activity, he says.