A new study headed by University College London researchers has found that AI does a better job than human experts at predicting the outcome of neuroscience studies. Large language models, a kind of AI trained on huge swaths of text, can predict scientific outcomes more accurately than humans, this study shows. This research underlines the immense potential of using LLMs as powerful tools in furthering scientific discovery.
LLMs can analyze scientific literature to detect patterns and predict results with impressive accuracy. The research team, led by Dr. Ken Luo from UCL, focused on how LLMs could synthesize knowledge to predict future outcomes rather than simply answering questions based on past data. Dr. Luo explained that scientific progress often relies on trial and error, which is time-consuming and resource-intensive. Sometimes, even active and prolific researchers miss these important insights. This research sought to investigate if AI does better and could predict outcomes more effectively.
To this effect, the researchers constructed a tool called BrainBench. It consists of pairs of neuroscience study abstracts, each having a real outcome alongside a similarly plausible but incorrect one. The team tested 15 different general-purpose LLMs. And 171 neuroscience experts to see which could more accurately identify the correct abstract. It was a clear outcome: LLMs outperformed human experts. The results showed that LLMs achieved 81% accuracy, while human experts only reached 63%. Even when the human experts had the highest level of expertise, their accuracy was still lower at 66%. Well-calibrated models might prove to be really valuable collaborators for researchers.
Finally, the team took a pre-existing LLM, Mistral, and fine-tuned it by training in the neuroscience literature. This model, termed BrainGPT, achieved even higher performance at an accuracy of 86%. That would suggest LLMs are even better at predicting study outcomes when specialized in one or another field.
Tools like these will soon be assisting how scientists design experiments,” said Professor Bradley Love, a senior author of the study. The investigation focused on neuroscience, but the methods should apply across all scientific fields. Another important point the researchers made was that much of science follows patterns found in existing research. That, of course, suggests scientists may not be as creative as they think.
It has opened the door to the prospect of a future whereby AI helps researchers make predictions on how experiments may turn out. Dr. Luo said that probably AI would help researchers design experiments more effectively. As it would offer forecasts regarding probable experimental results. And consequently make the process of doing research faster and decisions wiser.
ANI
Also Read: Enparadigm Launches AI-Enhanced Learning Platform
