In an analysis published December 5 in the journal Nature Machine Intelligence, the CSHL researchers introduce their "unified evaluation framework" and use it to compare various binary and quantitative models trained to predict chromatin accessibility data.

"We highlight various modelling choices that affect generalization performance, including a downstream application of predicting variant effects. In addition, we introduce a robustness metric that can be used to enhance model selection and improve variant effect predictions. Our empirical study largely supports that quantitative modelling of epigenomic profiles leads to better generalizability and interpretability," the authors wrote.

AI programs have great potential for research innovation. For example, they can find patterns in the human genome -- which has three billion letters of code -- related to disease much faster than human beings can, and they are able spot things that humans miss.

The technology also identifies unique features of the genome that can lead to individualized medicine and treatments. However, the recent surge in AI's popularity has led to a bottleneck in innovation, according to CSHL.

AI researchers are constantly building new algorithms from various sources, but it's difficult to determine whether their creations will be valuable or not to their research. A team from the lab of Peter Koo, PhD, assistant professor at CHSL, set out to address this issue with their invention -- GenOmic Profile-model compreHensive EvaluatoR (GOPHER).

As a new method, GOPHER is meant to help identify the most efficient AI programs to analyze the genome. Using this method, the CHSL team found the parts of AI algorithms that drive reliability, performance, and accuracy. They also discovered that an unpopular type of AI algorithm, known as a quantitative model, outperformed the much more popular binary type by "lightyears."

Going forward, Koo's team hopes that GOPHER will help define the key building blocks for constructing the most efficient AI algorithms and optimize them to ensure they're learning the right things for the right reasons.