To advance next-generation energy storage technologies researchers must identify optimal catalyst materials for specific reactions. Using machine learning, researchers have improved a key step in catalyst discovery.
Building on previous research that demonstrated the prediction accuracy of adsorption energy using a transformer-based language model trained solely on human-readable text, researchers at Carnegie Mellon developed a methodology to enhance the model through multimodal learning.
"When it comes to chemical data, by integrating different modalities, we can construct a comprehensive view. Inspired by this, we used multimodal learning to improve the performance of the predictive language model," said Janghoon Ock, Ph.D. candidate in Carnegie Mellon University's Department of Mechanical Engineering.
By facilitating the connections between different models and enhancing their ability to complete tasks without task-specific labels, the method developed by the researchers reduces the mean absolute error of adsorption prediction by 7.4–9.8%. Additionally, the method can bypass the previously required structure information by incorporating a generative language model into the framework, allowing for initial energy estimate predictions without atomic coordinates.
"My ultimate goal is to build accessible and interactive methodologies that non-computational scientists can use," said Ock. "LLM can be a key to achieve that accessibility and interactiveness. While it's not the case right now, we are moving in the right direction."
"Being able to produce energy estimates with just chemical symbols and surface orientations is a leap forward for accessible ML models," added Barati Farimani, associate professor of mechanical engineering.
The team believes they can create a more comprehensive language-based catalyst design platform in the future by adding additional functionality and equipping the platform with agent-like framework planning capabilities.