
“Injuring” large language models has allowed researchers to gain insight into how these models use language and could help stroke victims with communication challenges in the future.
“Our motivation for doing this research was simple. Language models are trained on human language, and aphasia is an impairment of human language,” said corresponding author Julius Fridriksson of the University of South Carolina. “So, the question was whether we could perturb layers in the language model, give it the same clinical tests we give our patients, and see whether the output looked like aphasia.”
To do this, they established an approach they call lesion-based interpretability, in which they intentionally introduced errors, or lesions, into the LLMs. The researchers found that these "injuries" led to errors in language processing that were similar to their aphasia patients.
“Until now, comparisons between human and artificial intelligence language have been almost entirely correlational,” Fridriksson adds. “This approach for the first time establishes cause and effect.” But he notes that it’s important to recognize that what the paper is suggesting is correspondence between patterns of failure, not correspondence between actual structures.
The researchers say these models can help to interpret what’s happening inside the black box of LLMs by identifying parts of the models that are redundant and unnecessary. This could lead to future LLMs that are smaller and more energy efficient. The models could also be used to create “digital twins” of patients with aphasia to simulate clinical trials and speed up the translation of new discoveries.
The team says the framework also has applications for studying neurological conditions beyond aphasia.
Data from Cell Press