How AI is changing scientists’ understanding of language learning


New AI models may show language learning is possible without a built-in grammar model, contrary to what many linguists have believed

Unlike the carefully scripted dialogue found in most books and movies, the language of everyday interactions tends to be messy and incomplete, full of false starts, interruptions, and people talking to each other. From informal conversations between friends, to quarrels between siblings, to formal discussions in a boardroom, authentic conversation is chaotic. It seems miraculous that anyone can learn a language given the random nature of linguistic experience.

For this reason, many language scientists – including Noam Chomsky, a founder of modern linguistics – believe that language learners need a kind of glue to master the unruly nature of everyday language. And that cement is grammar: a system of rules for generating grammatical sentences.

Children must have a grammar model hardwired into their brain to help them overcome the limits of their language experience – or so we think.

This template, for example, may contain a “super-rule” that dictates how new elements are added to existing sentences. Children then only have to learn whether their first language is one, like English, where the verb precedes the object (as in “I eat sushi”), or a language like Japanese, where the verb follows the object (in Japanese, the same sentence is structured as “I eat sushi”).

But new insights into language learning are coming from an unlikely source: artificial intelligence. A new breed of great AI language models can write newspaper articles, poetry and computer code and answer questions honestly after being exposed to large amounts of language input. And even more amazing, they all do it without the help of grammar.

Grammatical language without grammar

Even though their the choice of words is sometimes strange, absurd or contains racist, sexist and other harmful prejudices, one thing is very clear: the overwhelming majority of the output from these AI language models is grammatically correct. And yet, there are no hard-wired grammar patterns or rules – they rely solely on linguistic experience, however messy.

GPT-3, presumably the the best known of these modelsis a gigantic deep learning neural network with 175 billion parameters. It was trained to predict the next word in a sentence given what happened before on hundreds of billions of words from the internet, books and Wikipedia. When it made a bad prediction, its parameters were adjusted using a machine learning algorithm.

Remarkably, GPT-3 can generate believable text reacting to prompts such as “A summary of the latest ‘Fast and Furious’ movie is…” or “Write a poem in the style of Emily Dickinson”. In addition, GPT-3 can respond to SAT-level analogies, reading comprehension questions, and even solving simple arithmetic problems — all while learning to predict the next word.

Comparison of AI models and human brains

The similarity to human language does not end there, however. Research published in Nature Neuroscience has demonstrated that these artificial deep learning networks appear to use the same calculation principles as the human brain. The research group, led by neuroscientist Uri Hassonfirst compared how GPT-2 — a “little brother” to GPT-3 — and humans could predict the next word in a story from the “This American Life” podcast: People and AI predicted the exact same word almost 50% of the time.

The researchers recorded the brain activity of the volunteers while listening to the story. The best explanation for the activation patterns they observed was that people’s brains – like GPT-2 – were not just using the previous word or two to make predictions, but were relying on the accumulated context so far. to 100 previous words. Overall, the authors conclude: “Our finding of spontaneous predictive neural signals when participants listen to natural speech suggests that active prediction may underpin lifelong language learning in humans.”

A possible concern is that these new AI language models receive a lot of input: GPT-3 was trained on linguistic experience equivalent to 20,000 human years. But a preliminary study which has yet to be peer-reviewed found that GPT-2 can still model human next-word predictions and brain activations, even when trained on just 100 million words. This is well below the amount of language input an average child could hearing during the first 10 years of life.

We are not suggesting that GPT-3 or GPT-2 learn language exactly like children. In effect, these AI models don’t seem to understand muchif necessary, of what they say, while comprehension is fundamental to the use of human language. Yet what these models prove is that a learner – albeit a silicon – can learn language well enough from mere exposure to produce perfectly good grammatical sentences and do so in a way that resembles the processing of the human brain.

Rethinking language learning

For years, many linguists believed that language learning is impossible without a built-in grammar model. New AI models prove otherwise. They demonstrate that the ability to produce grammatical language can be learned from linguistic experience alone. Likewise, we suggest that children do not need innate grammar to learn the language.

“Children should be seen, not heard” goes the old adage, but the latest AI language models suggest nothing could be further from the truth. Instead, children should be engaged in back and forth conversation as much as possible to help them develop their language skills. Linguistic experience – not grammar – is essential to becoming a proficient language user. –

This article originally appeared on The conversation.

Morten H. ChristiansenProfessor of Psychology, Cornell University

Pablo Contreras KallensPh.D. Student in Psychology, Cornell University


Comments are closed.