Low ratings for language in textbooks in Germany

The education researchers and linguists digitized roughly 3,000 texts from 35 textbooks approved by the state of Baden-Württemberg. They used methods from computational linguistics to define a set of linguistic features on the basis of which the texts can be compared in terms of their linguistic complexity. For instance, the texts were analyzed with regard to how many different words are being used, the average number of words per sentence, grammatical structures, how often the genitive -- an indicator for high education in German -- is used, or the number of connectors, meaning words that signal semantic relations such as "however" or "instead."

Subsequently, the researchers compared texts from textbooks for different grade levels and school tracks. Their assumption was that texts would be more complex for higher grade levels and higher levels of proficiency. Also, they were interested in whether there were differences between individual publishers. Since many books comprised two grade levels, the researchers compared textbooks for levels 5 and 6 with books for levels 7 and 8 as well as books for levels 9 and 10. On the basis of their computational linguistic analyses, the researchers were able to find out whether the texts intended for higher grade levels are indeed more complex than those for lower grade levels, whether textbooks for lower and higher school tracks differ from each other, and whether there are differences between publishers.

Their results: In the textbooks that were analyzed, linguistic complexity increased systematically only to a limited extent and there were marked differences between publishers with regard to this increase. This leads the researchers to the assumption that many texts are not entirely suited to the development of students. "Our study does not yield results on whether the texts are actually too difficult or too easy, since there are no official standards for the competencies of students in the individual grade levels and school tracks," explains Karin Berendes. "A better fit would however be desirable." For if texts are too difficult, learning becomes impaired: Working memory is overloaded and students have difficulty to form meaningful connections between text elements and the information the text wants to convey, resulting in problems to comprehend the text. However, texts should also not be too easy because the optimal learning effect will only be triggered once the new learning content is slightly above the current level of proficiency of learners.

Detmar Meurers, professor for computational linguistics at the University of Tübingen, summarizes: "Authors of textbooks should consider much more systematically what makes texts suitable for students of different age groups and performance levels in order to advance students in terms of language and content. There are too few experts in publishing houses who are knowledgeable in a broad, linguistically sound analysis of texts. Since not every textbook author can also be an expert in linguistics, computational linguistic tools as we used them in our study can in principle also support authors in the selection and composition of texts." Accordingly Ulrich Trautwein, director of the Hector Research Institute of Education Sciences and Psychology, calls for a better integration of research and practice: "Our study is further evidence of how school practice might benefit from the insights of education sciences."