Closing the literacy gap with AI and eye tracking

A success story by Lexplore and Tobii

I find it astonishing that even some of the richest countries in the world haven’t been able to solve the problem of kids leaving primary school with low reading abilities. Despite the knowledge, materials, and resources we have access to, some kids are still falling through the net. Children who will continue to face learning difficulties throughout their lives, resulting in low self-esteem, and lower than average wellbeing. Yet with proper intervention at the right moment, the problem can be solved.

One of the root causes is the lack of a systematic approach to reading development and common objectives — problems that can’t be solved by throwing more resources into the mix but where digital transformation is ideal. That’s what Lexplore, one of our partners, set out to do: to use AI and eye tracking to carry out the grunt work of screening and analysis, to systematize reading assessment, and create common objectives so that teachers can focus on remediation and follow-up.

Their story has many sides. First off, it reveals how AI technologies can transform manual assessment in education — as well as other domains. It’s a textbook tech-for-good story — a classic tale of how startups leverage research to solve a problem, how they reach commercialization through prototyping and a lot of patience, and in doing so, how they generate insight. And as if that wasn’t enough, it highlights how AI and cloud computing leverage insights to generate deeper insights — shifting data collection out of the research lab into live capture.

While Tobii’s technology lies at the root of the story, we cannot take credit for what Lexplore has achieved, yet without eye tracking, their solution couldn’t exist. And even though I’ve worked at Tobii for well over a decade, I still get blown away by the things our partners do with what is fundamentally a measuring device — albeit a special one.

So, that’s where I am going to start — with the measurement.

Measuring stuff

You are probably familiar with your heart rate, your max pulse, how that declines with age, and a rough idea of what your values should be for your age bracket. Heart rate is a simple but highly indicative measurement of what’s going on in your body. If an athlete cannot perform at max pulse, they might be fighting an infection, or under stress. Other useful measurements include body temperature, weight, blood pressure, BMI, and so on. In isolation, these metrics provide us with a momentary insight into our wellbeing. We know this because a lot of people have spent many hours doing the research — measuring people, observing results, and delivering the evidence.

When measured consistently over time, data points show a trend — which often tells a more interesting story. Spikes and troughs might reveal the impact of our actions, what we eat, how much we exercise, and our stress levels. Hardly newsworthy, I know. But there is one significant detail that’s easy to miss, skim over, or even ignore because it seems so commonplace. And that’s the concept of consistency — consistent measurement over time.

At the extreme, consistent measurement is not feasible because — if nothing else — time changes, and so any two measurements are at least differentiated by that factor. But there are plenty of other influences that can impact a measurement: the measuring process, the person carrying out the measurement, the tools they use, how distracted a person is, their stress level, the time of day, ambient lighting, room or air temperature — I think you get my point. For analysis to deliver accurate results, data needs to be collected scientifically.

The best way to generate uniform data points is with a machine. Machines — even if they are programmed with bias — will always measure the same way, they don’t have mood swings and if you design them to withstand the fluctuations of the operating environment, they will deliver consistent sets of data.

If you can easily reproduce that machine, you’ve created a universal measuring device that can generate objective data that transcends borders, class, and human bias.

And that’s what we have done for Lexplore. We have provided them with a universal measuring device — a machine that they can ship anywhere in the world, one that just plugs in, and plays.

How do you know what to measure?

But how do you know what to measure? How did we figure out that heart rate was a vital sign? The simple answer is research. Obviously, you need to have an idea, be curious, and be prepared to put the work in. Many of our partners have roots in scientific research, specifically studies related to eye movements, and Lexplore is no exception.

The story begins deep in the most extensive research made on the reading and writing capabilities of children. Carried out in Sweden in the eighties, the study aimed to better understand dyslexia and followed some 200 children from about the age of ten into early adulthood. The result was a set of statistical models that could accurately identify which students would end up with a low reading ability, based on the eye movement patterns they generated while reading.

In a more recent study (2007), scientists Gustav Öqvist Seimyr and Mattias Nilsson Benfatto built a prototype assessment system using the statistical models developed during the initial research. The two scientists understood the potential of their work to close the literacy gap — and that’s when Lexplore was born.

From prototype to application

Once you’ve figured out what you want to measure, and maybe you’ve built a couple of prototypes, the next step is commercialization — building an application that will run on any machine, with as little extra hardware as possible, in a way that meets security standards and modern design principles. As Lexplore began to explore commercialization, Tobii’s technology wasn’t as plug-and-play as it is today, but its level of accuracy was ideal for Lexplore to take its prototypes to the next level.

I think it’s worth mentioning that when they started building their solution, Lexplore made a couple of fundamental observations that were key to their overall success. The first one was ease-of-use. Like so many other sectors, time is a precious commodity in education. Time spent on administrative tasks is time not spent teaching kids. So, whatever solution they were going to come up with, Lexplore knew that they had to develop something with a low entry barrier, but also something that would provide greater value than the existing tried-and-tested methods.

The second choice they made relates to their system architecture — opting for a split construction, with some processing and storage on a local PC, but most of the heavy lifting and analysis performed in a secure cloud, so that the solution can run on any PC.

These two decisions led to Lexplore investing a significant amount in UX design and process optimization, using color coding and a unique grading system to provide teachers with an immediate and readily understandable analysis of a student’s reading ability.

Understanding the data

I’m not a teacher, but I can say that the results Lexplore delivers are so easy to grasp, you don’t need to be, nor do you need to be a rocket scientist — a basic understanding of statistics will help though.

This diagram shows how Lexplore translates microscopic eye movements into a reading pattern of fixations (bubbles) and saccades (lines). The size of the fixation indicates how long a student lingers on a word. The larger the bubble, the longer it takes for the pupil to understand what’s in front of them. The saccades show how a person skips back and forward through a piece of text. With just a glance, you can see if a student is struggling.

Based on silent reading of a text and out-loud reading of another, the solution measures reading speed and assigns a unique Lexplore score based on the pupil’s grade.  

Students are color-coded according to the stanine their score represents. Instantly providing teachers with a visual clue of the child’s ability. With a score of 390, this student falls into stanine 5, indicating an average reading ability for their age.

Comparisons can be made. You can, for example, see how a class performs over time, and how they compare with the national average.

Data to disrupt

It’s all about the data. Somebody said it at work the other day “insights breed insights” and while that may sound a bit vague, I think what they meant is that once you start digging into data, patterns begin to emerge. But you need to generate the data first, and you need to do that over time.

And that’s just what Lexplore has done. Through systematization and regular screenings, they generate a history for each student. Once anonymized, that data can contribute to aggregated views. In the same way that we have developed healthy heart rates for different age groups, Lexplore has generated average scores for the reading ability of children in different age groups, they can compare schools within the same region, and create national averages — delivering the systematic approach and common objectives lacking in traditional approaches.

What this means is that by screening children we are also generating statistics for how reading ability is evolving. Lexplore has, for example, used live data to show the actual drop in reading abilities due to the coronavirus pandemic.

This kind of live insight provides schools, local councils, and governments with the data they need to add resources where they are needed, and not where they are assumed to be needed.

Uncovering unknown unknowns

What the Lexplore solution provides is a rapid way to screen and analyze the reading ability of schoolchildren. Using traditional methods, it might take teachers months to screen and analyze a class of kids. With Lexplore it takes just a couple of hours. But what’s truly magical with their solution is the unknowns it reveals.

Special-ed teachers are good at identifying children with low reading ability (red zone in the above screenshots) as well as those who excel (dark aqua). The difficultly lies with the in-betweeners (orange) especially when it comes to kids who have developed coping skills that traditional screening techniques cannot detect. The feedback from schools is encouraging because the teachers who are using the solution say that it creates a culture of reading that previously didn't exist.

Because Lexplore chose eye tracking as their fundamental measurement tool, and because people cannot fully control how their eyes move, assessments based on eye movements can’t be cheated — that’s a whole bunch of kids no longer falling through the screening net.

To wrap up, I’d like to extend my gratitude to the researchers, and to everyone at Lexplore who has made this solution real. My only wish is for every school, everywhere to have access to solutions like this, so that no kid falls through the net. There’s a long way to go, and many languages to add before we get there, but at least we know it’s possible.

If you want to dig deeper into the Lexplore solution, take a look at our Lexplore case study. You can find out more about this amazing company at Lexplore.com.