The fields near my house in the UK are pretty special to me, which is strange having grown up in a city. I've walked them numerous times, with a variety of purposes and during COVID, when outdoor interactions were restricted, I would take my daughter for our daily walk past the brambles, the looming oaks and the river to the weir. It is not that long, and when I first started walking it with my daughter, I was keen to see how far we could go and how quickly we could walk it as it was something else to do when stimulation was limited. With my trusty phone and smart watch, I gathered as much data as I could.
Sometimes I did not feel like going. Sometimes my daughter didn’t either. On other days, we could not wait to get out, my body hunched from hours at the laptop teaching classes, and my daughter wanting to escape the confines of the house where she could not run around and make noise as my wife and I were both working. We would, on a usual walk, see many things: Red Kites, Woodpeckers, Ducks, Kingfishers (if we were lucky) and maybe a Lapwing or two (though I had no idea at the time).
On one particular walk, we saw so many butterflies that we (she) decided that we too would be butterflies, gliding around the fields. I can tell you, this would be a sight to behold. We both ran around and flapped our arms, skipping along the path. Sometimes we would chase the butterflies we saw, sometimes we chased each other. Sometimes we just flapped our arms as we walked the distance. When it rained and became a bog, there was no flapping of arms and laughter. There was a lot of moaning, and I would often end up carrying her on my shoulders, trying to make sure I did not lose a wellington boot in the mud and gain a mud bath for us both and the inevitable tears that would follow.
At the end of the walks, I had precise data on the distance travelled, how quickly we were walking, my average heart rate, and the route we took on the map. It is still stored somewhere on my device, but I never look at it. What I do remember most about the route is walking with my daughter, sometimes being a butterfly, throwing water on each other when we got into the river at the weir, and dealing with my own and her grumpiness when it was raining, boggy, and little joy in our hearts as we trampled round. This was the terrain we encountered together, and it brought laughter, tears, frustrations and images of not caring what anyone else thought as they were also on their daily COVID walks around the map route.
I was reminded of maps and journeys as I watched the Diary of a CEO podcast on AI and its effects on society. The discussion included Amjad Masad, CEO and founder of Replit, who made the following claim about education and AI (edited lightly):
"I spent a lot of time in education technology one thing that is as we say on the Internet, a black pill about education, and general education intervention, There's a lot of data that shows that they're very little interventions you can make an education to generate better outcomes. So, you know, there's a lot of experiment around pedagogy, around how to configure the classroom that have resulted in fairly marginal improvements. There's only one intervention, and this has been reproduced many, many times that creates two-sigma, two standard deviation positive outcomes in education, meaning you're better than 99% of everyone else, and that is one-on-one tutoring…so what can create one-on-one tutoring opportunity for every child in the world? AI."
The research Masad was referring to was Benjamin Bloom's 1984 article The 2-Sigma problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. This is the same paper referred to by Sal Khan of Khan Academy in his talk about AI tutors, and it sounds convincing.
Bloom wanted to find a way to improve teaching to the level of one-to-one tutoring which apparently led students to perform better than classrooms (two standard deviations – 2 sigma) that were geared to feedback and mastery of the content taught. I remember being really impressed and have used the graph too, until I read this piece by Paul T. von Hippel which challenged me to go back and read the original research papers more carefully myself.
Joanne Anania's research, which was cited by Bloom, was designed around three conditions. A conventional classroom lesson where topics were taught to all students, a mastery learning frame where the topic was taught to a group and then given tests and corrective information, and a tutoring experience, where tutors worked with students in the lessons using mastery techniques. What was interesting is the tutors that were used.
A skilful tutor continually assesses the effectiveness of cues by observing the responses of the student, readjusting and adding cues when the need is indicated, and gauges the amount and kind of practice required to assure maximal participation by the student. Reinforcement is also based on individual need, and the close working relationships which evolve in successful tutorials allow the tutor to identify and supply the forms of reinforcements most effective for the individual.p.26
Anania's research, depends on the following:
· Tutoring replacing whole-class teaching
· Mastery learning as a component
· considerable training
· assessment specifically designed to test content
· short duration
This appears to be more than just 'tutoring'.
In more real-world conditions, tutoring normally look like this:
· Tutoring outside of the class
· variable training for the tutor
· assessment designed by exam boards
· long duration (years)
Cohen, Kulik and Kulik (1982) in their meta-analysis of studies stated that tutoring had an effect size of 0.6, which was matched in maths tutoring, but not when it came to reading instruction. Another meta-analysis (Nickow, Oreopoulos and Quan, 2020) cited in the von Hippel challenge suggested that tutoring has an effect size of 0.37. This is still impressive, but it is not two-standard deviations, and it may not be uniform across all subjects. There is, however, something that has been elided in the figures. Masad, relying on a particular interpretation of Bloom relying on Anania, presents a map of tutoring effectiveness that is synonymous with an education. It is clear, measurable and reflected in test scores. As a result, AI is a natural facilitator. But Anania's research hints at a territory more complex, where tutors gauge student needs, build relationships and adapt in real-time by their physical presence.
Masad appears to miss the social dimensions and emotional dimensions of an education, focusing solely on measurable outcomes. AI provides an array of possibilities, but in the techno-optimism displayed misses these moments; when a student whispers, 'this is hard', and the right look and words from the teacher acknowledges the difficulty and motivates and supports the student to do more than they thought was possible. An education is more than scores; it is also who the students want to become and how they should relate to other people in the world around them.
The point is that AI is here in education, and it can help us map and offer direction to support student outcomes. It can provide valuable data, help identify knowledge gaps and suggest particular pathways. This is certainly meaningful and helpful.
But it is a poor education that does not have someone else to engage with, to help you adapt to the terrain, as you move through space and time with your body and your feelings. The terrain of education includes the teacher who notices that someone is hungry, that role models how to deal with racist ideas, that feels and amplifies the energy in the room when everything clicks for a student/students and they are elevated to a different level of understanding. It is also shaped by the shared sense of struggle that builds resilience and strengthens the bonds between students in a match, working on a team challenge or grappling with a new concept.
These are the meaningful experiences that exist beyond what can be mapped; in the terrain of education that we navigate, together, sometimes stumbling, sometimes soaring, but fully human, living in a world where whirling your arms around to be a butterfly can be the most sensible and educative thing to do.