Redux: What is the hardest times table question?

Way back in 2013 we reported on a research project at Caddington Village School on the hardest times table questions where

we logged a total of 60,000 questions across 232 children in years 5-8 using our intuitive smartphone/tablet app. Each child logged in to a centralised server using a fast graphical routine (no tricky passwords!) and played games that were batches of 20 random questions across the 12 times tables. All the answers (and the time they took) were then returned to the server

Sue Teague (the Headteacher at Caddingon Village School) and I followed this up with a published paper in Mathematics Teaching called Why is 48 so hard to remember?.

Well, we thought it was about time we updated some of the data on this area using all of the results we recorded for 433 children across 10 of our schools for the 2015-16 school year (encompassing years 2-8). All results recorded were from playing games of 20 randomised questions giving a total of 1,303,260 answers.

From these data (and with a little data wrangling in R) we have produced the visualisation below (left) which shows the percentage incorrect and can be directly compared to the original 2013 data (right). Some initial things worth pointing out:

• there are alot more questions
• there are alot more children
• it's not a snapshot in time, but represents all games over the whole year. The children will have memorised more answers as time went on
• whilst not the purpose of this post, ongoing research is looking at the impact of using Number.fy to memorise times tables. This would utilise a snapshot approach at the start and end of the year (similar to the startling year trial at Caddington Village School)

OK, so what are the takaways here:

• the number of errors has reduced from 20% to 15%!! And it's worth noting that this dataset is year 2-8 (with very few year 7-8), whereas the origial trial was year 5-8. The improvement is therefore far more significant.
• the hardest 21 questions in 2013 had a range of error rates from 40-62.5%. That extended "tail" of very high error rates now no longer exists with a maximum error rate in 2015 of 36%.
• the order of hardest question has changed, although is very similar. The top two hardest questions remain the same: 6x8 and 8x6.
• perhaps not surprisingly, those questions that are the hardest occupy the "middle" of the chart on the left and are shown by the distinctly red hot spots!!
• and the 12x questions are being answered correctly more often.
• after practise, this data probably represents the best estimate of the times tables children find the hardest and that the 12x is perhaps not as cognitively difficult as adults might think (or maybe it simply represents greater practise?)
• those 6s, 7s, 8s, and 9s still remain dang hard!

2015-16

2013