As you may know (if you follow us on Twitter or check our website), on the 18th October 2019 Cambridge Maths launched an app called CM Define It. The app is a survey tool investigating how the mathematics education community perceives definitions of key terms that already exist. Since the beginning of the Cambridge Mathematics project, the team has read vast amounts of literature in the areas of mathematics, mathematics education and psychology. Such in-depth research has highlighted some of the issues that currently exist in mathematical definitions. One example is the fact that many definitions (internationally, not just in the UK) contain inaccuracies that may promote misunderstandings of what a term really means. Furthermore, many misconceptions about the meanings of mathematics words exist. It takes a long time and a sound understanding to see such misconceptions in the mathematical world around us and to spot them in key terms.1 All of this has encouraged the team to find out what makes for a good definition of a mathematical key term.
We reported on how the app was doing after Christmas in our Apps, apps, apps blog. Since then, we have also been selecting key words in the Framework to see how popular they have been over the years using Google Books Ngram Viewer. You can see the frequency of the use of the word probability in one of our blogs and the frequency of the word graph in a more recent one.
Some of you may be aware that from February 2020 we made a few changes to the CM Define It app. We sent out a short survey to those who signed up to the app but who stopped responding after week 4. The results showed that 40% of the respondents stopped engaging with the app because the more detailed questions about each definition took too long to answer. Additionally, 60% of survey respondents said that they would perhaps be more likely to engage with the app if definition ratings were the only compulsory element and the further questions were optional. Cambridge Maths listened to this feedback and made changes to the app. Since February, the only compulsory part is rating each definition on a five-star rating scale.
In addition, 40% of survey participants suggested that an incentive of some kind would increase their engagement with CM Define It, so we also introduced a reward. Once participants rate the definitions, they are presented with a fun, unique or quirky mathematical key term (have you ever heard of sexy prime?). They then have the option to stop there or to provide more information on why they gave the ratings they did and then to find out the meaning of the weird mathematical term (who wouldn’t want to know what sexy prime means!?). The second part asking for more detailed responses is completely optional. We hope that these changes reflect what our app users had in mind.
Let’s take a look at where we are now…
The state of play
It has been just over 25 weeks since CM Define It was launched. We thought we would look at the main job roles of all app users as a proportion of those who have signed up to the app since its launch.
Figure 1. A waffle diagram displaying proportions of main user roles of CM Define It users between October 2019 and April 2020
This is the same as the proportions reported back in January 2020.
Weekly responses
Figure 2 shows the weekly attrition of responses over 25 weeks, starting with the launch on 18th October 2019 until the 15th April 2020.
As you can see, there was a steep decline between Week 1 and Week 2, with the number of responses dropping by almost half in Week 2 – down to 54%. Then from Week 13, responses remained rather stable with a slight dip around Weeks 21–23, levelling out to 12% in Week 25.
Figure 2. The attrition of responses on CM Define It between the launch and April 2020
Although the graph shows a stable proportion of responses over the last few weeks, we expected a slight increase in responses around Weeks 16–17 when the changes described above were made to the app. This is not demonstrated in the graph, which is quite surprising.
So why did the changes not increase the number of responses around February? Perhaps not all of our existing (and potential) users knew about the changes we were making? Is this a demonstration of the usual trend – greater initial engagement with a new app or product, followed by a steady decline in interest until a relatively stable user base is reached? Or perhaps we did not promote the subtle, yet amazing changes to the app’s interface well enough – shame on us!
What’s next?
Our next task is to see what we have learnt from the app and the responses provided by users. Then we need to figure out how to use this to inform the glossary embedded in the Cambridge Mathematics Framework.
If you have been responding to our words of the week – thank you! If you haven’t – it is not too late to get involved! Let us know what you think about the definitions on offer and find out what weird mathematical word we have in store for you.
References:
1For more information about the app and its development see Glossary App: The development and pilot phase of CM Define It (Majewska, 2019)
Join the conversation: You can tweet us @CambridgeMaths or comment below.