Pupil Insights: skip the guesswork and measure what matters

Dr Beatrice Hayes covers the value of using pre-validated scientific measures to assess pupils’ thoughts, feelings & behaviours.

When exploring the needs of pupils, it is typical to rely on anecdotal evidence and surface-level data (eg attendance figures). Although useful for an overview, such approaches fail to address what school leaders truly wish to know:

• What do pupils think and feel?

• What are the underlying mechanisms shaping their behaviours?

• What is at the heart of what we see in school?

  
  

Utilising the right methods to explore these questions is crucial. Embedded within psychological science, pre-validated measures provide a far more reliable method of exploring the needs of pupils in depth.

Our new service ​‘Pupil Insights’ embodies an in-depth data-driven approach for supporting schools. Using only pre-validated measures, Pupil Insights combines the reliability of science with the reality of schools to explore questions that school leaders really care about.

Better questions, better clarity: What is a pre-validated measure?

Pre-validated measures are carefully constructed, statistically evaluated tools rooted in psychological theory. Pre-validated measures ensure that we are measuring what we think we are measuring.

Let’s say you were interested in pupils’ sense of belonging at school. You could simply ask pupils, ​“Do you feel you belong at school?”.

How likely is it that you will receive reliable responses to this question?

• Pupils will likely say ​”yes” in an attempt to please their teachers – known as the good-participant role.

• Some pupils may say ​“no” in an attempt to challenge their teachers – known as the bad-participant role (or the ​“screw you” effect!).

Either way, pupils are likely to guess the true aim of the question and alter their response accordingly; the cues that they use to make this guess are known as ​‘demand characteristics’.

So, if using this question, are you measuring pupils’ sense of belonging or are you actually measuring their sense of obedience to the teacher… or something else entirely?

Pupil Insights combines the reliability of science with the reality of schools to explore questions that school leaders really care about.

Pre-validated measures minimise the effects of ​‘demand characteristics’. Rather than one direct question, we can ask a series of items that tap into various components of a related psychological theory.

Looking again at a sense of belonging at school, we could ask pupils to what extent they agree with items such as:

• I can really be myself at school.

• I see myself as a part of this school.

• I think that people care about me in this school.

  
  

Each item taps into a component of school belonging theory developed within the research literature (interested? Read more about this here).

Now, we are capturing a greater breadth of pupil responses, at far greater depth, and, importantly, with much greater accuracy.

Trust the tools: How can we be sure what they are measuring?

So, can I just break down my question into a variety of items?

No. Within psychological science, once a measure has been created it must undergo intense evaluation to prove its reliability.

A lot of responses are collected and subject to rounds of statistical analyses. Within the analyses, every item is checked to ensure it is related to all of the other items in the scale. 

The analyses also checks that there aren’t any unusual patterns forming within the responses – this could indicate that something else is being measured… a theoretical imposter, if you will.

Such a rigorous process ensures theoretical and statistical validity. We can therefore trust that the final product is indeed measuring what it was intended to.

Raising the bar: How can pre-validated measures improve teaching and learning?

Imagine it had come to your attention that Year 9 girls were increasingly absent from PE lessons. What should be done to improve attendance?

Before this is even considered, the motivation behind absenteeism must be investigated. Staff may suspect that self-esteem and confidence is particularly low amongst the current cohort of Year 9 girls, but this is likely anecdotal and potentially influenced by biases. At this point, we cannot be sure whether self-esteem and confidence are associated with this issue.

Cue: pre-validated measures.

Year 9 pupils could be invited to complete a survey including pre-validated measures of self-esteem and confidence.

Mean scores from these responses may highlight that girls report lower scores of self-esteem and confidence. We cannot be certain that these scores are directly causing the absenteeism, but it certainly gives us reliable evidence to inform next steps.

Initiatives could be developed with a focus on fostering Year 9 girls’ self-esteem and confidence; for example, PE teachers could share best practice on including activities within lesson plans that are embedded within developing self-esteem and confidence.

Importantly, the use of a pre-validated measure in this case has provided the school with reliable evidence that may explain motivations behind a behaviour. This knowledge can thus be used to inform targeted approaches to teaching and learning with the ultimate purpose of supporting pupils’ needs.

The use of a pre-validated measure provides the school with reliable evidence that may explain motivations behind a behaviour.

Pupil Insights

Sourcing pre-validated measures is time-consuming and tricky, particularly where many are hidden within jargon-rich empirical articles. Our Pupil Insights service makes life easier by providing pre-validated measures ready-to-use in school settings. We provide themed packages targeted at key areas of interest to schools (eg inclusion, wellbeing) with custom combinations also available.

Drawing upon my statistical expertise, deeper analyses can be conducted to provide answers to more specific questions (eg does a gender difference exist between pupils’ sense of connectedness depending on their social relationships?).

Importantly, Pupil Insights combines scientific knowledge with school know-how to provide evidence-based data-driven acumen. Findings from Pupil Insights can be used to inform goal-driven next steps to support pupil needs.