Constructing a Novel Instrument to Measure Social-Emotional Self-Regulation in Students Aged 8-18
Abstract
Background
The authors identified a need for instruments that measure a task model of self-regulation as opposed to an emotion model of self-regulation. Such instruments are particularly applicable to student populations.
Objective
This paper explains the construction of one such instrument, Steer Tracking. A four-factor bi-polar conceptual model of four necessary self-regulatory tasks is presented: Trust of Self, Trust of Others, Self-Disclosure and Seeking Change.
Methods
A novel assessment method was developed, requiring a participant to imagine a mental space in which they perform the four self-regulatory tasks. The instrument was deployed in populations of students aged 8-18 attending UK primary and secondary schools. Principal Component Analyses evaluated the proposed four-factor structure across two age groups: 8 to 12 years olds (n = 2171) and 13 to 18 years old (n = 658). A Support Vector Machine (SVM) model in a separate sample (n = 2518) evaluated the assessment’s utility in identifying students who display risk on three wellbeing measures: experiencing bullying, thinking about or engaging in self-harm, and struggling to cope with pressure at school.
Results & Discussion
Analyses provided initial support for the validity of the conceptual model and its ability to identify at-risk students. Key instrument features such as non-standardisation and generalised versus in-school comparison are explained.
Conclusion
Application, relevance and potential benefits of the Steer Tracking instrument for educators and school managers are explored.
1. INTRODUCTION
Social-emotional self-regulation has been defined as “taking in information, weighing choices and consequences, and making adaptive choices to attain a particular goal” [1]. There is considerable longitudinal research to evidence that early self-regulation is associated with future psychological functioning, social competence and academic outcomes [2, 3]. As such, self-regulation is fundamental to school readiness and underlies positive psychosocial development [4].
Existing self-regulation self-report instruments have been predominantly built on an emotion-regulation model. Measures identify the subject’s management of negative (and some positive) emotional states, for example, anxiety, sadness and anger. Prominent examples include the Emotional Regulation Questionnaire (ERQ) [5], Cognitive Emotion Regulation Questionnaire (CERQ) [6], Difficulties in Emotion Regulation Scale (DERS) [7], and Children’s Emotion Management Scale (CEMS) [8]. Some instruments, such as CERQ, categorise the mental strategies used by the subject to manage their emotional state. For example, Self-blame, Other-blame, Rumination, Catastrophizing, Putting into perspective, and Positive refocusing.
As an alternative to emotion regulation, self-regulation can also be understood as the maintenance and adaptation of the self in relation to others around you. This concept focuses on the tasks which are required to maintain and negotiate inter-personal relations. For example, one task may be to retain a sense of distinctiveness in the relationship; another may be to establish a bond of trust; another may be to explore novel opportunities together; another may be to manage boundaries around personal information and self-disclosure. Tasks are formative towards one’s behaviours, constructive to one’s identity and ecological in being a response to the environment one finds oneself in.
Because of the specific context of education, there may be particular utility in developing self-report instruments which are able to measure the development of self-regulation in students [9]. Students are in a period of self and social formation during adolescence within which self-regulation is specifically relevant [1]. They are engaged in a school environment which has a duty of care to have a positive impact on their formation; a means of measuring that impact would, therefore, be of benefit.
A novel instrument would be required to measure self-regulation as task management within schools. In this paper, we set out a novel four-factor model of self-regulation related to tasks. We then explain an assessment - Steer Tracking - developed to measure the self-regulation of these tasks in students between the ages of 8 and 18. We present two studies with samples of UK primary and secondary school students which demonstrate the validity of the four factor structure and the practical utility of the Steer Tracking instrument. Finally, we consider its potential application to support mental health management in schools.
1.1. A Four-factor Model of Self-regulation
The authors conducted literature reviews to identify tasks necessary for healthy social-emotional self-regulation. We gave priority to concepts that shared three common features: empirical foundation; clarity and accessibility of meaning to a potential lay person; logical conceptual coherence when related to the other concepts under consideration. The purpose was to arrive at an overall model that was both evidentially robust but also useable by a lay person. The potential praxis of the instrument in the field as a utility was central to its design methodology.
Four factors are defined and summarised in the sections below and described further elsewhere [10-14]. These four tasks were identified as necessary components of social-emotional self-regulation but should not be regarded as an exhaustive list of self-regulatory domains:
- Self-Disclosure: the degree to which a person chooses to share or hold back their thoughts, feelings, ideas and opinions
- Trust of Self: the degree to which a person trusts or questions their own qualities, skills, ideas and opinions
- Trust of Others: the degree to which a person trusts or questions others’ qualities, skills, ideas and opinions
- Seeking Change: the degree to which a person seeks to bring about or reduce change, novelty and risk
1.1.1. Self-Disclosure
Self-Disclosure is a process of communication through which one person reveals themself to another [15]. It comprises everything an individual chooses to tell the other person about themself, making themself known to the other [16].
Being able to purposively self-disclose to another is widely recognised as an important component of a young person’s development. Some psychologists would say that self-disclosure serves not only as an indicator of a healthy personality, but also as a means by which one may be achieved [17]. Seminal research shows that young infants are capable of some management of self-disclosure, internalising their feelings whilst choosing to display something different to those around them [18]. Maturation improves self-monitoring skills enabling children to reflect on the consequence and purpose of disclosing aspects of themself in different contexts and to different people [19]. Self-disclosure is “intrinsically rewarding” [20] and associated with positive relationship development [21, 22]. In contrast, the intentional keeping of secrets is associated with both intra- and inter-personal costs, including stunted psychosocial development [23].
Self-disclosure is likely to include the sharing of feelings as well as personal ideas and thoughts. To function healthily, it is suggested a person needs at least one significant relationship in which they can disclose to a very significant degree [24]. Self-disclosure between peers and with trusted adults provides young people with foundational experiences that equip them with the skills to form appropriate levels of intimacy within their later relationships [25].
While self-disclosure is beneficial to healthy adolescent functioning, and particularly the initiation, maturation and sustaining of close relationships [26, 27], the ability to purposively choose to hold back from disclosing aspects of ourselves to others is also an important developmental goal of childhood and adolescence [19, 21]. Indeed, choosing when to disclose, what to disclose and to whom is critical in the formation of healthy relationships [25]. For instance, sharing overly personal information may lead to social rejection [21]. Sharing negative experiences and emotions can lead to emotional reactivation and prolong the negative state [27]. Within relationships, the reciprocal sharing of problems and negative feelings (“co-rumination”) can be associated with depressive symptoms [28].
High self-disclosure also has implications for personal safety, particularly online. Increasingly, adolescents use the internet both to interact with their peers and form relationships with people they have never met in person [29]. Dangers of inappropriate self-disclosure online include cyberbullying, harassment, and possible victimisation [30]. Children and adolescents who develop a strong bias towards high disclosure may lack the self-monitoring to avoid putting themselves, or others, at risk.
1.1.2. Trust of Self
We trust our own qualities, skills, ideas and opinions when we place value upon who we are. We recognise ourselves as distinct and expect others to notice, value and respect our contribution in the world. In trusting our qualities, skills, ideas and opinions, we exhibit a favourable evaluation of ourselves. In questioning these qualities we exhibit a less favourable evaluation of ourselves; we see ourselves as less distinct, and are less expectant of being noticed and valued in the world.
Trust of Self is a model of relational self-definition rather than self-esteem. Whilst self-esteem describes an intrinsic sense of self-worth [31], self-definition is related to self-concept and describes the degree to which one sees oneself as distinct from others around you. Self-definition incorporates the concept of closedness and autonomy. To have a high Trust of Self is to be strongly self-defined, independent of, and closed to others’ evaluations. To have a low Trust of Self is to be weakly self-defined, dependent on, and open to others’ evaluations.
Though the two concepts are distinct, self-esteem can contribute to self-definition. Leary and colleagues provide evidence that children with higher self-esteem have an implicit assumption that they are relationally acceptable to others; when faced with critical feedback, these children are more likely to rebuff it [32]. Recent reviews and meta-analyses affirm that healthy self-esteem is longitudinally associated with positive relationship development [33, 34].
Within our bi-polar model of self-regulation, biases towards either high or low Trust of Self may be associated with poor social-emotional development. High Trust of Self may be informed by a child’s appropriate understanding of their qualities, skills, ideas and opinions, yet it can also be informed by an inflated, superior view of self. In the same way, a child’s low Trust of Self may be influenced by an inaccurate perception of their qualities, skills, ideas and opinions which diminishes their sense of value.
People with elevated views of themselves sometimes exhibit socially undesirable interpersonal behaviours such as interrupting and talking over others [35]. They can alienate others who see them as haughty, conceited or snobbish [36], and may exhibit narcissistic traits which contribute to behavioural problems [37] that are, in turn, associated with negative psychosocial outcomes [38].
Conversely, children with lower Trust of Self may have an implicit assumption that they are less relationally acceptable. This may manifest as greater rejection sensitivity, which is detrimental to psychological well-being [39]. When faced with critical feedback, these children may be more likely to assume some deficit on their part, potentially manifesting in a set of defensive behaviours which manage the perceived threat of rejection [40]. Children who anticipate rejection may adapt their behaviour to increase relational acceptance. As such, they might offer different and less stable social performances to different audiences [41].
1.1.3. Trust of Others
A central component of children’s psychosocial functioning is the development of interpersonal trust [42]. When we trust others, we see them as supportive of our needs. We believe that the requests they make of us are of good intent and will benefit us, so we are responsive to their requests. By extension, children’s interpersonal trust facilitates positive relationships with peers and teachers [43] and is associated with school adjustment [44].
Attachment theorists suggest that the degree to which we trust or question others is shaped by our early attachments with caregivers which form working models of future relationships [18, 45]. However, these internal working models can change in response to impactful life events and aspects of the family dynamic including parent-child communication, family conflict, and parental separation [46-48].
Research indicates that a child’s trust of their caregiver to meet their needs has a significant impact on a child’s subsequent emotional self-regulation [48, 49]. Over time, effective emotional self-regulation gives children and adolescents a repertoire of skills including the ability to find healthy strategies to deal with uncomfortable feelings, exhibit impulse control, delay gratification, inhibit inappropriate responses in social situations, and resist distraction [50, 51, 43, 44].
However, children who have an implicit trust in the availability of others are not necessarily advantaged. If a parent leaps in to protect their child from conflict or distress, this over-involvement may deny their child the opportunity to develop self-confidence and leadership skills [52] and may contribute to the emergence of depression and anxiety [53]. If an adolescent becomes expectant of a teacher’s responsibility to instantly support them they are unlikely to develop the self-efficacy essential for high performance [50].
Low Trust of Others can also have deleterious effects. Children who experience their early caregivers as less emotionally available are likely to exhibit a reluctance to rely on others to acknowledge, support and regulate their emotional state [48]. They are more likely to develop a repertoire of self-reliant strategies, which may have a limiting impact on their social-emotional development [48, 54] and later self-regulation [50, 51].
Indeed, there is a body of research to support that both polar high and low levels of interpersonal trust are detrimental to psychosocial development [55]. Rotenberg and colleagues have demonstrated that, compared to children in the middle range of trust beliefs, those with very high or low levels of trust exhibit lower social skills, face greater peer rejection, experience more interpersonal aggression, and spend more time alone [56]. These findings support the “centralist approach to trust”, in which trusting “too much” or “too little” is associated with poor emotional and social functioning [57].
1.1.4. Seeking Change
The task of managing the interplay between change and stability is something that we do every day from early childhood [58]. An infant uses a range of seeking behaviours to seek proximity from their caregiver, while also detaching from them to explore their world [59].
Healthy development requires change and risk, as well as stability and predictability. Teaching children and adolescents to optimally self-regulate their response to change and risk is crucial in supporting their mental health, social competencies and achieving good learning outcomes [60].
Children and adolescents with a fixed bias towards high Seeking Change will direct their attention towards what they do not yet have. Their drive for change may be fuelled by a desire for novel sensation seeking: to experience something new and exciting, or feel a sense of thrill or anticipation [61, 62]. Such young people may struggle to develop perseverance and stamina when tasks are challenging, or sustain attention and focus [63]. They may exhibit poor impulse control, and be less able to monitor and regulate their emotional responses [64], which contributes to poor school adjustment [65]. They may struggle to delay gratification with negative consequences; studies evidence that children and adolescents who struggle to delay gratification are more likely to use cigarettes, alcohol, and drugs and underperform academically [66, 67]. They may gravitate towards highly arousing or risky situations [68], something that the teenage brain is particularly predisposed towards, especially in the company of their peers [69].
In contrast, some children and adolescents exert their influence to reduce change and risk; they seek security, stability, predictability and consistency. In seeking to reduce change, a child or adolescent exerts control to make their world more predictable. Children who develop a fixed bias toward reducing change may withdraw from novel situation and may be timid, fearful, and shy with unfamiliar people [70], exhibiting a heightened risk of developing social anxiety [71, 72]. Such children may lack the resilience to cope with risk, change or uncertainty [73]. They may also be more prone to internal rumination and fixating on concerns, which ultimately will increase their levels of anxiety [74].
1.2. An Ecological Understanding of Self-regulation
Importantly, the proposed four factor model of self-regulation is ecological. It presupposes that healthy self-regulation is not fixed but rather relies on the ability to adjust one’s Trust of Self, Trust of Others, Self-Disclosure, and Seeking Change according to the circumstances. This ability to adjust gives a person the ability to flexibly and appropriately respond to the situation at hand. Flexibility is contrasted with having a fixed position or bias (either high or low) for any single factor. A fixed position presents as a habitual and unchanging response pattern, which can lead to mis-reaction to the signals of people and situations around you.
To self-regulate flexibly and ecologically, an adolescent must pay attention to the cues around them, as well as their own internal cues, and make a judgement about whether this is a time to, for example, trust or question themselves. For instance, a student may need to lower their Trust of Self when engaging in an unfamiliar topic in order to be open to guidance, but then increase their Trust of Self when standing on a stage, presenting confidently to an audience.
Structurally, each factor is therefore construed as a bi-polar scale that a person must learn to regulate. For example, as depicted in Fig. (1), a child with a fixed bias of low Self-Disclosure would endorse that they always keep their thoughts to themselves, while a child with a fixed bias of high Self-Disclosure would endorse that they never keep their thoughts to themselves. Either can have negative consequences for social-emotional development. Alternatively, a child demonstrating healthy, context-dependent flexibility would endorse that they can choose when to share their thoughts with others and when to keep their thoughts to themself.
An analogy of steering a car can be employed: an individual who develops a persistent bias in their self-regulation may be subject to a higher risk of crashing on the social-emotional road. In this analogy, self-regulating the four tasks is akin to steering safely and appropriately on the social-emotional road. Students who can steer are able to adjust their social-emotional responses appropriately to their situation, thus reducing their social and emotional risks.
1.3. Using Imagination to Measure Ecological Self-regulation
Having defined a conceptual model, we needed a method for measuring a subject’s self-regulation of these four tasks. Because tasks are neither traits nor states but are choices made in response to circumstance, they may be conscious or unconscious. It is, therefore, important that the measure does not rely upon a conscious perception and articulation of a subject’s self-concept by asking subjects to report on their historic attitudes or behaviours. Rather, this measure should prime subjects to make choices in real-time response to relational contexts.
An instrument designed to recruit the subject’s imagination is therefore well-suited to our self-regulation model. The imagination may provide the de-coupled mental environment in which experimental actions, choices and thoughts are simulated, played out, selected or inhibited [75-77]. Studies have identified the link between decision-making and imagination; Decety and Grezes find that imagination plays a central role in organising our behaviours [75], and Schacter and colleagues present evidence that the brain projects forward a method of self-operation prior to then enacting that projected sequence, serving as a route map directing action [78, 79].
There is also evidence that imagination integrates with other circuits in the executive function system, which provide a mechanism for self-regulatory decision making, attentional focus and effortful control [80]. Brain imaging research using fMRI techniques has described the role that imagination plays in mental tasks associated with goal setting, self-representation and self-organisation and prospective, future memory [81]. Sub-regions within the hippocampus play various roles in the mental simulation of possible events and actions [78, 79, 82].
The involvement of the imagination in self-regulation suggests that utilising imaginative processes may be a useful method to measure a subject’s ability to self-regulate. Rather than (traditionally) asking a subject to review how they self-regulate in different scenarios, it may be possible to prime a subject with imagined scenarios and trigger an imagined response which itself reveals their capacity to self-regulate. On this basis, a novel instrument named “Steer Tracking” was developed as an imagination-based measure of ecological self-regulation. The name is derived from the task of steering each factor, as well as the name of the entity funding the research.
1.4. Steer Tracking: A Novel Four Factor Measure of Ecological Self-regulation
We began to develop Steer Tracking in 2012. Item wording was evolved through a series of trials with student samples testing the accessibility of language, instrument layout and design, and length. Language was designed to be accessible for students aged 8 to 18 in mainstream education. Efforts were made to minimise the number of items required in order to reduce the burden on school time. The computer-based assessment reduced the demand on schools in administering the assessment. Initial pilots in 2013 involved students from two UK schools, one primary and one secondary, involving students aged 9 to 10 and 13 to 15. Further trials in 2014 involved 16 UK secondary schools and five UK primary schools. Since its commercial launch in 2016, more than 300,000 students in 250 schools have used the assessment to date.
The Steer Tracking assessment is separated into two parts: the first evaluates the student’s self-regulation in general, while the second focuses on their self-regulation within the context of school. These two parts will be respectively referred to as the General and School assessments hereafter. The General and School data are compared relative to each other for each student, rather than against a standardised norm, with the purpose of revealing the relative impact of the school context on the child’s self-regulation. This impact may be positive or negative; scores may identify students who are struggling particularly within the context of school and, conversely, struggling outside the context of school. This provides valuable information for schools, who are able to quantify the effects of their environment on students’ social-emotional self-regulation.
1.4.1. Assessment Design
Students were first presented with the General assessment. To begin, students followed instructions on a digital dashboard which prompted them to create an imagined space in their minds. Students listened through headphones to the following script being read slowly with pauses to allow time for the subject to visualise their imagined space:
Imagine yourself in a place that you know, or a place that you can imagine. You can choose anywhere you like. Take a look around. What can you see? What can you hear? What can you smell? Reach out your hand and touch something. What does it feel like?
Stop for a moment and look around. Choose a part that you would like to keep for yourself. From now on we are going to call this YOUR SPACE.
What can you put around the edge of YOUR SPACE to show it is yours? Perhaps a rope, a fence, a line, a ditch, a wall or anything that you can think of. What is happening outside your space? What is happening inside your space?
Having generated this imagined space, the subject is then prompted with imagined scenarios by the dashboard. These scenarios were designed to trigger relevant self-regulatory responses related to the four tasks of the conceptual model. The structure of the prompts and the response options are in the form of a question followed by a six-item Likert scale. For example,
Would a visitor looking at your space know that it belongs to you?
(Definitely not / No / Not really / Maybe / Yes / Definitely)
Four items of such structure were developed to assess each of the four tasks, giving 16 items in all:
- Items for Trust of Self explore the distinction and definition of the subject’s imagined personal space from the space they imagine around it. For example, Is your space different to what is outside your space? and Something changes outside your space. Does your space change too?
- Items for Trust of Others explore the degree to which another person would be trusted in the subject’s imagined space. For example, Someone in your space asks you to do something, will you do it? and You need something in your space; do you want other people to help you get it?
- Items for Self-Disclosure explore to the degree to which the subject keeps aspects or parts of their space private from those who might enter. For example, Someone comes into your space; will you let them see what you are thinking? and Imagine you can choose to keep part of your space private; how much would you keep private?
- Items for Seeking Change explore the degree to which the subject is interested in and open to their space changing. For example, How often do you like things to change in your space? and Do you like trying things you haven’t done before in your space?
Having responded to the 16 General assessment items, students then listen to a second recording to prime them for the School assessment stage:
Now imagine that you are back in your space. Imagine that you can see those from your school in your space with you. What do you see those in your school doing? What do you see yourself doing in your space with your school? How are you feeling about your school in your space with you?
The subject was allowed to imagine whatever aspect or version of school they chose when primed by the visualisation recording. The environment of school was deliberately not prescribed more narrowly, for example, to a particular class or peer group. The aim of this was to obtain as subject-generated a response as possible. Students are then presented with the same 16 questions as in the General assessment, with item text modified to reference the context of school.
For example, in the first stage, one item reads: Would a visitor looking at your space know that it belongs to you? In the second stage, this item reads: Would people in your school know that this space belongs to you, just by looking at it?
1.4.2. Assessment Scoring
Students’ 32 item responses for the two stages of Steer Tracking generate raw scores which are then computed to give a score from 0-15 for each of the four tasks, separately for the General and School assessment. Items are reversed in a random pattern across the assessment to reduce the effects of passive responding; the pattern differs slightly for the instrument used with 8-12 and 13-18 year students. In addition, algorithms check patterns of scores against likely manipulation or disengagement (for example, repeated scores across the assessment, unlikely sequences of scores or rapid speed). Where the algorithm detects such patterns, the software shows students a message inviting them to re-complete the assessment. Items are equally weighted. There are no fake items.
The degree to which a score deviates from the numerically median factor score of 7.5 indicates that the student self-regulates that task either higher (7.5-15) or lower (0-7.5). This deviation reflects the degree of bias a student is exhibiting in that task. The comparative difference in General versus School scores can be attributed to the impact of the inserted priming context, providing a measure of the impact the context of School on a student’s self-regulation (Fig. 2).
Students, for example, who obtain a score of 4.5-11 will have answered most items with maybe/sometimes. Their responses indicate that how they manage that task would be contingent on the situation. According to the conceptual model, these students show an ability to flexibly self-regulate; they pay attention to the situation they are in and adapt or adjust their tasks according to the needs of that context.
In contrast, students who obtain a score of 0-3 or 12-15 will have answered the items with definitely/ definitely not. Their responses indicate they would always tend to manage that task in the same way, regardless of the situation. These students, according to the conceptual model, show less ability to self-regulate – are less likely to adapt or adjust their tasks according to the needs of that context.
Low bias scores of 0-3 and high bias scores of 12-15 were determined as cut-offs for flagging as ‘polar’ on the basis of statistical and practical considerations. Statistically, across pilot samples, these represented roughly the 85th percentile. Practically, teachers need to prioritise a manageable percentage of students, and feedback supports this proportion to flag. Students flagged as having polar biases in more than two factors were flagged as ‘highest priority’. In addition, the tool ranks numerical discrepancy between School and Generalised scores; this enables teachers to locate the context of risk. For example, polar biases exhibited in School but not General scores would indicate that the context of school was a source of dysregulation. By contrast, polar biases in the General assessment but not in School scores would signify concerns relating to domestic or personal factors.
The teacher interface (Fig. 3) presents data in a simple row-by-row array, using numbers, colours and icons to indicate students of priority concern. Options to rank, sort and filter students optimise the identification process. Interactive student names allow teachers to open further sub-screens to expand a particular student’s data and obtain verbal explanations of associated risks. From these screens, teachers can complete an action planning process by which they select suggested school actions which may be put in place to better support that particular student.
1.5. An Initial Validation of the Steer Tracking Model of Self-regulation
To support the validity and practical utility of the Steer Tracking model of social-emotional self-regulation, this paper presents analyses in two samples which evaluate the assessment’s four factor structure. In addition, associations with three specific wellbeing outcomes were measured: bullying, self-harm and not coping with pressure. We selected these from a previous informal field review we had conducted with teachers about student wellbeing risks of greatest concern. These three risks were often described by teachers as being ‘hard to detect’, indicating that existing wellbeing instruments may face obstacles in helping teachers identify students vulnerable to these specific risks. In particular, teachers referred to adolescents masking their responses to direct questions about these risks in surveys. As such, the ability of the Steer Tracking instrument to detect these risks was considered a suitable context in which to evaluate its potential ultimate utility to schools as a wellbeing tool.
2. STUDY ONE: VALIDATION OF THE “STEER TRACKING” FOUR FACTOR STRUCTURE
2.1. Methods
2.1.1. Participants
Data were collected from 2,829 students across three primary and six secondary schools in England. Of these, five schools were fee-paying and four were state funded. Of the fee-paying schools, three were day schools and two were mixed day and boarding students. The sample included 991 girls and 1180 boys aged 8 to 12 years old, and 167 girls and 491 boys aged 13 to 18 years old. Students were not asked to disclose further demographic information (e.g., ethnicity or socioeconomic status).
2.1.2. Procedures
Data collection took place in school classrooms during 2013 and 2014, under the supervision of teachers. Students completed the Steer Tracking General and School assessments. Schools were approached to participate in the study directly by the authors. Schools were selected on their willingness to participant in research as part of a process to develop a new tool to support student social-emotional development. Schools and students were not paid to participate, and Steer Tracking was provided free of charge. Schools received a study report providing wellbeing conclusions at a population level for their school and had the option to view assessment data at an individual student level.
Schools signed a data sharing agreement through which the school gave consent as a surrogate for minors (article 16 Declaration of Helsinki). The data were collected pre-GDPR. Schools used their own policies to determine how to communicate participation with parents. Students were given the choice to opt out of the assessments and if so, were provided with alternative activities by their teachers. Students understood that their responses would be visible at an individual student level to the school.
2.2. Results
2.2.1. Age and Gender Descriptive Statistics and Norms
Descriptive statistics are presented in Table 1. First and second standard deviations for the two main age groups (ages 8 to 12 and ages 13 to 18) are provided for assessment scores. Distributions behave within normal parameters, with approximately 95% of scores falling within 2 standard deviations of the mean. Mean and median scores are also approximately equal. It is appropriate to use mean and first and second standard deviations as representations of spread to provide norms for age and gender. Distribution norms show variance by each age, gender and assessment type. This indicates that developmental, gender and ecological factors may contribute to a student’s self-regulation of the four tasks.
| Age | Gender | n | Mean | Median | Mode | SD |
|---|---|---|---|---|---|---|
| 13-18 | F | 167 | 7.61 | 7.5 | 7.5 | 2.54 |
| 13-18 | M | 491 | 8.38 | 8.25 | 8.25 | 2.45 |
| 8-12 | F | 991 | 9.11 | 9 | 9.75 | 2.28 |
| 8-12 | M | 1180 | 9.24 | 9 | 9.75 | 2.62 |
| Age | Gender | N | Mean | Median | Mode | SD |
| 13-18 | F | 167 | 9.47 | 9 | 9 | 2.62 |
| 13-18 | M | 491 | 8.83 | 8.25 | 8.25 | 2.71 |
| 8-12 | F | 991 | 10.39 | 10.5 | 9 | 2.14 |
| 8-12 | M | 1180 | 9.94 | 9.75 | 9 | 2.37 |
| Age | Gender | N | Mean | Median | Mode | SD |
| 13-18 | F | 167 | 6.51 | 6.75 | 6.75 | 2.31 |
| 13-18 | M | 491 | 7.31 | 7.5 | 7.5 | 2.52 |
| 8-12 | F | 991 | 5.96 | 6 | 7.5 | 2.55 |
| 8-12 | M | 1180 | 6.5 | 6.75 | 6.75 | 2.73 |
| Age | Gender | N | Mean | Median | Mode | SD |
| 13-18 | F | 167 | 7 | 6.75 | 7.5 | 2.34 |
| 13-18 | M | 491 | 7.64 | 7.5 | 7.5 | 2.66 |
| 8-12 | F | 991 | 7.46 | 7.5 | 5.25 | 2.45 |
| 8-12 | M | 1180 | 7.38 | 6.75 | 5.25 | 2.95 |
2.2.2. Principal Component Analysis, Age 8-12 Group
Within the Age 8-12 group, 46.8% of the variance was accounted for by four latent factors, each of which showed an eigenvalue of >2 (Table 2). The remaining 53% was accounted for by more than 16 latent factors (Fig. 4). Factor loadings for items across the four principle latent factors show relatively strong associations with the conceptual constructs of the data model: Trust of Self, Trust of Others, Self-Disclosure and Seeking Change. Factor loadings for items of Trust of Others and Self-Disclosure were strongest overall, but also exhibited some cross-factor loading (Table 2). Overall factor divergence was moderate, indicating that some items could be refined to improve factor discrimination (Fig. 5).
| Item | Factor | F1 | F2 | F3 | F4 |
|---|---|---|---|---|---|
| 1 | Trust of Self | -0.411 | -0.371 | 0.096 | -0.369 |
| 2 | Trust of Self | 0.447 | 0.374 | -0.353 | -0.077 |
| 3 | Trust of Self | -0.577 | 0.037 | 0.414 | -0.358 |
| 4 | Trust of Self | 0.348 | 0.798 | -0.397 | 0.900 |
| 5 | Trust of Others | 0.279 | 0.544 | -0.458 | -0.642 |
| 6 | Trust of Others | -0.082 | 0.677 | -0.581 | -0.305 |
| 7 | Trust of Others | -0.283 | 0.834 | 0.243 | -0.074 |
| 8 | Trust of Others | -0.062 | 0.727 | -0.156 | -0.395 |
| 9 | Self-Disclosure | 0.256 | -0.454 | -0.577 | -0.371 |
| 10 | Self-Disclosure | -0.153 | 0.620 | 0.436 | -0.083 |
| 11 | Self-Disclosure | 0.116 | 0.607 | 0.713 | 0.167 |
| 12 | Self-Disclosure | 0.212 | -0.493 | -0.178 | 0.022 |
| 13 | Seeking Change | 0.150 | 0.547 | 0.064 | -0.452 |
| 14 | Seeking Change | -0.261 | -0.171 | -0.660 | 0.096 |
| 15 | Seeking Change | 0.206 | 0.041 | 0.549 | 0.165 |
| 16 | Seeking Change | 0.908 | -0.309 | 0.648 | -0.452 |
| Eigenvalue | 4.522 | 3.317 | 2.364 | 2.096 | |
| Variability (%) | 17.201 | 12.620 | 8.993 | 7.974 | |
| Cumulative % | 17.201 | 29.821 | 38.813 | 46.787 |
2.2.3. Principal Component Analysis, Age 13-18 Group
Within the Age 13-18 group, 42.4% of the variance was accounted for by four latent factors, each of which showed an eigenvalue of >2 (Table 3). In other characteristics, factor loadings reflected similar patterns to the 8-12 age group (Table 3), with moderate factor divergence (Fig. 5).
| Item | Factor | F1 | F2 | F3 | F4 |
|---|---|---|---|---|---|
| 1 | Trust of Self | -0.083 | -0.331 | 0.200 | -0.097 |
| 2 | Trust of Self | -0.565 | 0.401 | -0.247 | -0.217 |
| 3 | Trust of Self | -0.148 | -0.071 | 0.462 | 0.694 |
| 4 | Trust of Self | 0.738 | 0.873 | -0.159 | -0.449 |
| 5 | Trust of Others | -0.553 | 0.653 | -0.479 | 0.099 |
| 6 | Trust of Others | -0.367 | 0.673 | -0.181 | -0.019 |
| 7 | Trust of Others | 0.342 | 0.533 | 0.331 | -0.526 |
| 8 | Trust of Others | -0.159 | 0.723 | -0.152 | 0.347 |
| 9 | Self-Disclosure | -0.270 | -0.292 | -0.519 | -0.017 |
| 10 | Self-Disclosure | -0.023 | 0.505 | 0.450 | 0.368 |
| 11 | Self-Disclosure | 0.025 | 0.418 | 0.907 | 0.316 |
| 12 | Self-Disclosure | -0.171 | -0.361 | -0.101 | -0.428 |
| 13 | Seeking Change | -0.383 | 0.539 | 0.052 | -0.175 |
| 14 | Seeking Change | 0.256 | -0.071 | -0.632 | 0.456 |
| 15 | Seeking Change | 0.091 | -0.055 | 0.412 | -0.211 |
| 16 | Seeking Change | -0.734 | -0.238 | 0.588 | -0.469 |
| Eigenvalue | 3.725 | 2.955 | 2.338 | 2.068 | |
| Variability (%) | 14.269 | 11.320 | 8.958 | 7.921 | |
| Cumulative % | 14.269 | 25.590 | 34.548 | 42.469 |
3. STUDY TWO: ASSOCIATIONS BETWEEN “STEER TRACKING” AND WELLBEING OUTCOMES
3.1. Methods
3.1.1. Participants
Data were collected from 2518 students aged 12 to 18 years old across 16 secondary schools in England. Of these, fourteen were fee-paying schools and two were state-funded. Of the fee-paying schools, six were day schools, and eight were mixed day and boarding students. The sample included 48% girls and 52% boys. Students were not asked to disclose further demographic information (e.g., ethnicity or socioeconomic status).
3.1.2. Procedures
Data collection took place in school classrooms between November and December 2014, under the supervision of teachers. Schools were invited to participate in the study directly by the authors. Schools were selected on the basis of an existing interest in research, as indicated by participation in previous studies. The range of types of school (fee-paying, day and boarding) reflected a market mix in the UK. Schools and students were not paid to participate, and Steer Tracking was provided free of charge. Schools were offered a study report and were invited to attend a webinar explaining over-arching wellbeing conclusions at the school level. Schools chose to participate based on their understanding of the potential advances the assessment might make in supporting students’ social-emotional development.
Schools signed a data sharing agreement through which the school gave consent as a surrogate for minors (article 16 Declaration of Helsinki). The data was collected pre-GDPR. Schools used their own policies to determine how to communicate participation with parents. Students were given the choice to opt out of the assessments and if so, were provided with alternative activities by their teachers.
Students completed the Steer Tracking General and School assessments. Students were informed that their Steer Tracking assessments may be seen by their teachers as well as researchers. Students were then asked to respond to three questions about their wellbeing. Students understood that their responses to these questions would be strictly anonymous and would not be seen by their school. Students understood that they would not have the opportunity to view their personal assessment results.
3.1.3. Measures
In addition to the Steer Tracking General and School assessments described in section 1.4, students responded to three questions about their wellbeing which were written by the researchers for the purpose of this study.
- Bullying: Have you been bullied in the past year? (Never / No / Sometimes / Yes / Often)
- Self-harm: Have you self-harmed or thought about self-harming in the past year? (Never / No / Sometimes / Yes / Often)
- School Pressure: How able are you to manage the pressure of work at school? (Well / Quite well / Ok / Not very well / Not well)
3.1.4. Data Analysis
Theoretically, we did not expect any single factor to be associated with pressure/ bullying/ self-harm. Real-world use of the tool suggested that patterns across the factors would be more likely to show specific associations to these risks.
A Support Vector Machine (SVM) model was therefore used as a method of testing the fit of many models of the combined four factors to students experiencing bullying, thinking about or engaging in self-harm, or struggling with school pressure. SVM is a supervised machine learning model that analyses data using learning algorithms to solve classification and regression tasks.
To begin, binary variables were generated based on responses to the three wellbeing questions:
- Bullying: 0 = Never, No, Sometimes 1 = Yes, Often
- Self-harm: 0 = Never, No 1 = Sometimes, Yes, Often
- School Pressure: 0 = Well, Quite well 1 = Ok, Not very well, Not well
The model was first trained on the dataset with a Radial Basis Function (RBF) kernel to account for non-linear data (i.e. both high and low Steer Tracking scores are expected to be associated with bullying, self-harm, and school pressure). The parameters of the model were determined using 10-fold cross-validation. The SVM model was then applied to the non-training data.
The model was cross-validated to test for the probability of accurately classifying students who belong to either category for each wellbeing variable (e.g., bullying/no bullying) based on STEER Tracking factor scores; consistency between the two measures indicates a low chance of false readings.
3.2. Results
3.2.1. Bullying
The model achieved an 80% accuracy (83/78% both classes) in identifying students who had experienced bullying in the past year. Experiencing bullying was associated with a significantly higher level of poor self-regulation and specifically with low Self-Disclosure. Students who had a bias towards low Self-Disclosure, as well as a high degree of dysregulated bias (a high deviation from the mean across their item scores), were more likely to report bullying.
3.2.2. Self-Harm
The model achieved an 80% (82/78% both classes) accuracy in identifying children who endorsed self-harm thoughts or behaviours. Self-harm was associated significantly with overall poor self-regulation as well as with low Self-Disclosure and high Seeking Change. Students who show a combination of these factors (high Seeking Change, low Self-Disclosure and high dysregulation) were the most likely to report self-harm.
3.2.3. School Pressure
The model achieved an 83% (88/77% both classes) accuracy in identifying children who were not coping well with pressure at school. Struggling to cope with pressure was statistically associated with overall poor self-regulation as well as, specifically, with Self-Disclosure and Seeking Change. Students who displayed a bias toward low Self-Disclosure or low Seeking Change, or a high degree of dysregulated bias (a high deviation from the mean across their item scores) reported that they managed less well with the pressure experienced at school.
3.2.4. Cross Validation
Cross-validation checks compared the accuracy of both exhibited and non-exhibited conditions (e.g., bullying/no bullying). Across the three wellbeing variables, the difference between conditions was small (bullying 83/78, self-harm 82/78), though slightly higher for school pressure (88/77) with both positive and negative predictions showing high accuracy, thus largely eliminating the explanation of false positive/negative readings. Thus, the assessment exhibited power 8 times greater than statistical chance of correctly identifying students who are experiencing bullying, self-harm, or struggling to cope with school pressure.
4. DISCUSSION
We constructed Steer Tracking as a novel means of measuring self-regulation in student populations. This involved a novel four factor bi-polar conceptual model constructed on the basis of a literature review of necessary self-regulatory tasks: Trust of Self, Trust of Others, Self-Disclosure, and Seeking Change. Healthy self-regulation was conceptualised as the task of “steering” each factor appropriately to the situation at hand (i.e., responding flexibly depending on the context), whilst risk was conceptualised as displaying a fixed bias or rigid response pattern (i.e., responding the same way regardless of context). We developed a novel assessment method, utilising the imagination as a means of priming relevant self-regulatory tasks. In this way, two comparative sets of self-regulatory data were obtained for each subject: General and School context-specific data.
Analyses across two studies provide the first steps in demonstrating the psychometric validity of the Steer Tracking assessment. Principal Component Analyses support the proposed four factor conceptual model across samples of children (ages 8-12) and adolescents (ages 13-18) in UK primary and secondary schools. A Support Vector Machine model indicates the assessment is accurate in identifying at-risk students, as shown by the associations between factor scores and three wellbeing indicators: experiencing bullying, thinking about or engaging in self-harm, and struggling to cope with pressures at school. Thus, results support the practical utility of the Steer Tracking assessment as a mental health tool for schools.
4.1. Implications
The Steer Tracking assessment meets a pressing need for evidence-based, accessible tools to help schools identify and provide support to students who are struggling earlier, before formal mental health conditions persist, particularly in the wake of the COVID-19 pandemic. Research demonstrates that the pandemic has negatively impacted self-regulation and contributed to the emergence of anxiety and depression in young people [83]. National evidence confirms that adolescent mental health risks have increased during the pandemic; in 2017, one in nine children was identified as having a probable mental health condition, with estimates rising to one in six by 2021 [84]. One consequence of this increased mental health burden has been increased pressure on the UK Child and Adolescent Mental Health Services (CAMHS) [85].
A UK government 2017 Green Paper recommends early identification and intervention for mental health risks in schools to reduce the cases that reach a clinical level [86]. Funding was subsequently allocated to train and appoint a mental health lead in every school by 2025. Whilst a positive step forward, with 18% of 7 to 16 year olds having a probable mental health disorder, a single mental health lead in secondary schools would be unable to provide individualised support. Steer Tracking has the potential to support the early identification and intervention of young people’s emerging mental health risks. Providing data in a format that a general class teacher can understand can increase the population of teachers ‘with eyes to see beyond the mask’ of students who are struggling to steer the social-emotional road. Additionally, by providing practical guidance to teachers about activities and messages that can be effective in directing a student back onto the social-emotional road, Steer Tracking can broaden the intervention capacity of a school.
4.2. Limitations
This paper presents an initial evaluation of the Steer Tracking assessment model and its cross-sectional associations with relevant well-being outcomes. However, further validation of the assessment’s psychometric properties, including convergent validity, temporal stability, factor divergence and cross-factor loading, is warranted. The percentage variance accounted for by the four principal factors was relatively low. Further investigation is needed to evaluate the assessment’s measurement invariance across age, gender, and other demographic characteristics, which are associated with differences in self-regulation [87, 88].
4.3. Future Directions
Evaluation of the assessment’s properties across different student populations would support its application across cultures and geographies. Longitudinal research is needed to demonstrate the assessment’s predictive utility for identifying at-risk students and tracking socio-emotional development over time. One application of this longitudinal research would be to evaluate the impact of interventions which support the development of self-regulation [89]. Future studies may also investigate the developmental relationships between the distinct components of self-regulation [90] and potential reciprocal relationships between self-regulation and psychosocial outcomes [91].
CONCLUSION
This paper presents the conceptual development and initial validation of a new instrument, Steer Tracking, to assess social-emotional self-regulation in students aged 8 to 18. Whilst it is widely known that the ability to self-regulate underpins academic progress, social competencies and mental health, few tools exist to assess its development across whole-school populations of students. The benefits of such a tool are threefold: for educators, to anticipate student mental health risks earlier, enabling targeted support/referral; for school managers, to obtain data on the impact of their school culture on student social-emotional development; and for students, to empower greater social-emotional self-awareness through personalised data.
LIST OF ABBREVIATIONS
| ERQ | = Emotional Regulation Questionnaire |
| CERQ | = Cognitive Emotion Regulation Questionnaire |
| DERS | = Difficulties in Emotion Regulation Scale |
| CEMS | = Children’s Emotion Management Scale |
| RBF | = Radial Basis Function |
ETHICAL STATEMENT
STEER Education is an independent company and, therefore could not access formal ethical review and approval from an Independent Review Board or Research Ethics Committee. However, all research was carried out in accordance with ethics standards and data protection principles.
Schools signed a data sharing agreement through which the school gave consent as a surrogate for minors (article 16 Declaration of Helsinki). The data was collected pre-GDPR. Schools used their own policies to determine how to communicate participation with parents.
CONSENT FOR PUBLICATION
Schools and students voluntarily participated in the present studies and provided informed consent for their data to be used for research purposes, including publication.
STANDARDS OF REPORTING
COREQ guidelines were followed.
AVAILABILITY OF DATA AND MATERIALS
The datasets generated and analysed during the current study are not publicly available due to stipulations in the consent process that data will not be shared outside of STEER Education.
For any enquiries regarding the STEER Tracking assessment, please contact the corresponding author [S.W].
FUNDING
This research was fully funded by STEER Education, a private limited company co-founded by authors Simon Walker and Jo Walker.
CONFLICT OF INTEREST
The authors declare no conflict of interest, financial or otherwise.
ACKNOWLEDGEMENTS
Declared none.
