Ecological Momentary Assessment with smartphones for measuring mental health problems in adolescents

Abstract

AIM

To analyze the viability of Ecological Momentary Assessment (EMA) for measuring the mental states associated with psychopathological problems in adolescents.

METHODS

In a sample of 110 adolescents, a sociodemographic data survey and an EMA Smartphone application over a one-week period (five times each day), was developed to explore symptom profiles, everyday problems, coping strategies, and the contexts in which the events take place.

RESULTS

The positive response was 68.6%. Over 2250 prompts about mental states were recorded. In 53% of situations the smartphone was answered at home, 25.5% of cases they were with their parents or with peers (20.3%). Associations were found with attention, affective and anxiety problems (P < 0.001) in the participants who took longer to respond to the EMA app. Anxious and depressive states were highly interrelated (rho = 0.51, P < 0.001), as well as oppositional defiant problems and conduct problems (rho = 0.56, P < 0.001). Only in 6.2% of the situations the subjects perceived they had problems, mainly associated with inter-relational aspects with family, peers, boyfriends or girlfriends (31.2%). We also found moderate-high reliability on scales of satisfaction level on the context, on positive emotionality, and on the discomfort index associated with mental health problems.

CONCLUSION

EMA methodology using smartphones is a useful tool for understanding adolescents’ daily dynamics. It achieved moderate-high reliability and accurately identified psychopathological manifestations experienced by community adolescents in their natural context.

Key Words: Ecological Momentary Assessment, Mental health problems, Smartphone, Coping, Adolescents

Core tip: Adolescence is a stage of life characterized by a great many changes. If they are not coped with effectively, these changes may trigger mental health problems. Among the range of methodologies used to assess the impact of daily problems, Ecological Momentary Assessment allows the recording of mental microprocesses and fluctuations as they happen. We found anxious and depressive states were highly interrelated, as well as oppositional defiant problems and conduct problems in daily life. This methodology based on mobile technology using smartphones is a useful tool with high viability for measuring psychopathological mental states in adolescents in their natural context.

INTRODUCTION

Adolescence is a stage of life characterized by a great many changes which, if not addressed effectively, may trigger problems of mental health[1-5]. There is a substantial body of literature assessing the possible impact of individual everyday problems on the development of mental health disorders[6].

One of the formats used to assess the impact of everyday problems is Ecological Momentary Assessment (EMA). This methodology allows the recording of the expression of mental microprocesses and their fluctuations in several situational contexts as they happen[7]. Its applicability has been demonstrated in a variety of populations in studies of general health[8].

Specifically, in adolescent samples, EMA has been used to assess or predict health needs[9], mental status[10] emotional instability[11], drug use[12], stress associated with traumatic events[13], and anxiety problems[14].

In fact EMA has been used for decades[7], but reports of its use in the field of mental health are relatively recent. At present there is no evidence of its use in adolescent community populations to measure broad areas of psychopathological symptoms (sadness, anxiety, somatic, thought, behavioral and attentional problems) in real time in their natural setting, focusing on the coping strategies used and their relation to their situational and personal contexts.

The aim of this study is to explore the viability of EMA for measuring the mental states associated with psychopathological problems in adolescents, taking into account the situational context and the coping strategies they apply.

MATERIALS AND METHODS

Participants

The sample was constituted initially by 110 adolescents from the province of Barcelona, of whom 101 successfully completed the EMA study. Following the recommendations of previous researchers regarding the quality of information[6,15], only subjects who completed at least 12 of the 35 possible recordings (roughly 33%) were considered.

Instruments

Sociodemographic data: A survey sheet was created ad hoc to gather basic data on age, gender, school year, citizenship, family and employment status.

EMA

Each participant received a smartphone with an Android-based EMA application already installed. This application was programmed to give a series of alarms associated with the task of answering mini-questionnaires at five semi-random moments over the course of the day between 9 am and 9 pm for a complete week. The mini-questionnaires comprised 21 items (five with multiple choice answers, two with yes/no answers, 14 to be rated on a 5-point Likert scale) which participants were required to answer within three minutes of hearing the first alarm. If the user did not start to answer within this period of time the application stopped the smartphone alarm and blocked the unit.

Users could not delay their answers beyond five minutes after their last interaction with the application, or take longer than ten minutes to answer the entire mini-questionnaire, in order to ensure that the information was instantaneous and not retrospective. If users were unable to answer within these time limits the application blocked the smartphone and they were told to wait until the next random alarm; this reply attempt was considered empty.

These mini-questionnaires comprised questions regarding situational context and broad areas associated with psychopathological problems such as behavior problems, anxiety, sadness, lack of concentration, and so on. It also covered everyday problems and how to cope with them. Figure 1 shows an example of the questions asked.

Figure 1 Example of questions asked within Ecological Momentary Assessment app in this study.

The application stored the data collected for a week (7 d). Once the evaluation period finished, the data were downloaded by the research team. As a result, a maximum of 35 moments of semi-random evaluation were obtained from each subject. Figure 2 displays the EMA cycle used in this study.

Figure 2 Assessment cycle used in this study with Ecological Momentary Assessment methodology.

Procedure

Several social service centers and schools in the metropolitan Barcelona area and its surroundings were contacted. Two centers agreed to participate after a full explanation of the project and its logistic implications with regard to the distribution and application of EMA methodology with smartphones. After obtaining the centers’ approval, 30-min information sessions were arranged for students from eighth to eleventh grade. In these sessions, students received explanations of their role in the study and the implications of their participation. Informed consent forms were then distributed among the participants for authorization, either by the participants themselves or by their parents and/or tutors. All the procedure and assessment protocols had been previously authorized by the research ethical committee of the University, and complied with the guidelines of the Declaration of Helsinki and legislation regarding confidential data protection.

After obtaining written consent, meetings were arranged for explaining the instruments and the workings of the smartphone devices and EMA app functioning. On receiving the smartphone, participants were assigned an individual alphanumeric code to protect their identity in case of loss or theft of the device. They were informed that they would have the phone for a whole week and that it would ring five times a day at semi-random times, and that they should answer as often as possible. They were also asked as well to sign a commitment to take good care of the smartphone and received an information sheet explaining how the smartphone functioned and how to answer and containing contact data in case of any technical problem during the experiment.

Statistical analysis

The χ² test was used to calculate differences between proportions in frequencies, the t-student test to calculate differences of means between two groups, ANOVA for comparisons between several subgroups with Bonferroni correction, and Spearman correlations to calculate associations between variables.

RESULTS

Socio-demographic data

The sample comprised 101 adolescents (age M = 14.68 ± 1.64; 61% women). Sixty percent were Spanish natives and 40% were foreigners (28% were Latin Americans). Regarding family structure, between 67% and 71% of participants’ parents were married and between 21% and 23% separated or divorced. As for parents’ employment, 77% of fathers were employed and 70% of mothers.

EMA answer rates

In all, 68.6% of questionnaires were completed, while in 31.4% data were missing: Ignored alarms accounted for 23.9% of the missing data (due to a lack of time to answer or not paying attention), rejected answer attempts for 4.5%, and incomplete records not quantified in the analysis for 3.0%. The difference between the types of answer was significant (χ² = 3712.60; P < 0.001). The mean time taken to answer the questionnaires was 80.6 s ± 56.5 s. These data suggest a mean answer time of between one and three minutes.

Contextual variables

Variables about the context (Where, who with, and what were they doing?). In 53% of situations the smartphone was answered at home, followed by 24.3% at school and 15.5% outside in the street. There were significant differences between these three response contexts (χ² = 1072.38; df = 2; P < 0.001). On a 1-5 point Likert scale, in 69% of cases the situational context was assessed as pleasant (a score of 4) or very pleasant (a score of 5). Only in 11.2% of the situations was the context unpleasant or very unpleasant (scores of 1 or 2).

Regarding the people being with the adolescents at the time of assessments, in 25.5% of cases they were with their parents, in 20.3% with peers and in 19.1% alone. There were significant differences between these three groups (χ² = 25.28; df = 2; P < 0.001). The company was regarded as pleasant or very pleasant in 77.9% of situations and unpleasant or very unpleasant in only 7.1%.

The three activities that participants were most frequently engaged in at the time of answering the EMA questionnaires were school tasks or activities (26.7%), talking to somebody (19.7%), and watching TV or the computer (16.8%). There were significant differences between these three activities (χ² = 58.11; df = 2; P < 0.001). The adolescents regarded the activities they were engaged in when they answered the smartphone as pleasant or very pleasant in 64% of cases and as unpleasant or very unpleasant in only 8.9% (χ² = 113.49; df = 3; P < 0.001). The momentary level of satisfaction associated with the context (physical, relational and activity-related) composed by three items, obtained a Cronbach’s alpha of 0.75.

Momentary emotional status and behaviors

In a collection of around 2250 responses obtained from the participants during the week, it was found that in most of situations, these people reported the absence of problems associated with the following symptoms: Affective (79%), anxious (78.2%), somatic (84.3%), inattentive (81%), oppositional (93.7%), aggressive (95%), obsessive-compulsive (82.6%) or traumatic (93.6%). All these situational problems were significantly absent (P < 0.001). Regarding the intensity of problems according to gender, girls presented more affective (t = -9.077; P < 0.001), anxious (t = -4.808; P < 0.001), somatic (t = -6.603; P < 0.001) and post-traumatic symptoms (t = -4.040; P < 0.001) than boys. Boys presented slightly more inattentive-hyperactive problems (t = 2.046; P = 0.041), but there were no significant differences in other disruptive behaviors or obsessive-compulsive problems in daily life. All eight items constituting the general discomfort index associated with momentary mental health problems obtained a Cronbach’s alpha of 0.76.

It was also observed that subjects who presented strong situational problems of inattention, sadness or anxiety, they also took longer to respond (120 s ± 78 s for inattention; 111 s ± 57 s for sadness; and 122 s ± 79 s for anxiety) than subjects who did not present these problems (75 s ± 52 s in absence of inattention, 76 ± 53 s in absence of sadness and 75 s ± 53 s in absence of anxiety). The difference was significant (F = 37.15; df = 4; P < 0.001 for inattention; F = 28.31; df = 4; P < 0.001 for sadness; and F = 31.33; df = 4; P < 0.001 for anxiety). Bonferroni’s post-hoc results showed that significant differences appeared essentially between subjects who did not have these problems and subjects who had them with a certain degree of intensity.

With regard to situations associated with positive emotionality, in particular to feeling happy or loved, subjects reported being happy or very happy (64.2% of cases) and feeling loved or loved very much (66% of cases). There were significant differences in comparison with unhappy situations (χ² = 978.15; df = 4; P < 0.001) or situations in which one does not feel loved (χ² = 1101.58; df = 4; P < 0.001). The intensity of positive emotionality of these two items obtained a Cronbach’s alpha of 0.81 in this study.

Anxious and depressive states were interrelated in the whole sample (rho = 0.51; P < 0.001), and oppositional/defiant problems were associated with conduct problems (rho = 0.56; P < 0.001). Table 1 displays the relationship between problems associated with mental health symptoms by gender. Specifically, girls presented strong relationships between anxiety and affective problems, oppositional/defiant problems and conduct problems, posttraumatic stress problems and disruptive behaviors, while in boys the only strong relationship observed was between oppositional/defiant problems and conduct problems.

Table 1 Mental health symptomatology areas measured by Ecological Momentary Assessment in boys and girls.

Variables	1	2	3	4	5	6	7	8
Affective problems	--	0.38	0.26	0.33	0.15	0.11	0.2	0.32
Anxiety problems	0.57	--	0.23	0.35	0.18	0.14	0.27	0.18
Somatic problems	0.22	0.23	--	0.14	0.12	0.09	0.20	0.10
Hyperactivity-Inattention problems	0.39	0.38	0.18	--	0.22	0.21	0.34	0.28
Oppositional-defiant problems	0.27	0.30	0.19	0.29	--	0.58	0.29	0.31
Conduct Behavior problems	0.27	0.31	0.20	0.27	0.56	--	0.16	0.30
Obsessive compulsive problems	0.25	0.34	0.22	0.31	0.31	0.32	--	0.31
Posttraumatic stress problems	0.34	0.36	0.19	0.34	0.47	0.46	0.39	--

Above the diagonal, boys’ results, below the diagonal, girls’ results. All the results are significant (P < 0.001). In bold-face, the correlations with high-moderate index.

Everyday problems and coping strategies

Only in 6.2% of the situations in which they were asked to answer the mini-questionnaire with the smartphone did the subjects perceive they had a problem. The most common problems, in order of frequency, were those associated with inter-relational aspects with family, peers, boyfriends or girlfriends (31.2%), followed by schoolwork or worrying about exams (29.1%), and arguments or behavior problems (9.9%). Significant differences were found between these three types of everyday problems in adolescents (χ² = 14.80; df = 2; P = 0.001).

The most frequently used kind of coping strategy when a problem appeared was seeking relaxing diversions (21.5%), followed by trying to see the positive side of the situation (20.1%) and then seeking help from peers (16%). The least used strategies were looking for help in the family (6.3%) and seeking spiritual support (1.4%). Significant differences were found in the types of coping (χ² = 38.23; df = 7; P < 0.001).

Regarding their satisfaction with their coping strategies, on more than a third of occasions (35%) adolescents felt neither happy nor unhappy with the strategy chosen to face a specific problem. In 11.2% of cases the subjects were discontent with their choice of strategy, but in 28% they felt very satisfied. There were differences in the appreciation of the application of coping strategies (χ² = 35.92; df = 4; P < 0.001), in that case the majority believed them to be effective.

DISCUSSION

The most relevant result in this study is the finding that in most situations in daily life (between 78% and 93%) adolescents do not present problems that trigger mental health symptoms. When these problems appear, they tend to be closely related to states of sadness or anxiety. Similarly, oppositional defiant behaviors were associated with conduct problems, a finding that corroborates the syndromic daily patterns associated with the two broad areas of symptoms (internalizing/externalizing) regularly found in the field of child and teenager psychopathology[16-18]. It was also observed that the positive response rate of the EMA app (68.6%), and the time taken (about 1-2 min) reflect an accurate measurement of associated context, the broad areas of clinical symptomatology and coping strategies in real time. We also found moderate-high reliability on scales of satisfaction level on the context, on positive emotionality, and on the discomfort index associated with mental health problems. These results show that EMA methodology based on mobile technology offers high viability for measuring mental health states in adolescents.

Among the contextual variables, we stress that during the week-long EMA, the adolescents were regularly at home when answering the smartphone application. This result should be borne in mind, due to the contextual relation of family dynamics in the promotion and development of symptomatology or solutions to everyday problems.

For their part, the results regarding satisfaction in the immediate context show that in most cases adolescents feel satisfied with the surrounding environment (in this case their home). Moreover, in relation to variables of interpersonal contact, it is interesting that the people with whom they have the most contact are their parents, followed by their colleagues or peers: This finding may challenge the concept of adolescence as a stage in which subjects prefer to be with their peers or alone[4,5].

Nevertheless, this does not mean that they prefer to be with their parents, because the adolescents can share a great deal of time together. Perhaps this result reflects the fact that our adolescents regularly answered the application at home (53%), where they are more likely to record direct contact with their parents. However, this shows us that the parents may play a significant role in their children’s development of functional patterns of psychological adaptation via this continuous contact during the adolescent stage, as stated previously by other authors[19]. In adolescent-parent relationships related to parenting styles, it is highlighted the importance of authoritative homes (where parents are both demanding and responsive) with more psychosocial and academic competence[5], that prevent some potential risk situations such as a problematic drug use[3], internalizing/externalizing symptoms[4] or victimization problems[1].

When studying daily patterns it is important to bear in mind that the context and the people with whom we are in contact have a strong effect on the activities we perform. Being in tune with the evolutive stage and the main function to be developed by adolescents at these ages. The adolescents reported that when they were had to answer the questionnaire, in more than a quarter of the cases they were doing school homework, followed by talking and leisure activities (TV-PC). Here the level of pleasure was influenced considerably by their interaction with the place and the people present[4]. Being the homework an activity regularly imposed, they reflect their perception of not being completely satisfied with this activity development, not necessarily meaning extreme dissatisfaction level, in this contextual activity.

In relation to emotional states, momentary symptomatology was broadly absent in the different areas studied (affective, anxious, somatic, inattentive, oppositionist, aggressive behavior, obsessive-compulsive problems, and trauma situations).

In general, the low results for impairment obtained reflect the type of population studied (i.e., a community population). Higher reports of impairment would be likely if the assessment had been applied in a clinical sample[6,20]. The intensity of impairments may also depend on the level of clinical assistance received (outpatient, day hospital or inpatient). These hypotheses should be verified in further studies of clinical samples of adolescents with psychiatric disorders.

The perception of everyday problems and the use of coping strategies is an interesting result. Only rarely did the adolescents record the presence of problems in the assessment, the most frequent being inter-relational and academic performance problems. This reflects the main worries of adolescents at this development stage, characterized by openness to new experiences in the field of social relations[5] and an increase in academic pressure prior to entry to university or to a vocational training center. Similar results has been obtained by other others using traditional methodology[21]. On the other hand, it is important to promote a high parental self-efficacy, which would be highly related to ecological variables and parenting competence, in such a way that environmental conditions and ecological contexts may influence and undermine parent’s confidence and parenting competence[22].

This study has a number of limitations that may affect the interpretation of the results. The first is the sample size. The application of the EMA methodology for seven consecutive days raises a series of logistical issues that complicate the recruitment and maintenance of participants. Compared with the pencil-paper assessment at a specific moment, EMA offers a considerable number of benefits (assessment in real time, decrease in memory bias, contextual association) and drawbacks (time-limited evaluation, sampling, loss of important events and overload) as noted by Shiffman et al[7] 2008, van Os et al[20] 2014, and Stone[23] 2007. All these points should be taken into account when applying EMA. In fact, the sample size in this study is within the range of most other studies in clinical[12-14,24] or community[9-11,25] adolescent populations.

Another issue to be considered is the type of population studied. As our data are from a community sample, they cannot be directly extrapolated to the clinical population or to a specific subgroup with psychiatric disorders. Nevertheless, one should bear in mind that the clinical population is a particular and acute subgroup of the community population. This study may represent a first step in the advancement of knowledge of daily patterns associated with mental health problems in adolescence and the assessment of contextual variables and coping strategies. Third, as this assessment study was carried out over a week, it would be interesting to compare these results with those from wider populations with specific psychiatric disorders, over different time periods, and with a longitudinal design. Despite these limitations, this is, to our knowledge, the first study in adolescents to apply the smartphone-based EMA methodology to measure the triad of contextual variables, symptoms associated with mental health problems and coping strategies.

EMA methodology using smartphones is a useful tool for assessing daily dynamics. It provides a sufficiently accurate measure of the psychopathological manifestations experienced by community adolescents in their natural context. In the study of momentary states associated with mental health symptomatology over a one-week period, we found that in most cases adolescents do not present emotional alterations or problems in their daily life. Girls were slightly more affected in their momentary emotional status and behaviors in daily life than boys. And among the situations in which a conflict is generated - on the one hand, anxious-depressed states, and on the other the oppositional-aggressive behavior are closely inter-related. Our results show that the family and home context could be crucial for the potential development of training interactions, both positive and negative, in the mental health field, and they also stress the importance of individuals’ coping resources in relation with their formative, relational, and physical context.

COMMENTS

Background

Adolescence is a stage of life characterized by a great many changes which, if not addressed effectively, may trigger problems of mental health. There is a substantial body of literature assessing the possible impact of individual everyday problems on the development of mental health disorders.

Research frontiers

Ecological Momentary Assessment (EMA) is a methodology allows to assess the impact of everyday problems in several situational contexts as they happen. Its applicability has been demonstrated in a variety of populations in studies of general health.

Innovations and breakthroughs

This study may represent a first step in the advancement of knowledge of daily patterns associated with mental health problems in adolescence and the assessment of contextual variables and coping strategies. The results show that EMA methodology based on mobile technology offers high viability for measuring mental health states in adolescents.

Applications

EMA methodology using smartphones is a useful tool for assessing daily dynamics. It provides a sufficiently accurate measure of the psychopathological manifestations experienced by community adolescents in their natural context.

Terminology

EMA is a methodology that allows the recording of the expression of mental microprocesses and their fluctuations in several situational contexts as they happen. Compared with the pencil-paper assessment at a specific moment, EMA offers a considerable number of benefits (assessment in real time, decrease in memory bias, contextual association) and also drawbacks (time- limited evaluation, sampling, loss of important events and overload).

Peer-review

The authors have reported their findings based on EMA with Smartphones for measuring mental health problems in adolescents. The study was well taken and the results indicate that such an evaluation is helpful to assess whether using smartphones is a useful tool for assessing daily dynamics or sufficiently accurate measure of the psychopathological manifestations experienced by community adolescents.