Review of key telepsychiatry outcomes

Abstract

AIM: To conduct a review of the telepsychiatry literature.

METHODS: We conducted a systematic search of the literature on telepsychiatry using the search terms, “telepsychiatry”, “telemental health”, “telecare”, “telemedicine”, “e-health”, and “videoconferencing”. To meet criteria for inclusion, studies had to: (1) be published in a peer-reviewed journal after the year 2000; (2) be written in English; (3) use videoconferencing technology for the provision of mental health assessment or treatment services; and (4) use an adequately-powered randomized controlled trial design in the case of treatment outcome studies. Out of 1976 studies identified by searches in PubMed (Medline database), Ovid medline, PsychInfo, Embase, and EBSCO PSYCH, 452 met inclusion criteria. Studies that met all inclusion criteria were organized into one of six categories: (1) satisfaction; (2) reliability; (3) treatment outcomes; (4) implementation outcomes; (5) cost effectiveness; and (6) and legal issues. All disagreements were resolved by reassessing study characteristics and discussion.

RESULTS: Overall, patients and providers are generally satisfied with telepsychiatry services. Providers, however, tend to express more concerns about the potentially adverse of effects of telepsychiatry on therapeutic rapport. Patients are less likely to endorse such concerns about impaired rapport with their provider. Although few studies appropriately employ non-inferiority designs, the evidence taken together suggests that telepsychiatry is comparable to face-to-face services in terms of reliability of clinical assessments and treatment outcomes. When non-inferiority designs were appropriately used, telepsychiatry performed as well as, if not better than face-to-face delivery of mental health services. Studies using both rudimentary and more sophisticated methods for evaluating cost-effectiveness indicate that telepsychiatry is not more expensive than face-to-face delivery of mental health services and that telepsychiatry is actually more cost-effective in the majority of studies reviewed. Notwithstanding legal concerns about loss of confidentiality and limited capacity to respond to psychiatric emergencies, we uncovered no published reports of these adverse events in the use of telepsychiatry.

CONCLUSION: A large evidence base supports telepsychiatry as a delivery method for mental health services. Future studies will inform optimal approaches to implementing and sustaining telepsychiatry services.

Key Words: Telepsychiatry; Telemental health; Videoconferencing; Treatment access; Implementation

Core tip: Telepsychiatry represents a highly promising approach to reducing the treatment gap by making it easier for patients, especially those in isolated contexts, to access expert mental health care. There is a robust evidence base for the use of telepsychiatry as a delivery method for mental health services. Given sufficient empirical justification for telepsychiatry in routine clinical settings, future research studies should focus on clarifying best practices for implementing and sustaining telepsychiatry services.

INTRODUCTION

Innovative approaches to delivering mental health services are urgently needed to increase access to evidence-based care. Telepsychiatry, which in its contemporary use refers to the delivery of mental health services via video-based conferencing, has great potential to address mental health disparities by extending the reach of mental health care to those living in rural areas or to those who otherwise have limited access to care. Rapid changes in technology and the medical landscape have undoubtedly accelerated the growth and reach of telepsychiatry. The enthusiasm for the delivery of mental health services via telepsychiatry is evident in its adoption in large health care organizations such as the United States Department of Veterans Affairs[1] and establishment of national practice guidelines[2].

Prior reviews have focused on different domains of the evidence based for telepsychiatry[3,4]. We sought to build on prior reviews by systematically reviewing and critically summarizing the evidence base for telepsychiatry. For the purposes of this review, telepsychiatry is defined as the provision of mental health services by a mental health professional via videoconferencing technology.

MATERIALS AND METHODS

We conducted a review of the telepsychiatry literature in PubMed (Medline database), Ovid medline, PsychInfo, Embase, and EBSCO PSYCH. We used the keywords, “telepsychiatry”, “telemental health”, “telecare”, “telemedicine”, “e-health”, and “videoconferencing”. To meet criteria for inclusion, studies had to: (1) be published in a peer-reviewed journal after the year 2000; (2) be written in English; (3) use videoconferencing technology for the provision of mental health assessment or treatment services; and (4) use an adequately-powered randomized controlled trial (RCT) design in the case of treatment outcome studies. Additionally, we searched reference lists of included studies to identify additional publications not captured by our literature search. The last search was conducted in June 2015.

The first two authors (Sam Hubley and Sarah B Lynch) reviewed all abstracts to identify eligible studies. Studies that met all inclusion criteria were organized into one of six categories: (1) satisfaction; (2) reliability; (3) treatment outcomes; (4) implementation outcomes; (5) cost effectiveness; and (6) and legal issues. All disagreements were resolved by reassessing study characteristics and discussion. When consensus was not reached between the first two authors, the last author (Jay Shore) made the final decision.

RESULTS

We identified a total of 1668 full-text articles based on our literature search and excluded 1534 based on the inclusion criteria (Figure 1). Of the remaining 134, 86 reported on satisfaction with telepsychiatry, 38 evaluated reliability of clinical assessments conducted via telepsychiatry, 32 were RCTs, 43 reported on implementation outcomes, 29 estimated cost-effectiveness, and 23 evaluated legal issues associated with telepsychiatry mental health services. Note that some studies reported on more than one outcome (n = 117) and were thus included in each relevant section.

Figure 1 Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram.

Satisfaction

Adequate patient and provider satisfaction is a prerequisite for wide scale implementation of telepsychiatry. This is especially true in light of the emphasis on patients’ experience of care as a key component of the Triple Aim Framework[5], by which many healthcare innovations are evaluated. There is a substantial body of literature focusing on patient and provider satisfaction with telepsychiatry. Studies employed a range of descriptive, qualitative, experimental, and mixed-methods designs to assess satisfaction outcomes.

Patient satisfaction: The majority of studies summarize patients’ responses to quantitative self-report questionnaires with descriptive statistics and report high satisfaction with telepsychiatry services. The Client Satisfaction Questionnaire[6] is a commonly used measure in these studies and consists of 8 items rated on a 5-point Likert scale. Many investigators also developed their own satisfaction measures that consist of similar items rated on a Likert-scale. Of the 31 studies reviewed, 23 concluded that patients rated their experience with telepsychiatry services as “good” to “excellent”[7-28], while the remaining 7 studies reported mixed reactions among participants[29-38]. For example, Hilty et al[33] reported generally high satisfaction among outpatients seeking specialty mental health care but satisfaction scores were statistically higher for rural patients compared to suburban patients. Studies that used qualitative (n = 6) qualitative methods to better understand patients’ experiences with telepsychiatry services suggest a less uniform pattern of findings compared to results from studies using quantitative methods only[32,34,37,39,40]. Participants’ responses to individual interviews and focus groups follow themes characterized by both positive and negative reactions to telepsychiatry. Prominent positive themes include ease of use and decreased burden of transportation to and from appointments; whereas prominent negative themes include privacy concerns, challenging to establish patient-doctor relationship, and technical challenges. Finally, some studies (n = 7) used an experimental design to assess patient satisfaction[10,21,26,36,41-43]. In a study that compared reactions of 48 outpatients randomized to telepsychiatry or face-to-face (FTF) psychiatric consultation, telepsychiatry patients reported comfort in disclosing the same information they would disclose in FTF consultation but reported slightly lower levels of satisfaction regarding feeling supported and encouraged than did FTF patients[36].

Provider satisfaction: Based on 11 studies that use qualitative self-report methods, providers tend to have mixed reactions to telepsychiatry[25,33,34,44-52]. Some studies have shown that adult and child psychiatrists reported adequate to high satisfaction with telepsychiatry[53] and one study demonstrated that mental health providers prefer telepsychiatry over telephone-based consultation[51]. Allied providers such as primary care providers (PCPs)[49] and emergency room providers[25,44] have also expressed satisfaction with telepsychiatry. Other studies have yielded mixed results as rural PCPs are more satisfied with telepsychiatry than are PCPs based in suburban locations[33], providers perceive patients to be less satisfied with telepsychiatry services than actual patient report[41] and provider concerns that their lack of experience delivering telepsychiatry may result in lower levels of care[54]. Finally, some investigators have documented negative reactions to telepsychiatry as providers are resistant to use telepsychiatry[55] and concern that perceived technological challenges associated with telepsychiatry may hinder doctor-patient interactions[56].

Qualitative interviews yield similarly mixed results as some allied health providers report satisfaction with telepsychiatry services[44], while others[45,46] express concerns about the potential adverse impacts of telepsychiatry on the therapeutic alliance and cited barriers such as difficulties incorporating telepsychiatry into their practice, including difficulty accessing trainings for telepsychiatry, and lack of resources. Experimental studies have confirmed the finding that providers are not uniformly satisfied with telepsychiatry services, based in large part about concerns over therapeutic rapport. For example, in an RCT comparing cognitive behavioral therapy (CBT) for bulimia nervosa delivered FTF and via telepsychiatry, patients and psychotherapists completed ratings of the therapeutic alliance[57]. Results showed that psychotherapists generally reported lower alliance scores with telepsychiatry participants than FTF participants whereas there no differences between alliance scores among FTF and telepsychiatry participants’ own evaluations of the therapeutic alliance.

Summary of patient and provider satisfaction: The evidence to date on patient and provider satisfaction is generally positive as most studies demonstrate that providers and patients find telepsychiatry acceptable. In general, (1) patients tend to report higher satisfaction than providers, and this appears to be especially true among parents seeking services for their children; (2) patients acknowledge potential changes to the therapeutic alliance inherent in providing services remotely but are less concerned than providers; (3) allied health professionals (e.g., PCPs and ED providers) report high satisfaction with telepsychiatry provided the technology works well and does not majorly interfere with workflows; and (4) study design appears to influence outcome as studies that used purely descriptive methods tend to report more positive outcomes than studies using qualitative and experimental methods.

Reliability

The reliability of assessments conducted via telepsychiatry compared to assessments conducted via the gold-standard of FTF interviews is a key area of research. The primary variables of interest are inter-rater reliability and inter-method reliability. Inter-rater reliability refers to the degree of agreement between two raters assessing the same patient with the same assessment tools, whereas inter-method reliability refers to the degree of agreement between one rater using one assessment tool with two different methods (i.e., telepsychiatry vs FTF). This area of research represents a hybrid between inter-rater and inter-method reliability.

Several statistical approaches are used to quantify the degree of agreement between two different ratings. Most relevant to evaluating reliability in telepsychiatry assessments are the kappa statistic, correlation coefficients, and the intra-class correlation coefficient (ICC). The kappa statistic assesses agreement on categorical outcomes (i.e., whether or not a patient receives a diagnosis of bipolar disorder) and Pearson’s r is a correlation coefficient that assesses agreement on continuous outcomes such as total scores on the Hamilton Depression Rating Scale (HAMD)[58]. Finally, the ICCs is a more sophisticated version of Pearson’s r because it accounts for variance in assessments due to true between-subject variability in addition to variance due to disagreement between the raters.

We identified 21 studies that evaluated reliability of clinical assessments conducted via telepsychiatry. Two studies compared the accuracy of diagnoses made using the SCID[59,60], two studies evaluated reliability of child assessments[42,61], four studies evaluated reliability of neuropsychological assessments[62-65], and three studies evaluated reliability of measures of depressive symptom severity[66-68]. The remaining studies evaluated a range of targets such as alcohol use severity[69], diagnostic accuracy[70-72], competency to stand trial[73], psychosis[26], and adult autism[21]. The majority of studies report moderate to high level of agreement between raters using telepsychiatry and FTF regardless of instrument, provider, or setting type.

Several authors used clever study designs to evaluate reliability at a more nuanced level. For example, two studies assessed reliability estimates between telepsychiatry conducted with varying levels of bandwidth to determine if connection quality impacted reliability. One report observed ICCs greater than 0.95 across all conditions of varying bandwidth quality among two psychiatrists rating eight psychiatric outpatients and nine health controls using the BPRS[74]. On the other hand, Yoshino et al[75] observed statistically significantly lower ICC scores in the narrow bandwidth condition compared to the broad bandwidth in for BPRS scores. In a rigorous comparison of inter-rater reliability that reduced artificial inflation of reliability estimates by requiring interviewers to generate their own clarifying questions and probes, Kobak et al[66-68] observed very high reliability estimates between four psychiatrists at three different locations using the HAMD via telepsychiatry or FTF[67]. And although diagnostic assessments via telepsychiatry appear to be reliable even when using interpreters[60], there are mixed results of the reliability of telepsychiatry assessments when visual information is required. For example, Jones et al[76] found that raters assessing a geriatric population on a psychiatric inpatient unit achieved lower reliability on the observational items based on participants’ behavior than the subjective items based on participants’ self-report. This difference in Kappa scores was statistically significant for assessments conducted via telepsychiatry, but not for assessments conducted FTF[76]. Conversely, Amarendran et al[62] found that assessment of abnormal movements among patients with at least 10 years of antipsychotic medication exposure was not significantly less reliable when using telepsychiatry compared to FTF.

Summary of reliability estimates in telepsychiatry: The studies included in this review suggest that, in general, assessments made via telepsychiatry are comparable to FTF assessments in terms of reliability. No studies provided strong evidence that telepsychiatry assessments are significantly less reliable than FTF assessments. Studies that have not reported uniformly high reliability estimates for telepsychiatry assessments represent important caveats. First, adequate reliability may be contingent on bandwidth quality as the quality of observations required for a given assessment decreases as video and audio quality deteriorate. Second, the use of interpreters does not appear to reduce the reliability of telepsychiatry assessments but more studies that attempt to replicate this finding are needed. Third, assessments that require objective observations (i.e., the Brief Psychiatric Rating Scale) may be more difficult to conduct via telepsychiatry.

Treatment outcomes

The crucial question in telepsychiatry research is whether or not mental health interventions delivered via telepsychiatry can achieve similar outcomes compared to interventions delivered FTF. We identified 13 RCTs that evaluated treatment outcomes for mental health interventions delivered via telepsychiatry (Table 1). Seven studies targeted depression[77-83], two targeted symptoms of PTSD[11,84,85], two targeted ADHD[86], one targeted bulimia nervosa[87], and two targeted common psychiatric disorders presented in an outpatient medical and mental health settings.

Table 1 Review of telepsychiatry randomized controlled trials.

Ref.	Participants				Interventions			Results
	n	Recruitment source	Mean age (SD)	Target disorder	Conditions	Duration (# of visits)	Provider	Attrition	Findings
Nelson et al[82]	38	Urban schools	10.3 (2.0)	Childhood depression	CBT TP	8	dnr	26%	Child Depression Inventory scores reduced from 14.36 (SD = 9.85) at baseline to 6.71 (SD = 4.78) at post-treatment for CBT TP and from 13.57 (SD = 8.75) to 11.64 (SD = 4.78) for CBT FTF [Wilks’ L (1, 26) = 0.83; Eta2 = 0.17]
					CBT FTF	8	dnr	26%
Ruskin et al[83]	119	VA outpatient mental health clinics	49.6 (12.8)	Depression	Pharmacotherapy TP	8	Psychiatrist	27%	Mean scores not reported. Differences between response rates according to the Hamilton Rating Scale for Depression for TP (49%) and FTF (43%) were not statistically significant (χ2 = 0.4, P > 0.05)
					Pharmacotherapy FTF	8	Psychiatrist	30%
Fortney et al[79]	395	VA community-based outpatient clinics	59.2 (12.2)	Depression	Stepped collaborative care TP	Flexible number of visits up to 12 mo	On-site PCP + Off-site psychiatrist, care manager, PharmD	10%	At 12 mo, TP participants had greater odds of qualifying for remission than usual care participants (OR = 2.4, P = 0.04) but were not more likely to qualify for treatment response (OR = 1.4, P = 0.18) using the Hopkins Symptom Checklist
					Usual care in primary care setting	Flexible number of visits up to 12 mo	PCP	9%
Hilty et al[80]	94	Rural primary care clinics	46	Depression	Psychiatric consultation TP, PCP training, disease management modules	5 with psychiatrist 5 with PCP	Psychiatrist, PCP	dnr	Mean scores not reported. Differences between response rates according to the Beck Depression Inventory-13 for TP (42%) and augmented usual care (42%) were equivalent and not analyzed with odds ratios. Similarly, response rates according to the Hopkins Symptom Checklist-90 for TP (53%) and augmented usual care (42%) were not analyzed with odds ratios
					Disease management modules, usual care in primary care setting	5 with PCP	PCP	dnr
Chong et al[77]	167	Community health center	43.0 (12.0)	Depression	Pharmacotherapy via TP + integrated primary care	7 with psychiatrist, no limit on other visits	Psychiatrist, PCP, mental health specialist	13.8%	Patient Health Questionnaire-9 scores reduced from 17.3 (SD = 4.9) at baseline to 6.8 (SD = 6.0) at post-treatment for TP and from 18.3 (SD = 4.5) to 4.7 (SD = 5.1) for FTF (F =1.1, P > 0.05. Eta2 = 0.17)
					Integrated primary care	No limit	PCP, mental health specialist	10.3%
Moreno et al[81]	167	Community health center	43.2 (11.9)	Depression	Pharmacotherapy via TP + integrated primary care	7 with psychiatrist, no limit on other visits	Psychiatrist, PCP, mental health specialist	dnr	Patient Health Questionnaire-9 scores reduced from 17.6 (SD = 7.6) at baseline to 5.1 (SD = 6.8) at post-treatment for TP and from 18.4 (SD = 4.9) to 4.5 (SD = 5.3) for FTF (t =2.30, P < 0.05. Eta2 =0 .11)
					Integrated primary care	No limit	PCP, mental health specialist	dnr
Fortney et al[79]	364	Federally qualified health centers	47.2 (12.6)	Depression	Enhanced collaborative care TP	Flexible number of visits in 12 mo	On-site PCP + Off-site psychiatrist, care manager, behavioral health, PharmD	23%	At 12 mo, TP participants had greater odds of qualifying for remission than usual care participants (25.8% vs 9.9%; OR = 3.2, P < 0.001) and were more likely to qualify for treatment response (47.7% vs 21.9%; OR=3.3, P < 0.001) using the Hopkins Symptom Checklist-20
					Collaborative care in primary care setting	Flexible number of visits in 12 mo	PCP, care manager	19%
Fortney et al[11]	265	VA community-based outpatient clinics	52.2 (13.8)	PTSD	Enhanced collaborative care TP	Flexible number of visits in 12 mo	On-site PCP + Off-site psychiatrist, care manager, psychologist, PharmD	16%	At 12 mo, Posttraumatic Diagnostic Scale scores decreased 4.17 (SD = 9.8) for TP and 1.32 (SD = 8.8) for FTF (t = 2.30, P < 0.05. Cohen’s d = 0.31)
					Collaborative care in primary care setting	Flexible number of visits in 12 mo	PCP, care manager, social worker	11%
Morland et al[84]	125	VA clinical sites and VA Vet Centers	54.7 (9.6)	PTSD	Group CBT TP	12	Clinical psychologist	10%	In a non-inferiority trial, State-Trait Anger Expression scores reduced from 56.7 (SD = 12.0) to 46.6 (SD = 12.2) in TP and from 55.0 (SD = 10.3) at baseline to 46.6 (SD = 12.2) at post-treatment for FTF. Using CIs and a priori cut-offs, criteria for non-inferiority met (Cohen d = 0.44 in favor of CBT TP)
					Group CBT FTF	12	Clinical psychologist	11%
Morland et al[85]	125	VA clinical sites and VA Vet Centers	55.3 (12.5)	PTSD	CPT-C TP	12	Clinical psychologist or master’s level social worker	18%	In a non-inferiority trial, Clinician-Administered PTSD Scale scores reduced from 72.0 (SD = 14.6) to 55.6 (SD = 18.8) in CPT-C TP and from 68.9 (SD = 13.0) at baseline to 58.7 (SD = 21.0) at post-treatment for CPT-C FTF. Using CIs and a priori cut-offs, criteria for non-inferiority met (Cohen d = 0 .27 in favor of CBT TP)
					CPT-C FTF	12	Clinical psychologist or master’s level social worker	14%
Myers et al[86]	233	Primary care	9.2 (2)	ADHD	Pharmacotherapy via TP + caregiver training	6	Psychiatrist, master’s level therapist	13%	At 12 mo, TP participants had greater odds of no longer meeting diagnostic criteria for ADHD-inattentive subtype according to Vanderbilt ADHD Rating Scale at post-treatment (12% vs 26%; OR = 0.149, P < 0.001)
					Psychiatric consultation with PCP + caregiver training	1	Psychiatrist, PCP	5%
Mitchell et al[87]	128	Patient panels of rural physicians and therapists	29.0 (10.7)	Bulimia nervosa	CBT TP	20	Clinical psychologist	34%	At post-treatment, abstinence from binge-eating episodes, purging episodes, and combined episodes ranged from 27%-50% for TP CBT and 29%-50% for FTF CBT with non-significant trend in favor of FTF. TP participants reported significantly more binge episodes (M = 6.2, SD = 12.3) than FTF participants (M = 3.7, SD = 11.2) at post-treatment (F = 6.76; P < 0.05)
					CBT FTF	20	Clinical psychologist	41%
De Las Cuevas et al[88]	140	Community mental health center	Adults	Psychiatric disorders	Pharmacotherapy, CBT TP	8	Psychiatrist	6%	Differences between improvement rates according to the Clinical Global Impressions scale for TP (67.2%) and FTF (62.5%) were not statistically significant (P > 0.05)
					Pharmacotherapy, CBT FTF	8	Psychiatrist	7%
O’Reilly et al[89]	495	Rural hospital and primary care clinics	Adults	Psychiatric disorders	Psychiatric consultation TP	Flexible number of visits in 4 mo	Psychiatrist	7%	In a non-inferiority trial, 22% of TP participants and 20% of FTF participants returned to functional status at post-treatment according to the Brief Symptom Inventory. Using CIs and a priori cut-offs, criteria for non-inferiority met
					Psychiatric consultation FTF	Flexible number of visits in 4 mo	Psychiatrist	3%

ADHD: Attention deficit hyperactivity disorder; CBT FTF: Cognitive behavioral therapy delivered face to face; CBT TP: Cognitive behavioral therapy delivered via telepsychiatry; CPT-C: Cognitive processing therapy-cognitive version only; dnr: Did not report; PCP: Primary care provider; PTSD: Post-traumatic stress disorder.

Telepsychiatry compared to usual care: RCTs are required to determine if telepsychiatry interventions are comparable to FTF interventions in terms of outperforming usual care. Of the seven studies that made this comparison, six studies compared psychotropic medication management via telepsychiatry to FTF delivery of usual care[78-81,83,88]. One study included delivery of psychotherapy via telepsychiatry[11]. Four studies demonstrated superiority of telepsychiatry over FTF usual care[11,78,79,86], while telepsychiatry failed to achieve superior outcomes in three studies[77,80,81]. Fortney et al[11] revealed that although cognitive processing therapy (CPT) for PTSD was available in both the collaborative care delivered via telepsychiatry and FTF, participants randomized to telepsychiatry were 18 times more likely to initiate CPT and suggested that long travel distances discouraged weekly psychotherapy.

Telepsychiatry compared to FTF: A second and distinct question is whether or not mental health services delivered via telepsychiatry generate outcomes that are equivalent to FTF services. Of the seven studies making this comparison[82-85,88-90], only Mitchell et al[87] - who evaluated CBT for Bulimia Nervosa - found FTF to be superior to telepsychiatry. It is important to note that this finding held despite a high attrition rate of over 33% in both arms that reduced statistical power. The remaining six studies[81-83,88,90,91] reported that telepsychiatry was equivalent to FTF but only three conducted a non-inferiority trial that was explicitly designed to determine equivalency between telepsychiatry and FTF[84,85,89].

Summary: Overall, mental health interventions delivered via telepsychiatry and FTF resulted in similar treatment outcomes. To summarize: (1) telepsychiatry appears to be better than usual care, except possibly in the case of depression treatment in primary care where telepsychiatry has failed to show superior treatment outcomes to usual care in multiple studies; (2) There were no differences in the patterns of findings for the delivery of pharmacotherapy or psychotherapy delivered via telepsychiatry; (3) With the exception of one study, current data suggest telepsychiatry interventions produce outcomes that are statistically equivalent to outcomes produced by FTF interventions; and (4) The treatment outcome data on telepsychiatry is strongest for the treatment of depression, whereas the compelling data for psychiatric disorders other than depression is strongest in publications that focus on program descriptions and retrospective single cohort designs.

Implementation

There are two primary approaches to developing frameworks that guide the implementation of telepsychiatry. First, “purist” approaches use a comprehensive review of theory, basic science, and expert consensus to develop theoretically- and empirically-derived implementation frameworks (CFIR, RE-AIM, PRISM). Second, “pragmatic” approaches emerge from a practical need to organize implementation efforts for a given type of intervention or intervention delivery method for a defined population (i.e., pharmacotherapy via telepsychiatry for primary care patients with mental illness). For the scope of this review, pragmatic approaches such as the “Lexicon of Assessment and Outcome Measures for Telemental Health” are most appropriate[92]. This lexicon, derived by consensus from 26 established telepsychiatry experts, contains 36 implementation variables to consider. We review access, utilization, and the impact on clinical skill and workflows as particularly relevant variables in the implementation of telepsychiatry services.

Studies from the implementation of telepsychiatry in the VA provide data documenting the power of telepsychiatry to reach great numbers of people. Since 2003, the VA has documented nearly 500000 telepsychiatry encounters[93]. In an assessment of telepsychiatry services for over 100000 patients between 2006 and 2010, researchers found that hospitalization utilization decreased by approximately 25%[94]. However, patients in other settings do not always use telepsychiatry services, even when they are freely available. For example, in the Fortney et al[79] trial reviewed above in which free psychotherapy was available via telepsyschiatry, Deen et al[30] demonstrated 76% of patients reported that psychotherapy was acceptable, 38% scheduled a telepsychiatry psychotherapy session, 17% attended a session and 8% attended at least 8 session. Out of a range of possible patient sociedemographic and clinical factors, perceptions depression would remit on its own and low treatment efficacy predicted treatment utilization. On the other hand, there is some evidence that telepsychiatry may be more efficient that FTF. In a clinic that compared utilization data for 7523 telepsychiatry appointments and 115148 FTF appointments, patients kept more telepsychiatry appointments than FTF appointments (92% telepsychiatry vs 87% FTF). Also, patients were less likely to cancel telepsychiatry appointments (3.5% vs 4.8%) and were significantly less likely to be no-shows (4.2% vs 7.8%)[95]. Finally, it appears that telepsychiatry delivered in primary care does not increase PCP or mental health provider burden[96]. Hilty et al[97] demonstrated successful uptake of skills among PCPs treating anxiety and depression following consultations with telepsychiatrists. In this study, PCPs’ ability to appropriately dose medications for depression and anxiety improved from 47% to 64% (P < 0.001).

Cost-effectiveness

The cost of telepsychiatry is widely debated and discussed. Several methods to estimate the cost of telepsychiatry compared to FTF mental health services in traditional clinic settings as well as specialty environments (e.g., emergency departments) exist in the literature. One of the simplest assessments of cost data compares the collection of direct and indirect costs associated with patient encounters in telepsychiatry and FTF psychiatry visits. Studies that assessed the cost effectiveness of telepsychiatry have evaluated the direct costs of provider time[98], medical supplies[96], technology[53], and reimbursement[99]. Measurement of indirect costs include resources that facilitate patient encounters, such as clinic space[9], administrative support[96], and transportation[53,100-103]. Other studies compare healthcare utilization, such as visits to the emergency department[101] or primary care encounters[96] as a proxy measure of cost associated with telepsychiatry of FTF treatment. Return on investment (ROI) is another approach to calculating cost comparisons between telepsychiatry and FTF consolations. Simply stated, ROI is a cost-benefit ratio that assesses the cost of the service relative to an outcome measure, such as quality adjusted life years (e.g., QALY) or disability adjusted life years (e.g., DALY). Typically, QALY and DALY are measured by taking the difference between number of days symptom free days and days with clinical symptoms over a set period (12-18 mo). That difference is then divided by 365 d to create a range of time spent fully symptomatic to symptom-free.

Assessment of direct and indirect costs: Several studies (n = 18) compared the direct or indirect costs associated with providing telepsychiatry services. The majority of these studies (n = 13) found that telepsychiatry was associated with less direct and indirect costs than FTF services. A handful of studies utilized expenses associated with patient travel time as a cost outcome measure and found telepsychiatry reduced costs associated with travel compared to FTF[53,99,102,104-107]. The literature reviewed suggests that telepsychiatry may have greater up-front costs associated with service delivery when compared to FTF; however, there appears to be “tipping points” at which telepsychiatry begins to eclipse the cost-effectiveness of FTF interventions. Several studies identified the number of consultations at which telepsychiatry became more cost effective than FTF, with the number of consultations delivered ranging from 131[99] to 249[98,100] to 379[103]. For example, Butler et al[98] found that the cost savings of telepsychiatry occur after the health center or provider delivers 249 consultations. In more rural settings, the costs saving effects of telepsychiatry for the provider and patient could be established in as little as 6 consultation sessions[102]. Of note, two studies found that telepsychiatry was more expensive than FTF[108,109]. Pyne et al[109] hypothesized that telepsychiatry may prompt patients to seek additional specialty care because the telepsychiatry intervention delivered promotes a more integrated approach to care than traditional FTF delivery. Modai et al[108] suggested that telepsychiatry leaded to more hospitalizations than FTF; however, it should be noted that Modai et al[108] study was limited by a small sample size (n = 49). When excluding hospitalization from the analysis, Modai et al[108] found telepsychiatry was associated with decreased travel costs when compared to FTF.

Return on investment: A smaller number of studies (n = 4) utilized ROI methodology to determine cost effectiveness of telepsychiatry compared to FTF. Three of these studies utilized the QALY as an outcome measure[109-111] and the remaining study utilized the DALY as an outcome measure[112]. Results using these more sophisticated methods to evaluate cost-effectiveness found telepsychiatry to be more cost effective than FTF in underserved primary care populations[111], management of pain and depression in cancer populations[110], and in depressed populations[112]. However, Pyne et al[109] found that telepsychiatry was not cost effective compared to FTF in treating depression in rural primary care settings; while telepsychiatry was effective in treating depression in rural populations, there were greater costs associated with the delivery of services and greater utilization of outpatient services among patients receiving telepsychiatry compared to FTF consultations.

Summary: The evidence to date on the cost effectiveness of telepsychiatry is generally positive as most studies demonstrate that telepsychiatry reduces direct and indirect costs and increases quality of life adjusted years when compared to FTF. For example, in one study the authors estimated that utilizing telepsychiatry to deliver 16 sessions of CBT for over 20 wk could save a clinic approximately $2025 per patient compared to FTF services[104]. In general, (1) telepsychiatry reduces costs associated with patient travel; (2) there are likely more upfront costs associated with telepsychiatry but the costs are recovered after 6-379 sessions depending in the population being served; (3) cost effectiveness of telepsychiatry may differ depending on setting (e.g., rural or urban). Significant differences emerged specific to operationalizing cost effectiveness and the method of assessing cost effectiveness. It appears that studies that utilized a simple approach to cost effectiveness by comparing direct and indirect costs had similar cost saving findings, whereas the handful of studies that utilized a more sophisticated approach suggest telepsychiatry is more cost-effective than FTF.

Legal issues

Despite the promise of telepsychiatry to reduce barriers associated with accessing mental health services and increase access to services, legal issues specific to telepsychiatry remain an extant and significant barrier. We identified two primary themes concerning legal issues associated with providing mental health services via telepsychiatry - licensing regulations and risks to patient confidentiality.

Several publications highlighted the legal challenges that telepsychiatry presents concerning provider licensure. Telepsychiatry increases providers’ catchment area such that patients living outside of the state where a provider is licensed to practice may request their services via telepsychiatry. However, each state has its own licensing boards that establish practice jurisdictions for providers licensed in the state, and some have specific regulations related to telepsychiatry[113]. As the field progresses, the topic of licensure jurisdictions within the United States will continue to be discussed on the National level[114]. Until a resolution is reached, telepsychiatry providers must have separate licenses for each state in which they provide services and are advised to be familiar with state-specific regulations. For example, Shore et al[115] suggests that a priori arrangements should be made with local law enforcement and social services that are responsible for initiating involuntary commitments.

Patient confidentiality is also a debated topic specific to telepsychiatry. Baker et al[116] outlined several considerations for patient confidentiality when utilizing telepsychiatry: Verification of the patient’s identity, establishing and ensuring privacy on both the provider and patient’s location, disruption of technology, involving others in treatment, and storage of information collected or recorded during the telepsychiatry consultation. Several studies encourage telepsychiatry providers to go above and beyond normal methods of informing patients about limits of confidentiality such that they are aware of possible privacy breaches that are less likely to occur in traditional FTF consultations[116-119].

DISCUSSION

There is a large evidence base for the use of telepsychiatry as a delivery method for mental health services. Contemporary healthcare innovations such as telepsychiatry are commonly evaluated according to the Triple Aim Framework which addresses patient satisfaction, care quality, and cost effectiveness[5]. We reviewed 569 studies that focus on Triple Aim domains and the evidence indicates that patients are satisfied, telepsychiatry is comparable to FTF delivery of mental health interventions, and telepsychiatry can be a cost-effective approach to increasing access to mental health care.

A socioeconomically and clinically diverse patient and provider population has reported on their experience with telepsychiatry. Their responses to self-report questionnaires, qualitative interviews, and mixed-methods assessments suggest that they are comfortable using this technology, appreciate the practical benefit of avoiding travel, and are less concerned than providers about potentially adverse impacts of telepsychiatry on the therapeutic alliance. In terms of care quality, the evidence reviewed suggest that telepsychiatry is comparable to FTF in the reliability of assessment and effective treatment of a range of behavioral and mental health disorders. Importantly, a small number of high quality studies have used non-inferiority designs to demonstrate statistically equivalence in treatment outcomes between telepsychiatry and FTF.

The remaining literature we reviewed focus on practical factors related to the implementation of telepsychiatry such as adaptability of telepsychiatry to routine care settings, cost-effectiveness, and legal issues. Several program descriptions discuss the actual usage of telepsychiatry services in routine clinical settings. Although it is not a foregone conclusion that patients will use telepsychiatry services, even when they are freely available, telepsychiatry is comparable to FTF in terms of service utilization patterns and can help allied health providers develop clinical skill in treating mental illnesses. Using both face-valid and sophisticated approaches to evaluating cost-effectiveness, telepsychiatry can be as cost-effective as FTF services and more studies are needed to determine how cost-effectiveness is affected by rurality, patient sociodemographic and clinical characteristics, provider type, and organizational characteristics. Notwithstanding legal concerns about loss of confidentiality and limited capacity to respond to psychiatric emergencies, we uncovered no published reports of these adverse events in the use of telepsychiatry.

Limitations and future directions

Despite many strengths evident in the evidence base for telepsychiatry, there are important limitations to consider. First, it is important to acknowledge the limitations inherent in assessing consumer attitudes with healthcare services. As reported over a decade earlier[119] there are several factors that continue to limit the quality and generalizability of research on patient satisfaction with telepsychiatry services such as: (1) over-reliance on self-report methodologies; (2) selection biases that over-represent patients amenable to telepsychiatry; (3) insufficient sample sizes; and (4) omission direct comparison of preferences for telepsychiatry vs FTF. Quantitative designs that rely solely on participant self-report are sufficient for demonstrating minimum standards of acceptability, but to obtain a more nuanced understanding of reactions to telepsychiatry, mixed-methods and experimental designs are strongly recommended. In light of the consistent finding that patients are satisfied with telepsychiatry services, we recommend that future studies focus less on assessing satisfaction, and more on clarifying the actual effects of telepsychiatry on therapeutic rapport. For instance, comparing ratings of therapeutic rapport by blind assessors listening to audio recordings of telepsychiatry and FTF mental health services would provide a more objective test of the notion that telepsychiatry impairs rapport. A more thorough understanding of reactions to telepsychiatry, especially as they pertain to provider skepticism based on the assumption that telepsychiatry impairs rapport and intervention quality, has the potential to increase acceptability and uptake of telepsychiatry services.

Second, although the studies reviewed suggest that telepsychiatry is a reliable method for assessment of mental health constructs, it is inappropriate to interpret a null hypothesis without using a non-inferiority design. In this case it is particularly problematic because there is a consistent pattern of higher kappa statistics for FTF assessments than for telepsychiatry assessments, and some authors interpret a lack of statistically significant differences between assessment modality as equivalence in their reliability. To claim that assessments conducted via telepsychiatry are as reliable as assessments conducted FTF, researchers must use study designs that either explicitly test for equivalency, or are adequately powered to detect clinically significant differences between the reliability of telepsychiatry and FTF assessments. We recommend that future studies shift focus from establishing equivalent reliability between telepsychiatry and FTF assessments to identifying which types of assessments are most amenable to the telepsychiatry modality, which types of assessments are most difficult to administer via telepsychiatry, and which types of adaptations help improve the accuracy, efficiency, and consumer experience of telepsychiatry assessments.

Third, we did not perform an intensive quality appraisal of the treatment outcome studies reviewed. Even rigorously designed and executed RCTs are subject to several sources of biases such as selection bias, performance bias, attrition bias, and reporting bias. Furthermore, several studies claim equivalency between telepsychiatry and FTF with using non-inferiority designs. However, we do not recommend more equivalency studies; rather, it is clear that telepsychiatry is comparable to FTF and the field now needs more research focusing on factors that increase telepsychiatry adoption, moderators to determine for which patients in which settings telepsychiatry is most effective, and strategies for integrating telepsychiatry services within the broach context of healthcare service delivery.

Finally, assessing costs associated with the delivery of telepsychiatry is an important, yet often overlooked, factor when evaluating telepsychiatry outcomes. There is room for improvement in cost evaluations of telepsychiatry in order to generate high-quality generalizable findings. Only four of the studies reviewed used sophisticated methods of cost effectiveness. Additionally, cost effectiveness has only been evaluated in a narrow range of patient populations. Future research would benefit from using methodologies that incorporate DALYs or QALYs to examine cost-effectiveness among diverse psychiatric populations, as there is evidence to suggest that telepsychiatry may be more economical than FTF with patients with specific demographic characteristics (e.g., patients seeking services in primary care vs specialty mental health, rural vs urban patients).

The gap between the need for mental health treatment and availability of mental health providers has prompted national and international organizations such as the National Institute of Mental Health (NIMH; 2013) and the World Health Organization (WHO; 2013) to prioritize the development and evaluation of novel service delivery methods. Telepsychiatry represents a highly promising approach to reducing the treatment gap by making it easier for patients, especially those in isolated contexts, to access expert mental health care. Just as the NIMH and WHO have shifted emphasis from efficacy to effectiveness testing, it also time for telepsychiatry researchers to focus less on pure outcome studies to document patient acceptability and high care quality, and more on studies that inform evidence-based approaches to implementing and sustaining telepsychiatry services.

COMMENTS

Background

Innovative approaches to delivering mental health services are urgently needed to increase access to evidence-based care. Telepsychiatry, which in its contemporary use refers to the delivery of mental health services via video-based conferencing, has great potential to address mental health disparities by extending the reach of mental health care to those living in rural areas or to those who otherwise have limited access to care.

Research frontiers

For many years, an important question for telepsychiatry researchers was, “Is telepsychiatry a via ble delivery method for mental health services relative to face-to-face delivery”? As experts in mental health service delivery have shifted emphasis from efficacy to effectiveness testing, so too is it time for telepsychiatry researchers to focus less on pure outcome studies to document patient acceptability and high care quality, and more on studies that inform evidence-based approaches to implementing and sustaining telepsychiatry services.

Innovations and breakthroughs

Telepsychiatry represents a highly promising approach to reducing the treatment gap by making it easier for patients, especially those in isolated contexts, to access expert mental health care.

Applications

This review suggests that telepsychiatry is a via ble and cost-effective method to increase access to mental health services.

Terminology

Telepsychiatry, in its contemporary use, refers to the delivery of mental health services via video-based conferencing.

Peer-review

In this review, the authors have provided a comprehensive commentary on the state of the research for key outcomes in telepsychiatry.