Page 7 Effect of telehealth on quality of life and psychological out comes over 12 months
P. 7
BMJ 2013;346:f653 doi: 10.1136/bmj.f653 (Published 26 February 2013) Page 7 of 20
RESEARCH
their interaction for each of the outcome measures (table 2⇓). these findings were not robust across sensitivity analyses, and
Tests of the effects of the covariates are not presented. Table 2 point estimates of effect size for these outcomes did not reach
shows that trial arm, time, and their interaction were not clinical significance at either assessment point (figs 8 and 9;
significant for any outcome measure in either cohort. table 3).
To assist with the interpretation of table 2, web table 3 presents The overall consistency of results demonstrates that the findings
unadjusted means at baseline and estimated marginal means are robust to variations in attrition (complete case v available
(EMMs) at short term and long term for all outcomes. EMMs case analyses), protocol fidelity (intention to treat v per protocol
were derived from a model that accounted for the intracluster analyses), and choice of outcome measure. The similarity of
correlation, all continuous or ordinal covariates, and baseline the patient reported outcomes across trial arms suggests that
outcome measure; but not categorical covariates (known as concerns about the potentially deleterious effect of telehealth 8 50
“factors” within SPSS). For both cohorts, the pattern of means are unfounded for most patients, since deterioration on any of
and EMMs closely mirror the parameter estimates in table 2 the five outcome measures over the assessment period was not
and reaffirm that differences between trial arms are clinically significant, compared with usual care. For the purposes of
insubstantial. Minor differences of interpretation for some service planning, the current findings should be considered with
outcomes between table 2 and web table 3 are explained by the other evidence from the WSD Evaluation on the effect of
81
underlying differences in the statistical models used to generate telehealth on hospital use and mortality and cost effectiveness. 82
the values reported.
Figures 6⇓ and 7⇓ show the adjusted effect sizes for trial arm Comparison with other studies
(telehealth v usual care). Outcomes for both cohorts at short When comparing our findings to existing research, it is important
term and long term failed to reach the trial defined MCID. to distinguish between the statistical results reported in the extant
Further, all confidence intervals crossed zero, suggesting that literature and the conclusions drawn by authors. When
estimates of the true treatment effect in the population could considering only studies evaluating broadly equivalent forms
favour either telehealth or usual care. The true direction of the of telehealth (that is, home based, vital signs monitoring using
effect is uncertain and the magnitude of the effect is clinically store and forward technology), systematic reviews have shown
insignificant. that fewer than half found any significant benefits to health
related QoL, 34 36 42 44 and those that did only found effects on
Secondary analyses: treatment efficacy only a minority of the QoL measures used. 49
In per protocol analyses, multilevel modelling generated no Despite methodological variation across studies, these findings
significant main effects for trial arm or time for any outcome suggest that the effect of telehealth on health related QoL is
measure in either cohort (table 3⇓). The two significant weak or non-existent. To this extent, the available literature
interaction terms for the SF-12 mental component score and concurs with the current findings. However, some authors have
CESD-10 in the complete case cohort reflect deteriorations for observed that the conclusions drawn in many telehealth studies
telehealth at short term, whereas scores remained stable over are often unduly positive. 42 53 With some notable exceptions, 42
time for usual care (web table 4). The interaction findings were the current study’s conclusions therefore differ markedly from
not replicated in the available case cohort (table 3). Differences most extant studies and reviews examining the effect on
between trial arms were unlikely to be clinically significant telehealth on health related QoL, which are typically interpreted
(figs 8⇓ and 9⇓). as showing benefits despite presenting equivocal evidence.
No adverse events or side effects related to any of the telehealth The scope for inappropriate inferences is increased when small
devices were reported in the intervention group throughout the and methodologically weak studies generate inconclusive results.
trial. The current findings underline the importance of using data
from adequately powered, high quality trials to make decisions
Discussion about telehealth implementation and caution against reliance
40 55
on meta-analyses based on small, poor quality studies.
Our
This large cluster randomised trial of second generation, home findings for second generation telehealth over 12 months mirror
based telehealth for patients with chronic obstructive pulmonary the recent null finding for third generation telehealth over 24
disease, diabetes, or heart failure found no main effect of months. Few studies have examined the effect of telehealth
41
telehealth on generic health related QoL, anxiety, or depressive on anxiety or depressive symptoms, and the current findings
symptoms over 12 months. These null findings were consistent extend our understanding of these outcomes.
across a series of sensitivity analyses for the five validated
outcome measures (tables 2 and 3). The null findings for the Strengths and limitations
primary intention to treat analyses show that telehealth is not
effective, while the null findings for the secondary per protocol The WSD telehealth trial is one of the largest randomised studies
analyses show that telehealth is not efficacious. Assessed against to evaluate the effect of telehealth on patient reported outcomes.
the trial defined MCID (equal to Cohen’s d=0.3), population A total of 1573 participants from 154 general practices across
estimates showed that the small, non-significant differences four primary care trusts (regional health authorities) provided
between trial arms in the primary analyses did not reach questionnaire data at baseline. For the intention to treat analysis,
clinically significant levels for any outcome, in any cohort, at 1201 participants from 150 practices were included in the
any time point (figs 6 and 7). available case cohort, and 759 from 131 practices in the
complete case cohort (fig 1). By including participants with any
Exploratory investigations of trial arm×time interactions showed of three long term conditions (chronic obstructive pulmonary
two significant effects for the mental component of health disease, diabetes, or heart failure), imposing minimal exclusion
related QoL and depressive symptoms (table 3). At face value, criteria, and assessing participants over 12 months, the
these findings suggest that telehealth participants deteriorated generalisability of the findings is maximised. The inclusion of
at short term assessment before recovering to levels closer to three assessment points, multiple outcome measures, and robust
baseline (and closer to usual care scores) at long term. However, statistical methods affords greater confidence in the findings.
No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe
RESEARCH
their interaction for each of the outcome measures (table 2⇓). these findings were not robust across sensitivity analyses, and
Tests of the effects of the covariates are not presented. Table 2 point estimates of effect size for these outcomes did not reach
shows that trial arm, time, and their interaction were not clinical significance at either assessment point (figs 8 and 9;
significant for any outcome measure in either cohort. table 3).
To assist with the interpretation of table 2, web table 3 presents The overall consistency of results demonstrates that the findings
unadjusted means at baseline and estimated marginal means are robust to variations in attrition (complete case v available
(EMMs) at short term and long term for all outcomes. EMMs case analyses), protocol fidelity (intention to treat v per protocol
were derived from a model that accounted for the intracluster analyses), and choice of outcome measure. The similarity of
correlation, all continuous or ordinal covariates, and baseline the patient reported outcomes across trial arms suggests that
outcome measure; but not categorical covariates (known as concerns about the potentially deleterious effect of telehealth 8 50
“factors” within SPSS). For both cohorts, the pattern of means are unfounded for most patients, since deterioration on any of
and EMMs closely mirror the parameter estimates in table 2 the five outcome measures over the assessment period was not
and reaffirm that differences between trial arms are clinically significant, compared with usual care. For the purposes of
insubstantial. Minor differences of interpretation for some service planning, the current findings should be considered with
outcomes between table 2 and web table 3 are explained by the other evidence from the WSD Evaluation on the effect of
81
underlying differences in the statistical models used to generate telehealth on hospital use and mortality and cost effectiveness. 82
the values reported.
Figures 6⇓ and 7⇓ show the adjusted effect sizes for trial arm Comparison with other studies
(telehealth v usual care). Outcomes for both cohorts at short When comparing our findings to existing research, it is important
term and long term failed to reach the trial defined MCID. to distinguish between the statistical results reported in the extant
Further, all confidence intervals crossed zero, suggesting that literature and the conclusions drawn by authors. When
estimates of the true treatment effect in the population could considering only studies evaluating broadly equivalent forms
favour either telehealth or usual care. The true direction of the of telehealth (that is, home based, vital signs monitoring using
effect is uncertain and the magnitude of the effect is clinically store and forward technology), systematic reviews have shown
insignificant. that fewer than half found any significant benefits to health
related QoL, 34 36 42 44 and those that did only found effects on
Secondary analyses: treatment efficacy only a minority of the QoL measures used. 49
In per protocol analyses, multilevel modelling generated no Despite methodological variation across studies, these findings
significant main effects for trial arm or time for any outcome suggest that the effect of telehealth on health related QoL is
measure in either cohort (table 3⇓). The two significant weak or non-existent. To this extent, the available literature
interaction terms for the SF-12 mental component score and concurs with the current findings. However, some authors have
CESD-10 in the complete case cohort reflect deteriorations for observed that the conclusions drawn in many telehealth studies
telehealth at short term, whereas scores remained stable over are often unduly positive. 42 53 With some notable exceptions, 42
time for usual care (web table 4). The interaction findings were the current study’s conclusions therefore differ markedly from
not replicated in the available case cohort (table 3). Differences most extant studies and reviews examining the effect on
between trial arms were unlikely to be clinically significant telehealth on health related QoL, which are typically interpreted
(figs 8⇓ and 9⇓). as showing benefits despite presenting equivocal evidence.
No adverse events or side effects related to any of the telehealth The scope for inappropriate inferences is increased when small
devices were reported in the intervention group throughout the and methodologically weak studies generate inconclusive results.
trial. The current findings underline the importance of using data
from adequately powered, high quality trials to make decisions
Discussion about telehealth implementation and caution against reliance
40 55
on meta-analyses based on small, poor quality studies.
Our
This large cluster randomised trial of second generation, home findings for second generation telehealth over 12 months mirror
based telehealth for patients with chronic obstructive pulmonary the recent null finding for third generation telehealth over 24
disease, diabetes, or heart failure found no main effect of months. Few studies have examined the effect of telehealth
41
telehealth on generic health related QoL, anxiety, or depressive on anxiety or depressive symptoms, and the current findings
symptoms over 12 months. These null findings were consistent extend our understanding of these outcomes.
across a series of sensitivity analyses for the five validated
outcome measures (tables 2 and 3). The null findings for the Strengths and limitations
primary intention to treat analyses show that telehealth is not
effective, while the null findings for the secondary per protocol The WSD telehealth trial is one of the largest randomised studies
analyses show that telehealth is not efficacious. Assessed against to evaluate the effect of telehealth on patient reported outcomes.
the trial defined MCID (equal to Cohen’s d=0.3), population A total of 1573 participants from 154 general practices across
estimates showed that the small, non-significant differences four primary care trusts (regional health authorities) provided
between trial arms in the primary analyses did not reach questionnaire data at baseline. For the intention to treat analysis,
clinically significant levels for any outcome, in any cohort, at 1201 participants from 150 practices were included in the
any time point (figs 6 and 7). available case cohort, and 759 from 131 practices in the
complete case cohort (fig 1). By including participants with any
Exploratory investigations of trial arm×time interactions showed of three long term conditions (chronic obstructive pulmonary
two significant effects for the mental component of health disease, diabetes, or heart failure), imposing minimal exclusion
related QoL and depressive symptoms (table 3). At face value, criteria, and assessing participants over 12 months, the
these findings suggest that telehealth participants deteriorated generalisability of the findings is maximised. The inclusion of
at short term assessment before recovering to levels closer to three assessment points, multiple outcome measures, and robust
baseline (and closer to usual care scores) at long term. However, statistical methods affords greater confidence in the findings.
No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe
