Page 6 Effect of telehealth on quality of life and psychological out comes over 12 months
P. 6
BMJ 2013;346:f653 doi: 10.1136/bmj.f653 (Published 26 February 2013) Page 6 of 20
RESEARCH
time points) nested within participants, and participants nested both assessments. Overall, 48.3% (759/1573) of questionnaire
within practices. The model included random intercepts and participants were included in the complete case cohort, and
random slopes at the practice level. 76.4% (1201/1573) in the available case cohort; again, with
Repeated measures for each outcome over the trial period were higher rates for telehealth (fig 1). Participants receiving
analysed with the linear mixed model procedures in SPSS. We telehealth in the WSD telehealth trial were thus more likely than
used restricted maximum likelihood to estimate model those receiving usual care to opt in to the questionnaire study,
parameters, with an ante-dependent (first order) more likely to provide data at both follow-up assessments, and
variance-covariance matrix structure. A separate analysis was consequently more likely to be included in complete and
conducted for each of the five outcome variables, and the main available case cohorts.
effect of trial arm (telehealth v usual care) was estimated to Table 1 presents sample characteristics for the 3230 participants
answer the principal research question. We estimated the main in the parent trial, the 1573 participants in the nested
effect of time to determine whether the outcome measures were questionnaire study at baseline, and those retained in the
different in the short and long term. The interaction between available case (n=1201) and complete case (n=759) cohorts.
trial arm and time (trial arm×time) was also estimated to Compared to the parent trial, the complete case cohort in the
determine whether the trial arm had differential effects at short questionnaire study had proportionally more participants from
term and long term. Kent, fewer non-white participants, fewer cases of chronic
In each model, the baseline measure of a respective outcome obstructive pulmonary disease, more cases of heart failure, a
variable was treated as a covariate, with the measures at short lower level of deprivation, and a higher level of education.
term and long term treated as the outcome. Covariates included The trial arms were closely balanced in sample size in the parent
in the model adjusted for baseline distributional differences trial cohort (telehealth=49.7% (1605/3230), usual care=50.3%
between trial arms on sociodemographic and trial related (1625/3230)) but showed a marked discrepancy in the
variables that could be related to the outcomes. questionnaire study’s complete case cohort (telehealth=56.8%
Sociodemographic covariates included age, sex, ethnicity (white (431/759), usual care=43.2% (328/759)). This discrepancy
or non-white); education (ordinal, five levels); deprivation reflected the differences in response rates already described.
(continuous data); diagnosis of chronic obstructive pulmonary Relative to the parent trial cohort, the questionnaire study’s
disease, diabetes, or heart failure; and total number of complete case cohort also had differences between trial arms
comorbidities (ordinal, nine levels). Trial related covariates in the proportion of participants in Cornwall and Newham, the
included WSD site, number of peripheral telehealth devices proportion of participants with each long term condition, and
installed (ordinal, five levels), and duration of exposure to the level of deprivation. The observed differences do not always
telehealth (in days) at short term and long term assessment show predictable patterns when comparing the parent trial cohort
(continuous data). For all parameter tests, the α level was set with the questionnaire study’s baseline, available case, and
to 0.05. complete case cohorts. Overall, table 1 shows that the
We did intention to treat analyses to assess treatment composition of the questionnaire study samples were subject
effectiveness, as the most appropriate strategy for analysing to potential bias—both in terms of patients who agreed to take
pragmatic randomised controlled trials. However, this approach part in the questionnaire study (baseline) and those who
is conservative and risks underestimating treatment effects. 79 80 completed follow-up assessments—which underlines the need
Complex healthcare interventions administered as part of a for case mix adjustment in the analyses.
pragmatic trial risk being administered suboptimally, compared
with being administered in routine care. Preliminary unadjusted analyses
Obtaining an estimate of treatment efficacy would require a Figures 4⇓ and 5⇓ present unadjusted means and 95%
heavily resourced explanatory randomised controlled trial. confidence intervals for all outcomes by trial arm at all time
However, an approximate evaluation of efficacy in pragmatic points for the two analysis cohorts. The overlapping confidence
randomised controlled trials can be achieved via per protocol intervals suggest that differences between arms at each
analyses. Thus, we conducted secondary per protocol analyses assessment point were non-significant. Web tables 1 and 2
that analysed patients according to the treatment received rather present similar analyses of unadjusted mean change scores.
than the treatment allocated (web appendix 3). Per protocol Some significant differences suggested that the telehealth arm
analyses risk overestimating the potential benefits that would had a slower rate of deterioration over time than the usual care
be observed in routine practice. Considering primary (intention arm in physical component score, anxiety, and depressive
to treat) and secondary (per protocol) analyses together can help symptoms. However, the magnitude of all mean differences in
to disentangle treatments effects from implementation effects. change scores failed to reach the trial defined MCID (web tables
Sensitivity analyses assessed the robustness of the findings to 1 and 2).
decisions taken at the analysis stage. Primary and secondary
analyses were conducted for complete and available case Primary analyses: treatment effectiveness
cohorts. Here, data in the results section are taken from the In intention to treat analyses, we used multilevel modelling to
primary analyses unless specified as being from secondary control for the baseline score of the respective outcome
analyses. measures, key covariates, and the intracluster correlation (see
methods). Key covariates included age; sex; ethnicity; education;
Results deprivation; diagnoses of chronic obstructive pulmonary disease,
Descriptive statistics diabetes, and heart failure; number of comorbidities; WSD site;
number of peripheral telehealth devices installed; and duration
At baseline, 1573 participants provided data in the questionnaire of exposure to telehealth at each assessment. Parameter
study (845 allocated telehealth, 728 usual care). The overall estimates, analogous to regression coefficients, and significance
response rate was 62.7% (986/1573) at four months and 61.9% level are shown for the main effects of trial arm (telehealth v
(974/1573) at 12 months, with a higher rate for telehealth at usual care) and time (short term v long term assessment) and
No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe
RESEARCH
time points) nested within participants, and participants nested both assessments. Overall, 48.3% (759/1573) of questionnaire
within practices. The model included random intercepts and participants were included in the complete case cohort, and
random slopes at the practice level. 76.4% (1201/1573) in the available case cohort; again, with
Repeated measures for each outcome over the trial period were higher rates for telehealth (fig 1). Participants receiving
analysed with the linear mixed model procedures in SPSS. We telehealth in the WSD telehealth trial were thus more likely than
used restricted maximum likelihood to estimate model those receiving usual care to opt in to the questionnaire study,
parameters, with an ante-dependent (first order) more likely to provide data at both follow-up assessments, and
variance-covariance matrix structure. A separate analysis was consequently more likely to be included in complete and
conducted for each of the five outcome variables, and the main available case cohorts.
effect of trial arm (telehealth v usual care) was estimated to Table 1 presents sample characteristics for the 3230 participants
answer the principal research question. We estimated the main in the parent trial, the 1573 participants in the nested
effect of time to determine whether the outcome measures were questionnaire study at baseline, and those retained in the
different in the short and long term. The interaction between available case (n=1201) and complete case (n=759) cohorts.
trial arm and time (trial arm×time) was also estimated to Compared to the parent trial, the complete case cohort in the
determine whether the trial arm had differential effects at short questionnaire study had proportionally more participants from
term and long term. Kent, fewer non-white participants, fewer cases of chronic
In each model, the baseline measure of a respective outcome obstructive pulmonary disease, more cases of heart failure, a
variable was treated as a covariate, with the measures at short lower level of deprivation, and a higher level of education.
term and long term treated as the outcome. Covariates included The trial arms were closely balanced in sample size in the parent
in the model adjusted for baseline distributional differences trial cohort (telehealth=49.7% (1605/3230), usual care=50.3%
between trial arms on sociodemographic and trial related (1625/3230)) but showed a marked discrepancy in the
variables that could be related to the outcomes. questionnaire study’s complete case cohort (telehealth=56.8%
Sociodemographic covariates included age, sex, ethnicity (white (431/759), usual care=43.2% (328/759)). This discrepancy
or non-white); education (ordinal, five levels); deprivation reflected the differences in response rates already described.
(continuous data); diagnosis of chronic obstructive pulmonary Relative to the parent trial cohort, the questionnaire study’s
disease, diabetes, or heart failure; and total number of complete case cohort also had differences between trial arms
comorbidities (ordinal, nine levels). Trial related covariates in the proportion of participants in Cornwall and Newham, the
included WSD site, number of peripheral telehealth devices proportion of participants with each long term condition, and
installed (ordinal, five levels), and duration of exposure to the level of deprivation. The observed differences do not always
telehealth (in days) at short term and long term assessment show predictable patterns when comparing the parent trial cohort
(continuous data). For all parameter tests, the α level was set with the questionnaire study’s baseline, available case, and
to 0.05. complete case cohorts. Overall, table 1 shows that the
We did intention to treat analyses to assess treatment composition of the questionnaire study samples were subject
effectiveness, as the most appropriate strategy for analysing to potential bias—both in terms of patients who agreed to take
pragmatic randomised controlled trials. However, this approach part in the questionnaire study (baseline) and those who
is conservative and risks underestimating treatment effects. 79 80 completed follow-up assessments—which underlines the need
Complex healthcare interventions administered as part of a for case mix adjustment in the analyses.
pragmatic trial risk being administered suboptimally, compared
with being administered in routine care. Preliminary unadjusted analyses
Obtaining an estimate of treatment efficacy would require a Figures 4⇓ and 5⇓ present unadjusted means and 95%
heavily resourced explanatory randomised controlled trial. confidence intervals for all outcomes by trial arm at all time
However, an approximate evaluation of efficacy in pragmatic points for the two analysis cohorts. The overlapping confidence
randomised controlled trials can be achieved via per protocol intervals suggest that differences between arms at each
analyses. Thus, we conducted secondary per protocol analyses assessment point were non-significant. Web tables 1 and 2
that analysed patients according to the treatment received rather present similar analyses of unadjusted mean change scores.
than the treatment allocated (web appendix 3). Per protocol Some significant differences suggested that the telehealth arm
analyses risk overestimating the potential benefits that would had a slower rate of deterioration over time than the usual care
be observed in routine practice. Considering primary (intention arm in physical component score, anxiety, and depressive
to treat) and secondary (per protocol) analyses together can help symptoms. However, the magnitude of all mean differences in
to disentangle treatments effects from implementation effects. change scores failed to reach the trial defined MCID (web tables
Sensitivity analyses assessed the robustness of the findings to 1 and 2).
decisions taken at the analysis stage. Primary and secondary
analyses were conducted for complete and available case Primary analyses: treatment effectiveness
cohorts. Here, data in the results section are taken from the In intention to treat analyses, we used multilevel modelling to
primary analyses unless specified as being from secondary control for the baseline score of the respective outcome
analyses. measures, key covariates, and the intracluster correlation (see
methods). Key covariates included age; sex; ethnicity; education;
Results deprivation; diagnoses of chronic obstructive pulmonary disease,
Descriptive statistics diabetes, and heart failure; number of comorbidities; WSD site;
number of peripheral telehealth devices installed; and duration
At baseline, 1573 participants provided data in the questionnaire of exposure to telehealth at each assessment. Parameter
study (845 allocated telehealth, 728 usual care). The overall estimates, analogous to regression coefficients, and significance
response rate was 62.7% (986/1573) at four months and 61.9% level are shown for the main effects of trial arm (telehealth v
(974/1573) at 12 months, with a higher rate for telehealth at usual care) and time (short term v long term assessment) and
No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe
