Page 14 Effect of telehealth on quality of life and psychological out comes over 12 months
P. 14
BMJ 2013;346:f653 doi: 10.1136/bmj.f653 (Published 26 February 2013) Page 14 of 20
RESEARCH
Table 3| Parameter estimates for trial arm, time, and their interaction, per protocol analysis
Complete case cohort (n=633) Available case cohort (n=1108)
Trial arm* Time† Time×trial arm Trial arm* Time† Time×trial arm
Outcome Estimate Estimate Estimate Estimate Estimate Estimate
measure (SE) P (SE) P (SE) P (SE) P (SE) P (SE) P
PCS (US −1.17 0.761 +0.01 0.975 +2.55 0.476 +1.04 0.696 −0.04 0.926 −0.02 0.994
1998 NBS) (3.82) (0.43) (3.58) (2.66) (0.40) (2.96)
scale
MCS (US +1.74 0.724 −0.87 0.105 −10.88 0.013 +4.54 0.179 −0.63 0.217 −7.07 0.058
1998 NBS) (4.91) (0.53) (4.37) (3.38) (0.51) (3.73)
scale
EQ-5D −0.12 0.316 +0.02 0.323 +0.14 0.264 −0.05 0.573 +0.01 0.357 +0.08 0.445
scale (0.12) (0.02) (0.12) (0.09) (0.01) (0.10)
Brief STAI −1.93 0.273 −0.15 0.463 +3.10 0.129 −1.29 0.315 −0.11 0.564 +1.62 0.258
scale (1.76) (0.21) (1.70) (1.28) (0.19) (1.43)
CESD-10 −2.63 0.294 −0.02 0.945 +6.41 0.007 −2.53 0.145 −0.12 0.636 +3.65 0.062
scale (2.51) (0.27) (2.37) (1.73) (0.26) (1.96)
PCS=physical component score; MCS=mental component score; NBS=norms based scoring; SE=standard error.
Data are based on multilevel models controlling for baseline outcome score, all covariates and intraclass correlation. No specific hypotheses were made about
the effect of telehealth on particular outcomes at particular time points; therefore, any investigation of time×trial arm interaction terms must be considered exploratory
(hypothesis generating) rather than confirmatory (hypothesis testing). The value afforded to such findings when drawing inferences must be weighted accordingly.
Moreover, sensitivity analyses across multiple outcomes, cohorts, analytical approaches (intention to treat v per protocol), and parameters (trial arm, time, trial
arm×time) leads to the reporting of 60 significance tests (tables 2 and 3). At the stated α level of 0.05, we would expect three of these to be significant by chance
alone, while reducing α to 0.01 would render one of the two significant interaction term in table 3 (complete case cohort) non-significant. The lack of significant
interaction terms in the primary analyses (for both cohorts) and secondary analyses (available case cohort) highlights the general lack of robustness. Furthermore,
trial arm×time interaction terms were not significant for PCS, EQ-5D, or CESD-10 in table 3 despite ostensibly measuring closely related constructs. When a trial
produces overwhelmingly null results, there is a danger of overemphasising any significant findings, but consideration of the salient factors shows that the two
significant interaction terms are not robust, with reasonable likelihood that they reflect chance effects resulting from the additional inclusion criteria applied in the
secondary analyses. They should be interpreted with caution.
*Telehealth=0; usual care=1 (reference category).
†Short term assessment (at four months)=2, long term assessment (at 12 months)=3 (reference category). The only a priori hypothesis made about telehealth
was that it would improve health related QoL and psychological outcomes relative to usual care.
No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe
RESEARCH
Table 3| Parameter estimates for trial arm, time, and their interaction, per protocol analysis
Complete case cohort (n=633) Available case cohort (n=1108)
Trial arm* Time† Time×trial arm Trial arm* Time† Time×trial arm
Outcome Estimate Estimate Estimate Estimate Estimate Estimate
measure (SE) P (SE) P (SE) P (SE) P (SE) P (SE) P
PCS (US −1.17 0.761 +0.01 0.975 +2.55 0.476 +1.04 0.696 −0.04 0.926 −0.02 0.994
1998 NBS) (3.82) (0.43) (3.58) (2.66) (0.40) (2.96)
scale
MCS (US +1.74 0.724 −0.87 0.105 −10.88 0.013 +4.54 0.179 −0.63 0.217 −7.07 0.058
1998 NBS) (4.91) (0.53) (4.37) (3.38) (0.51) (3.73)
scale
EQ-5D −0.12 0.316 +0.02 0.323 +0.14 0.264 −0.05 0.573 +0.01 0.357 +0.08 0.445
scale (0.12) (0.02) (0.12) (0.09) (0.01) (0.10)
Brief STAI −1.93 0.273 −0.15 0.463 +3.10 0.129 −1.29 0.315 −0.11 0.564 +1.62 0.258
scale (1.76) (0.21) (1.70) (1.28) (0.19) (1.43)
CESD-10 −2.63 0.294 −0.02 0.945 +6.41 0.007 −2.53 0.145 −0.12 0.636 +3.65 0.062
scale (2.51) (0.27) (2.37) (1.73) (0.26) (1.96)
PCS=physical component score; MCS=mental component score; NBS=norms based scoring; SE=standard error.
Data are based on multilevel models controlling for baseline outcome score, all covariates and intraclass correlation. No specific hypotheses were made about
the effect of telehealth on particular outcomes at particular time points; therefore, any investigation of time×trial arm interaction terms must be considered exploratory
(hypothesis generating) rather than confirmatory (hypothesis testing). The value afforded to such findings when drawing inferences must be weighted accordingly.
Moreover, sensitivity analyses across multiple outcomes, cohorts, analytical approaches (intention to treat v per protocol), and parameters (trial arm, time, trial
arm×time) leads to the reporting of 60 significance tests (tables 2 and 3). At the stated α level of 0.05, we would expect three of these to be significant by chance
alone, while reducing α to 0.01 would render one of the two significant interaction term in table 3 (complete case cohort) non-significant. The lack of significant
interaction terms in the primary analyses (for both cohorts) and secondary analyses (available case cohort) highlights the general lack of robustness. Furthermore,
trial arm×time interaction terms were not significant for PCS, EQ-5D, or CESD-10 in table 3 despite ostensibly measuring closely related constructs. When a trial
produces overwhelmingly null results, there is a danger of overemphasising any significant findings, but consideration of the salient factors shows that the two
significant interaction terms are not robust, with reasonable likelihood that they reflect chance effects resulting from the additional inclusion criteria applied in the
secondary analyses. They should be interpreted with caution.
*Telehealth=0; usual care=1 (reference category).
†Short term assessment (at four months)=2, long term assessment (at 12 months)=3 (reference category). The only a priori hypothesis made about telehealth
was that it would improve health related QoL and psychological outcomes relative to usual care.
No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe
