Online Appendix for the following JACC: Cardiovascular Imaging article TITLE: The Limit of Plausibility For Predictors of Response: Application to Biventricular Pacing (Cardiac Resynchronization Therapy), Systematic Review and Design Steps For Reliable Research AUTHORS: Sukhjinder S. Nijjer, BSC, MB CHB, Punam A. Pabari, MB CHB, PHD, Berthold Stegemann, PHD, Vittorio Palmieri, MD, PHD, Francisco Leyva, MD, Cecilia Linde, MD, PHD, Nick Freemantle, PHD, Justin E. Davies, BSC, MBBS, PHD, Alun D. Hughes, MB BS, PHD, Darrel P. Francis, MA, MD ___________________________________________________________________________ APPENDIX Determining the contraction factor Contraction factor imposed by dyssynchrony marker If a mechanical dyssynchrony marker is measured several times in one patient, the observed values (xobserved) will not all be identical, but rather be scattered randomly with a notional stable true underlying value (xtrue) and an error component (xerror), i.e. xobserved = xtrue + xerror. Here, the word “error” includes observer error, equipment error, operator error, and (most importantly) genuine biological variability, all of which cause the observed value to differ from the true underlying value. When one measurement in one patient is compared with one measurement in another patient, only part of the difference is due to true underlying difference between patients: the remainder is due to error. Across a population, the total observed variance between individuals is composed of two components: the true underlying variance between individuals (Variance of xtrue) and the size of the error component (Variance of xerror). The total variance of xobserved is: ππππππππ of π₯πππ πππ£ππ = ππππππππ of π₯π‘ππ’π + ππππππππ of π₯πππππ The component of variation arising from xerror is clinically completely uninformative because it is random, changing each time it is measured rather than representing anything meaningful for the patient. Therefore of the variation observed in x, only the remaining proportion is meaningful and therefore has any hope of correlating with another variable: Proportion of variation in π₯πππ πππ£ππ that is meaningful = =1− ππππππππ ππ π₯π‘ππ’π ππππππππ ππ π₯πππ πππ£ππ ππππππππ ππ π₯πππππ ππππππππ ππ π₯πππ πππ£ππ The final formula above expresses quantitatively the notion that if a measurement has wide test-retest irreproducibility (xerror), then only a small proportion of the observed differences between patients are actually meaningful. It goes on to say that if test-retest reproducibility is as wide as the observed spread in the population then all observed differences between patients are completely meaningless. Only the meaningful proportion of the variation can correlate with another variable, and so if there is any underlying relationship between x and another variable y, with an underlying association strength of R2underlying, the observed association strength R2observed will be contracted by this factor. Contraction factor imposed by response marker The same applies for a marker of response: yobserved = ytrue + yerror. Here, “error” refers to all variability between LV function measurements over time within individuals who do not receive biventricular pacing. HSSCSs characteristically do not report this. EMRCTs sometimes do, as the standard deviation of change in LV measurement (e.g., ΔLVEF) over time, in the control arm. In the control arm, there is no biventricular pacing is given, and so “error” is the only source of variation: this means Variance of yerror = Variance of ΔLVEF in the control arm. In the device arm, there is both “error” variation and variation due to biventricular pacing having different-sized effects in different patients. This means that the Variance of yerror + Variance of ytrue = Variance of ΔLVEF in the biventricular pacing arm. Applying the same reasoning as for x, we obtain: Proportion of Variance in π¦πππ πππ£ππ that is meaningful = 1 − ππππππππ ππ π¦ ππ πΆπππ‘πππ πππ ππππππππ ππ π¦ ππ πΆπ π πππ Only this meaningful proportion of the variation can correlate with another variable, and so if there is any underlying relationship between y and another variable x, with an underlying association strength of R2underlying, the observed association strength R2observed will be contracted by this factor. The combined contraction factor If there is an underlying relationship between x and y, the observed relationship strength will be lower, because of both of the above contraction processes: R2observed = R2underlying × [1 − Variance of xerror Variance of y in Control arm ] × [1 − ] Variance of xobserved Variance of y in CRT arm How to screen the plausibility of a reportedly observed R2 value If a study reports a high R2observed value, a plausibility check can quickly be carried out with the above formula, perhaps made more convenient by rewriting it like this: 2 2 πΌππππππ π π’ππππππ¦πππ = π πππ πππ£ππ [1 − ππππππππ ππ π₯πππππ ππππππππ ππ π¦ ππ πΆπππ‘πππ πππ ] × [1 − ] ππππππππ ππ π₯πππ πππ£ππ ππππππππ ππ π¦ ππ πΆπ π πππ For example, if the study’s reported standard deviation of dyssynchrony is 50 ms, and the blinded test-retest reproducibility standard deviation of xerror is 30 ms, then the first bracketed term is 1 – 302/502. The chosen response marker may have credible reports of spontaneous variation, for example from an RCT. If the SD of “response” was in the control arm 9 units, and in the biventricular pacing arm 11 units, then the second bracketed term is 1 – 92/112. Thus, if the reported R2 observed is 0.80, this would only be sustainable if the underlying R2 value were 0.80/[(1 – 302/502) (1 – 92/112)] = 3.8. This goes beyond “highly skilled” into “impossible.” Online Supplemental References 1. Pitzalis MV, Iacoviello M, Romito R, Massari F, Rizzon B, Luzzi G, Guida P, Andriani A, Mastropasqua F, Rizzon P. Cardiac resynchronization therapy tailored by echocardiographic evaluation of ventricular asynchrony. J Am Coll Cardiol. 2002;40:1615–22. 2. Yu CM, Fung WH, Lin H, Zhang Q, Sanderson JE, Lau CP. Predictors of left ventricular reverse remodeling after cardiac resynchronization therapy for heart failure secondary to idiopathic dilated or ischemic cardiomyopathy. Am J Cardiol. 2003;91:684–8. 3. Gorcsan J III, Kanzaki H, Bazaz R, Dohi K, Schwartzman D. Usefulness of echocardiographic tissue synchronization imaging to predict acute response to cardiac resynchronization therapy. Am J Cardiol 2004;93:1178–81. 4. Notabartolo D, Merlino JD, Smith AL, DeLurgio DB, Vera FV, Easley KA, Martin RP, León AR. Usefulness of the peak velocity difference by tissue Doppler imaging technique as an effective predictor of response to cardiac resynchronization therapy. Am J Cardiol. 2004;94:817–20. 5. Penicka M, Bartunek J, De Bruyne B, Vanderheyden M, Goethals M, De Zutter M, Brugada P, Geelen P. Improvement of left ventricular function after cardiac resynchronization therapy is predicted by tissue Doppler imaging echocardiography. Circulation. 2004;109:978–83. 6. Pitzalis MV, Iacoviello M, Romito R, Guida P, De Tommasi E, Luzzi G, Anaclerio M, Forleo C, Rizzon P. Ventricular asynchrony predicts a better outcome in patients with chronic heart failure receiving cardiac resynchronization therapy. J Am Coll Cardiol. 2005;45:65–9. 7. Yu CM, Zhang Q, Fung JW, Chan HC, Chan YS, Yip GW, Kong SL, Lin H, Zhang Y, Sanderson JE. A novel tool to assess systolic asynchrony and identify responders of cardiac resynchronization therapy by tissue synchronization imaging. J Am Coll Cardiol. 2005;45:677–84. 8. Duncan AM, Lim E, Clague J, Gibson DG, Henein MY. Comparison of segmental and global markers of dyssynchrony in predicting clinical response to cardiac resynchronization. Eur Heart J. 2006;27:2426–32. 9. Mele D, Pasanisi G, Capasso F, De Simone A, Morales MA, Poggio D, Capucci A, Tabacchi G, Sallusti L, Ferrari R. Left intraventricular myocardial deformation dyssynchrony identifies responders to cardiac resynchronization therapy in patients with heart failure. Eur Heart J. 2006;27:1070–8. 10. Porciani MC, Lilli A, Macioce R, Cappelli F, Demarchi G, Pappone A, Ricciardi G, Padeletti L. Utility of a new left ventricular asynchrony index as a predictor of reverse remodelling after cardiac resynchronization therapy. Eur Heart J. 2006;27:1818–23. 11. Suffoletto MS, Dohi K, Cannesson M, Saba S, Gorcsan J III. Novel speckle-tracking radial strain from routine black-and-white echocardiographic images to quantify dyssynchrony and predict response to cardiac resynchronization therapy. Circulation. 2006;113:960–8. 12. Yu CM, Zhang Q, Chan YS, Chan CK, Yip GW, Kum LC, Wu EB, Lee PW, Lam YY, Chan S, Fung JW. Tissue Doppler velocity is superior to displacement and strain mapping in predicting left ventricular reverse remodelling response after cardiac resynchronisation therapy. Heart. 2006;92:1452–6. 13. Bleeker GB, Schalij MJ, Boersma E, Holman ER, Steendijk P, van der Wall EE, Bax JJ. Relative merits of M-mode echocardiography and tissue Doppler imaging for prediction of response to cardiac resynchronization therapy in patients with heart failure secondary to ischemic or idiopathic dilated cardiomyopathy. Am J Cardiol. 2007;99:68–74. 14. Diaz-Infante E, Sitges M, Vidal B, Mont L, Delgado V, Marigliano A, Macias A, Tolosana JM, Tamborero D, Azqueta M, Roig E, Paré C, Brugada J. Usefulness of ventricular dyssynchrony measured using M-mode echocardiography to predict response to resynchronization therapy. Am J Cardiol. 2007;100:84–9. 15. Gorcsan J 3rd, Tanabe M, Bleeker GB, Suffoletto MS, Thomas NC, Saba S, Tops LF, Schalij MJ, Bax JJ. Combined longitudinal and radial dyssynchrony predicts ventricular response after resynchronization therapy. J Am Coll Cardiol. 2007;50:1476-83. 16. Sassone B, Capecchi A, Boggian G, Gabrieli L, Saccà S, Vandelli R, Petracci E, Mele D. Value of baseline left lateral wall postsystolic displacement assessed by M-mode to predict reverse remodeling by cardiac resynchronization therapy. Am J Cardiol. 2007;100:470–5. 17. Soliman OI, Theuns DA, Geleijnse ML, Anwar AM, Nemes A, Caliskan K, Vletter WB, Jordaens LJ, Cate FJ. Spectral pulsed-wave tissue Doppler imaging lateral-toseptal delay fails to predict clinical or echocardiographic outcome after cardiac resynchronization therapy. Europace. 2007;9:113–8. 18. Van de Veire NR, Bleeker G, De Sutter J, Ypenburg C, Holman ER, van der Wall EE, Schalij MJ, Bax JJ. Tissue synchronization imaging accurately measures left ventricular dyssynchrony and predicts response to cardiac resynchronization therapy. Heart. 2007;93:1034–9. 19. Yu CM, Gorcsan J III, Bleeker GB, Zhang Q, Schalij MJ, Suffoletto MS, Fung JW, Schwartzman D, Chan YS, Tanabe M, Bax JJ. Usefulness of tissue Doppler velocity and strain dyssynchrony for predicting left ventricular reverse remodeling response after cardiac resynchronization therapy. Am J Cardiol. 2007;100:1263–70. 20. De Boeck BW, Meine M, Leenders GE, Teske AJ, van Wessel H, Kirkels JH, Prinzen FW, Doevendans PA, Cramer MJ. Practical and conceptual limitations of tissue Doppler imaging to predict reverse remodelling in cardiac resynchronisation therapy. Eur J Heart Fail. 2008;10:281–90. 21. Delgado V, Ypenburg C, van Bommel RJ, Tops LF, Mollema SA, Marsan NA, Bleeker GB, Schalij MJ, Bax JJ. Assessment of left ventricular dyssynchrony by speckle tracking strain imaging comparison between longitudinal, circumferential, and radial strain in cardiac resynchronization therapy. J Am Coll Cardiol. 2008;51:1944–52. 22. Marsan NA, Bleeker GB, Ypenburg C, Ghio S, van de Veire NR, Holman ER, van der Wall EE, Tavazzi L, Schalij MJ, Bax JJ. Real-time three dimensional echocardiography permits quantification of left ventricular mechanical dyssynchrony and predicts acute response to cardiac resynchronization therapy. J Cardiovasc Electrophysiol. 2008;19:392–9. 23. Bank AJ, Kaufman CL, Kelly AS, Burns KV, Adler SW, Rector TS, Goldsmith SR, Olivari MT, Tang C, Nelson L, Metzig A; PROMISE-CRT Investigators. Results of the Prospective Minnesota Study of ECHO/TDI in Cardiac Resynchronization Therapy (PROMISE-CRT) study. J Card Fail. 2009;15:401-9. 24. Conca C, Faletra FF, Miyazaki C, Oh J, Mantovani A, Klersy C, Sorgente A, Pedrazzini GB, Pasotti E, Moccetti T, Auricchio A. Echocardiographic parameters of mechanical synchrony in healthy individuals. Am J Cardiol. 2009;103:136-42. 25. Deplagne A, Bordachar P, Reant P, Montaudon M, Reuter S, Laborderie J, Dos Santos P, Roudaut R, Jais P, Haissaguerre M, Laurent F, Clementy J, Lafitte S. Additional value of three-dimensional echocardiography in patients with cardiac resynchronization therapy. Arch Cardiovasc Dis. 2009;102:497-508. 26. Faletra FF, Conca C, Klersy C, Klimusina J, Regoli F, Mantovani A, Pasotti E, Pedrazzini GB, De Castro S, Moccetti T, Auricchio A. Comparison of eight echocardiographic methods for determining the prevalence of mechanical dyssynchrony and site of latest mechanical contraction in patients scheduled for cardiac resynchronization therapy. Am J Cardiol. 2009;103:1746-52. 27. Liodakis E, Sharef OA, Dawson D, Nihoyannopoulos P. The use of real-time threedimensional echocardiography for assessing mechanical synchronicity. Heart. 2009;95:1865-71. 28. Soliman OI, Geleijnse ML, Theuns DA, van Dalen BM, Vletter WB, Jordaens LJ,Metawei AK, Al-Amin AM, ten Cate FJ. Usefulness of left ventricular systolic dyssynchrony by real-time three-dimensional echocardiography to predict long-term response to cardiac resynchronization therapy. Am J Cardiol. 2009;103:1586-91. 29. Bordachar P, Lafitte S, Réant P, Reuter S, Clementy J, Mletzko RU, Siegel RM, Goscinska-Bis K, Bowes R, Morgan J, Bénard S, Leclercq C. Low value of simple echocardiographic indices of ventricular dyssynchrony in predicting the response to cardiac resynchronization therapy. Eur J Heart Fail. 2010;12:588-92. 30. Kaufman CL, Kaiser DR, Burns KV, Kelly AS, Bank AJ. Multi-plane mechanical dyssynchrony in cardiac resynchronization therapy. Clin Cardiol. 2010;33:E31-8. 31. Norisada K, Kawai H, Tanaka H, Tatsumi K, Onishi T, Fukuzawa K, Yoshida A, Hirata K. Myocardial contractile function in the region of the left ventricular pacing lead predicts the response to cardiac resynchronization therapy assessed by twodimensional speckle tracking echocardiography. J Am Soc Echocardiogr. 2010;23:181-9. 32. Park HE, Chang SA, Kim HK, Shin DH, Kim JH, Seo MK, Kim YJ, Cho GY, Sohn DW, Oh BH, Park YB. Impact of loading condition on the 2D speckle trackingderived left ventricular dyssynchrony index in nonischemic dilated cardiomyopathy. Circ Cardiovasc Imaging. 2010;3:272-81. 33. Van Bommel RJ, Ypenburg C, Borleffs CJ, Delgado V, Marsan NA, Bertini M, Holman ER, Schalij MJ, Bax JJ. Value of tissue Doppler echocardiography in predicting response to cardiac resynchronization therapy in patients with heart failure. Am J Cardiol. 2010;105:1153-8. 34. Wang CL, Wu CT, Yeh YH, Wu LS, Chang CJ, Ho WJ, Hsu LA, Luqman N, Kuo CT. Recoordination rather than resynchronization predicts reverse remodeling after cardiac resynchronization therapy. J Am Soc Echocardiogr. 2010;23:611-20. 35. Shanks M, Bertini M, Delgado V, Ng ACT, Nucifora G, van Bommel RJ, Borleffs CJW, Holman ER, van de Veire NRL, Shaliji MJ, Bax JJ. Effect of Biventricular Pacing on Diastolic Dyssynchrony. J Am Coll Cardiol. 2010;56:1567-75. 36. Lim P, Buakhamsri A, Popovic ZB, Greenberg NL, Patel D, Thomas JD, Grimm RA. Longitudinal strain delay index by speckle tracking imaging: a new marker of response to cardiac resynchronization therapy. Circulation. 2008;118:1130-7. 37. Lim P, Donal E, Lafitte S, Derumeaux G, Habib G, Réant P, Thivolet S, Lellouche N, Grimm RA, Gueret P. Multicentre study using strain delay index for predicting response to cardiac resynchronization therapy (MUSIC study). Eur J Heart Fail. 2011;13:984-91. 38. Marcus GM, Rose E, Viloria EM, Schafer J, De Marco T, Saxon LA, Foster E; VENTAK CHF/CONTAK-CD Biventricular Pacing Study Investigators. Septal to posterior wall motion delay fails to predict reverse remodeling or clinical improvement in patients undergoing cardiac resynchronization therapy. J Am Coll Cardiol. 2005;46:2208-14. 39. Bax JJ, Bleeker GB, Marwick TH, et al. Left ventricular dyssynchrony predicts response and prognosis after cardiac resynchronization therapy. J Am Coll Cardiol 2004;44:1834–40. 40. Jansen AHM, Bracke F, van Dantzig JM, et al. The influence of myocardial scar and dyssynchrony on reverse remodeling in cardiac resynchronization therapy. Eur J Echocardiogr 2008;9:483–8. 41. St John Sutton MG, Plappert T, Abraham WT, Smith AL, DeLurgio DB, Leon AR, Loh E, Kocovic DZ, Fisher WG, Ellestad M, Messenger J, Kruger K, Hilpisch KE, Hill MR; Multicenter InSync Randomized Clinical Evaluation (MIRACLE) Study Group. Effect of cardiac resynchronization therapy on left ventricular size and function in chronic heart failure. Circulation. 2003;107:1985-90. 42. Abraham WT, Young JB, León AR, Adler S, Bank AJ, Hall SA, Lieberman R, Liem LB, O'Connell JB, Schroeder JS, Wheelan KR; Multicenter InSync ICD II Study Group. Effects of cardiac resynchronization on disease progression in patients with left ventricular systolic dysfunction, an indication for an implantable cardioverterdefibrillator, and mildly symptomatic chronic heart failure. Circulation. 2004;110:2864-68. 43. Cappola TP, Harsch MR, Jessup M, Abraham WT, Young JB, Petersen-Stejskal S, Plappert T, St John Sutton M. Predictors of remodeling in the CRT era: influence of mitral regurgitation, BNP, and gender. J Card Fail. 2006;12:182-8. 44. St John Sutton MG, Plappert T, Hilpisch KE, Abraham WT, Hayes DL, Chinchoy E. Sustained reverse left ventricular structural remodeling with cardiac resynchronization at one year is a function of etiology: quantitative Doppler echocardiographic evidence from the Multicenter InSync Randomized Clinical Evaluation (MIRACLE). Circulation. 2006;113:266-72. 45. Cleland JG, Daubert JC, Erdmann E, Freemantle N, Gras D, Kappenberger L, Tavazzi L; Cardiac Resynchronization-Heart Failure (CARE-HF) Study Investigators. The effect of cardiac resynchronization on morbidity and mortality in heart failure. N Engl J Med. 2005;352:1539-49. 46. Ghio S, Freemantle N, Serio A, Magrini G, Scelsi L, Pasotti M, Cleland JG, Tavazzi L. Baseline echocardiographic characteristics of heart failure patients enrolled in a large European multicentre trial (CArdiac REsynchronisation Heart Failure study). Eur J Echocardiogr. 2006;7:373-8. 47. Cleland J, Freemantle N, Ghio S, Fruhwald F, Shankar A, Marijanowski M, Verboven Y, Tavazzi L. Predicting the long-term effects of cardiac resynchronization therapy on mortality from baseline variables and the early response a report from the CAREHF (Cardiac Resynchronization in Heart Failure) Trial. J Am Coll Cardiol. 2008;52:438-45. 48. Cleland JG, Calvert MJ, Verboven Y, Freemantle N. Effects of cardiac resynchronization therapy on long-term quality of life: an analysis from the CArdiac Resynchronisation-Heart Failure (CARE-HF) study. Am Heart J. 2009;157:457-66. 49. Ghio S, Freemantle N, Scelsi L, Serio A, Magrini G, Pasotti M, Shankar A, Cleland JG, Tavazzi L. Long-term left ventricular reverse remodelling with cardiac resynchronization therapy: results from the CARE-HF trial. Eur J Heart Fail. 2009;11:480-8. 50. Wikstrom G, Blomström-Lundqvist C, Andren B, Lönnerholm S, Blomström P, Freemantle N, Remp T, Cleland JG; CARE-HF study investigators. The effects of aetiology on outcome in patients treated with cardiac resynchronization therapy in the CARE-HF trial. Eur Heart J. 2009;30:782-8. 51. Moss AJ, Hall WJ, Cannom DS, Klein H, Brown MW, Daubert JP, Estes NA 3rd, Foster E, Greenberg H, Higgins SL, Pfeffer MA, Solomon SD, Wilber D, Zareba W; MADIT-CRT Trial Investigators. Cardiac-resynchronization therapy for the prevention of heart-failure events. N Engl J Med. 2009;361:1329-38. 52. Solomon SD, Foster E, Bourgoun M, Shah A, Viloria E, Brown MW, Hall WJ, Pfeffer MA, Moss AJ; MADIT-CRT Investigators. Effect of cardiac resynchronization therapy on reverse remodeling and relation to outcome: multicenter automatic defibrillator implantation trial: cardiac resynchronization therapy. Circulation. 2010;122:985-92. 53. Linde C, Abraham WT, Gold MR, St John Sutton M, Ghio S, Daubert C; REVERSE (REsynchronization reVErses Remodeling in Systolic left vEntricular dysfunction) Study Group. Randomized trial of cardiac resynchronization in mildly symptomatic heart failure patients and in asymptomatic patients with left ventricular dysfunction and previous heart failure symptoms. J Am Coll Cardiol. 2008;52:1834-43. 54. St John Sutton M, Ghio S, Plappert T, Tavazzi L, Scelsi L, Daubert C, Abraham WT, Gold MR, Hassager C, Herre JM, Linde C; REsynchronization reVErses Remodeling in Systolic left vEntricular dysfunction (REVERSE) Study Group. Cardiac resynchronization induces major structural and functional reverse remodeling in patients with New York Heart Association class I/II heart failure. Circulation. 2009;120:1858-65. 55. Lubitz SA, Leong-Sit P, Fine N, Kramer DB, Singh J, Ellinor PT. Effectiveness of cardiac resynchronization therapy in mild congestive heart failure: systematic review and meta-analysis of randomized trials. Eur J Heart Fail. 2010;12:360-6. 56. Foley PW, Patel K, Irwin N, Sanderson JE, Frenneaux MP, Smith RE, Stegemann B, Leyva F. Cardiac resynchronisation therapy in patients with heart failure and a normal QRS duration: the RESPOND study. Heart. 2011;97:1041-7. 57. Beshai JF, Grimm RA, Nagueh SF, Baker JH 2nd, Beau SL, Greenberg SM, Pires LA, Tchou PJ; RethinQ Study Investigators. Cardiac-resynchronization therapy in heart failure with narrow QRS complexes. N Engl J Med. 2007;357:2461-71. 58. Pouleur AC, Knappe D, Shah AM, Uno H, Bourgoun M, Foster E, McNitt S, Hall WJ, Zareba W, Goldenberg I, Moss AJ, Pfeffer MA, Solomon SD; for the MADITCRT Investigators. Relationship between improvement in left ventricular dyssynchrony and contractile function and clinical outcome with cardiac resynchronization therapy: the MADIT-CRT trial. Eur Heart J. 2011;32:1720-1729.