Search Search

Self-reported versus proxy reported quality of life for breast cancer patients in the Islamic Republic of Iran

PDF version

F. Najafi, 1 S. Nedjat, 1,2 K. Zendehdel, 3 M. Mirzania 4 and A. Montazeri 5

جودة الحياة بالإبلاغ الذاتي مقابل الإبلاغ عبر وسيط لدى المصابات بسرطان الثدي في جمهورية إيران الإسلامية

فرشته نجفي، سحرناز نجات، كاظم زنده دل، مهرزاد ميرزانيا، علي منتظري

الخلاصة: نظراً لكون جودة الحياة أمر يقرره الشخص نفسه، فإن الإبلاغ الذاتي عنه هو المصدر الرئيسي للتقييم، إلا أن هناك بعض الحالات التي لا يستطيع فيها المريض تقييم وضعه. وفي هذه الدراسة المقطعية التي شملت 148 مصابة بسرطان الثدي ممن تم تحويلهن إلى معهد السرطان في جمهورية إيران الإسلامية ، واللاتي تم اختيار القائمين على تقديم الرعاية الصحية لهن بطريقة الاختيار التتابعي. ولقد قام خمسة اختصاصيي أورام يعملون في هذا المعهد بتقييم جودة حياة هؤلاء المريضات أيضاً. وقد استكملت المجموعات الثلاث استبيان جودة الحياة للمنظمة الأوربية لبحوث ومعالجة السرطان، وتمت مقارنة النتائج. واتضح أن معامل الترابط داخل الأصناف للمرضى وللقائمين على تقديم الرعاية الصحية لهم لدى جميع المجالات في الاستبيان وعددها 15 مجالاً يتراوح بين متوسط وجيد (معامل الترابط = 0.76 - 0.41). وكانت مواطن التوافق بين مقاييس جودة الحياة لدى المرضى ولدى اختصاصي الأورام تتراوح بين متوسطة وجيدة، باستثناء 4 مجالات. وفي مجال المقارنة بين المريضات والقائمين على تقديم الرعاية كان هناك 55% من التوافق التام، وكان هناك 45% من التوافق بين المريضات والأطباء. ويمكن الاستفادة من هذه النتائج في عملية اتخاذ المرضى للقرارات وفي التخطيط للرعاية عندما لا يكون بمقدور المصابات بسرطان الثدي الإبلاغ الذاتي عن جودة حياتهن.

ABSTRACT Since quality of life (QoL) is subjective, self-reported QoL is the main source of assessment; however, in some situations the patient cannot evaluate his/her own status. In this cross-sectional study, 148 patients with breast cancer referred to the Cancer Institute of the Islamic Republic of Iran and their caregivers were selected through the consecutive sampling method. Five oncologists from this centre also evaluated the QoL of these patients. The European Organization for Research and Treatment of Cancer Quality of Life Questionnaire was completed by these 3 groups and the results compared. The patient–caregiver intra-class correlation coefficient (ICC) for all 15 QLQ.C30 domains was moderate to good (ICC = 0.41–0.76). Agreements between QoL scores of patients and those of oncologists were moderate to good, except in the 4 domains. In the patient–caregiver comparison there was 55% exact agreement, and for the patient–physician comparison agreement was 45%. The findings can be used in the patients’ decision-making process and care planning when patients with breast cancer are unable to self-report the QoL.

Qualité de vie auto-évaluée et évaluée par un tiers pour des patients atteints d’un cancer du sein en République islamique d’Iran

RÉSUMÉ La qualité de vie étant de nature subjective, l’auto-évaluation constitue l’instrument de choix pour la mesure de celle-ci. Pour autant, dans certaines circonstances, le patient n’est pas capable d’évaluer sa situation. Au cours de cette étude transversale, 148 patients adressés à l’Institut du Cancer de République islamique d’Iran pour un cancer du sein ont été sélectionnés avec leurs aidants à l’aide d’une méthode d’échantillonnage consécutif. Cinq oncologues travaillant dans ce centre ont également évalué la qualité de vie de ces patients. Le questionnaire Qualité de vie (QLQ) de l’Organisation européenne pour la Recherche et le Traitement du Cancer a été rempli par ces trois groupes et les résultats ont fait l’objet d’une comparaison. Le coefficient intra-classe (CIC) patient-aidant pour les 15 domaines du QLQ-C30 allait de modéré à bon (CIC= 0,41-0,76). La concordance entre les scores des patients portant sur la qualité de vie et ceux des oncologues étaient compris entre « modéré » et « bon », excepté dans quatre domaines. La comparaison patient-aidant donnait une concordance exacte dans 55 %, et la comparaison patient-médecin une concordance de 45 %. Les résultats peuvent être utilisés au cours du processus de décision clinique et de planification des soins quand les patients atteints d’un cancer du sein ne sont pas en mesure d’évaluer eux-mêmes leur niveau de qualité de vie.

1Epidemiology and Biostatistics Department, School of Public Health,Tehran University of Medical Sciences, 2Knowledge Utilization Research Center 3Cancer Research Center, Cancer Institute of Iran, Tehran University of Medical Sciences, Tehran, Islamic Republic of Iran 4Hematology and Medical Oncology Department, Cancer Research Center, Cancer Institute of the Islamic Republic of Iran, Imam Khomeini Hospital Complex, Tehran University of Medical Sciences, Tehran, Islamic Republic of Iran. 5Mental Health Reearch Group, Health Metrics Research Center, Iranian Institute for Health Sciences Research, ACECR, Tehran, Islamic Republic of Iran. (Correspondence to: S. Nedjat: This email address is being protected from spambots. You need JavaScript enabled to view it.).

Received: 24/08/15; accepted: 26/07/16

Introduction

Quality of life (QoL) is considered an important outcome in both health research and cancer research. It has been measured to evaluate the effects of various curative and palliative treatments, to assess the function of patients in different domains and as a criterion for clinical decision-making (1–5). The World Health Organization defines QoL as "an individual’s perception of their position in life in the context of the culture and value systems in which they live and in relation to their goals, expectations, standards and concerns" (6,7).

Since QoL, a subjective concept, shows the person’s perception of his own health status and other aspects of life, it should be self-reported. In other words, the patient is the main source of QoL assessments (3,8,9). However, there are situations in which the patient is not able or does not want to properly respond to the QoL questions. For instance, in patients with cognitive or mental disorders or in patients with serious diseases like cancer where conducting the interviews is not physically or mentally feasible, proxy-reported QoL has been suggested (5,8,10–12). Patients with breast cancer who participate in QoL studies may not be able or may not wish to provide sufficient and valid information for self-reported QoL assessment. In these cases, the proxies that are in direct contact with the patient can be considered potential substitutes for self-evaluation (12,13). The main question is how close these evaluations are to the patient’s self-reported evaluation (3,10,14–16). Caregivers and health care providers are considered main candidates for the role of proxy. There are contradictory findings about QoL assessments by physicians: some studies show good patient–physician agreement (9,14,17), while others do not (3,18). On the other hand, caregivers such as a partners, parents or children may provide a more valid evaluation of the patient’s QoL since they have a longer and closer contact with, and more understanding of, the patient (8,10–22).

The characteristics of both patient and proxy, e.g. sex, age, education, the relationship between them and the contact frequency can affect agreement (8,15,21,23). This investigation, therefore, was designed to establish whether physicians or caregivers could be valid proxies for the self-assessment of QoL. As QoL is a culture-dependent concept and has different definitions in different countries, information obtained from other parts of the world cannot be used for our Region. To the best of our knowledge, there is no evidence from the Islamic Republic of Iran and the Eastern Mediterranean Region showing that agreement between self-reporting and proxy-reporting of QoL for breast cancer patients and their characteristics has been assessed.

The objective of our study was to determine whether QoL assessments by proxies (caregivers and physicians) are in agreement with self-reported QoL for breast cancer patients. The correlations between the scores obtained by the proxies and the patients themselves were calculated. The absolute level and direction of differences between the ratings was estimated. We also aimed to explore the association between patient and proxy demographic characteristics and agreement.

Methods

Study population and sampling

The study was conducted from June to December 2014. The study population comprised patients who had been referred to the medical oncology clinic of the Cancer Institute of the Islamic Republic of Iran and had been diagnosed by pathology tests as having breast cancer at least 2 months previously. The inclusion criteria were lack of any mental disorders which could interfere with answering the questionnaires, knowing Farsi, and not being involved in any other study. Patients should have completed at least 2 chemotherapy sessions because of the acute changes to QoL during the first 2 sessions. It should be noted that chemotherapy was conducted on more than 96% of the patients of the Cancer Institute; only those with very small tumours did not undergo chemotherapy. Thus, because of the availability of the chemotherapy patients, they were considered the study population. A consecutive sampling method was applied, i.e. every patient meeting the inclusion criteria was selected till the required sample size was achieved. The sample size was calculated as 148 pairs of patients–caregivers using the common statistical formula

where d = 7, σ = 21 and β = 0.80, with type I error of 5%. We considered in-home caregivers who were the patients’ family members or relatives in this study, not the facility-based caregivers. The caregivers were asked to fill out the questionnaires independently. Each patient’s oncologist was also requested to complete the questionnaires independently.

Study tools

The QoL was evaluated using the Farsi language third version of the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ-C30) (24).

This questionnaire consisted of 30 items (28 4-point items and 2 7-point items on visual analogue scales) on 5 functional scales (physical, role, cognitive, emotional, social), 3 symptom scales (fatigue, nausea and vomiting, pain), 6 single items (dyspnoea, insomnia, appetite loss, constipation, diarrhoea, financial difficulties) and a global health status/QoL scale. In line with EORTC guidelines, all scales and single items were scored 0–100. On the functional and global QoL scales, higher scores indicate better QoL, while on the symptom scales higher scores show a higher level of problems and symptoms (25).

Demographic questionnaire

The patients’ demographic variables covered age, marital status, education and self- reported social status (5-point Likert scale). Disease stage was obtained from the patients’ files.

The caregivers’ demographic information included age, sex, education, relationship with patient and the number of hours per day spent caring for the patient.

The caregivers and physicians were asked to review the patient’s situation and answer the QLQ-C30 whose questions had been changed to third-person to evaluate the patient’s QoL.

Statistical methods

The statistical analysis was conducted using SPSS, version 20, and STATA, version 12.

The descriptive analysis of the data was done to obtain the mean and standard deviation (SD) of the questionnaire scales in the 3 study groups.

To determine level of agreement between the study groups, intra-class correlation coefficients (ICCs) for patient–caregiver and patient–physician were estimated (26). An ICC ≤ 0.40 was considered poor, 0.41–0.60 moderate, 0.61–0.80 good and 0.81–1.00 excellent agreement (5,27).

The mean absolute patient–proxy differences for 15 QLQ-C30 measures was calculated, not considering the direction of differences. The mean for directional differences was also calculated, presenting the bias in proxy scores. Significant differences from zero in directional mean scores were determined by paired t-tests and interpreted as systematic bias in proxy QoL evaluations (5,28–30). We also determined 95% confidence intervals (CI 95%).

The directional mean scores were standardized by dividing them by their SD. They were interpreted as the Cohen's d effect (0.2 = small, 0.5 = moderate and 0.8 = large bias) (31).

The magnitude of the exact response agreement and differences of more than one response category (large patient–proxy discrepancies) were calculated for each question: if the 3 raters chose exactly the same response category for each question, complete agreement was reported (3).

The association between patient–proxy agreements and characteristics was assessed using simple and multiple linear regression. Absolute and directional differences in global health status/QoL scores were considered dependent variables while the patient’s and proxy’s characteristics were considered independent variables. All variables with P-value < 0.2 in the simple linear regression analysis were included in the regression model simultaneously (32). Variables with P< 0.05 were considered statistically significant.

Ethical considerations

Participation in the study was completely voluntary. The objective of the study was explained to respondents and they were informed that their decision about taking part in the study did not have any effect on their treatment by the medical team member, and also that their responses would be kept confidential. The project was reviewed and approved by Tehran University of Medical Sciences ethics committee (Project number: 9111111013).

Results

Of 168 patients approached, 155 consented to participate in the study (92% response rate). The reasons for refusing to participate were lack of time or interest. Seven caregivers did not agree to take part. All 5 oncologists also took part in the study. Thus, the final analysis was conducted on 148 patient/proxy pairs.

The characteristics of patients and caregivers are presented in Table 1. More than 90% of caregivers were family members, with about 65% living in the same household as the patient. The mean age of patients was 47.6 (SD 10.1) years and for caregivers it was 37.6 (SD 10.2) years. The average duration of schooling for patients was 5.9 (SD 4.7) years and for caregivers it was 10.8 (SD 3.9) years. The mean time since diagnosis was 17.2 (SD 1.4) months. The mean duration of in-home care was 16.3 (SD 8.3) hours.

Table 2 shows the mean QLQ-C30 scores for patients from their own and from the proxies’ point of view. The factor with the highest mean score on the symptom scale was financial difficulties according to all 3 groups. On the functional scale, cognitive functioning gave the highest score in the viewpoint of patients and physicians while the caregivers scored role functioning highest. Patients scored global health status lower than the physicians and caregivers.

The correspondence between patients’ and proxies’ scores is shown in Table 3. The patient–caregiver agreement was not classed as weak on any of the scales (ICC < 0.40), while on the cognitive function, role function, insomnia and diarrhoea scales, the patient–physician agreement was weak.

For patient–caregiver, the mean absolute difference ranged from 9.96 to 21.40; for patient–physician this was 15.59 to 24.55 (Table 3). The mean patient–caregiver directional difference ranged from –5.01 to 6.53 and for patient–physician it was between –7.16 and 7.21.

The caregivers reported statistically significantly lower levels of QoL for patients for cognitive function, social function and fatigue compared with the patients’ reports, while the patients’ emotional function QoL was significantly higher from the caregivers’ point of view.

The differences in the QoL reports of patients and physicians were statistically significant on 11 scales, with the emotional function and global health status/QoL of the patients significantly better in the physicians’ viewpoint (Table 3).

Patient–proxy Cohen's d-values ranged from small to moderate (Table 3).

We used 4144 comparisons (28 questions × 148 cases) to calculate the “exact response category” and “large patient–proxy discrepancies”. The exact response agreement, i.e. the proportion of cases for which 2 raters had chosen exactly the same response category, was 55.0% (2291) for patient–caregiver and 45.0% (1870) for patient–physician reports. Complete agreement (i.e. 3 raters choosing exactly the same response category) was 27.5% (1143). Large patient–caregiver discrepancy was seen in 6.5% (271) of cases; this discrepancy was 10.1% (419) for patient–physician.

Association between patient–proxy agreement and patients’ and proxies’ characteristics

None of the variables had a significant association with the patient–caregiver agreement on the Global Health Status/QoL in the simple linear regression analysis (Table 4). In the multiple linear regression, the only significant variable was patients’ age. i.e. older age was associated with a greater absolute difference.

In both simple and multiple linear regression, the longer the duration of schooling, the lower the directional difference (Table 4).

Patient–physician agreement on the Global Health Status/QoL score

In the simple linear regression analysis, directional difference was statistically significantly greater with older age of the patient. None of the variables was significantly associated with absolute difference in the simple and multiple linear regression (Table 4).

Discussion

This study compared the QoL rating of patients with breast cancer receiving chemotherapy with that estimated by caregivers and physicians using the EORTC QLQ. We found that the none of the ICCs between the patients and caregivers were ≤ 0.40 (weak agreement) on all scales (ICC range: 0.41–0.76). In a study on cancer patients in the Netherlands using the same questionnaire, the ICC range was 0.46–0.73 and the lowest agreement was reported for insomnia (5). The greatest absolute difference in our study was 21.4 on the insomnia scale, which was in accordance with the findings of the Netherlands study (5). The severity of pain, dyspnoea and nausea compared with the severity of sleep problems could be the reason for the better agreement.

In the patient–physician rating, agreements were generally moderate. The only 2 scales with good patient–physician agreement were fatigue and constipation. These were in line with the findings of Blazeby et al. in a study conducted in the United Kingdom on 52 oesophageal cancer patients (33). In that study, there was weak patient–physician agreement on certain scales, including insomnia, cognitive functional, and diarrhoea; the strongest agreement was seen for the pain scale, similar to our findings. In accordance with some other studies, patient–physician agreement for the symptom scales was greater than for the functional scales (3,33). The physicians appear to pay more attention to clinical problems and the complications caused by treatment, while patients’ daily activities, concentration and entertainment, covered in the role and cognitive functional scales, do not appear to have so much importance to them. Cognitive function and role function are the personal experiences of patients, which physicians cannot observe routinely. Since all our patients were non-hospitalized, physicians could not observe them long enough. Since the caregivers can see and feel the patients’ daily activities and problems, they have non-clinical viewpoints, and hence different kinds of agreements.

Directional differences show that the caregivers reported a worse situation for the patients in the physical, role functional, emotional and global health scales, considering the negative directional differences. The highest directional difference was –5.01 for the emotional scale. These are in line with the findings of Sneeuw et al. in the Netherlands (5).

On the emotional scale, the physicians overestimated the patients’ QoL status, while on the 9 other scales they underestimated the status. Generally in our study and in other research (5), most differences were seen in the more subjective scales, particularly in the emotional scale. The higher score given by proxies for the emotional scale shows that the mental status of patients appears better than their own estimates.

Large patient–proxy discrepancies were seen for 6.5% of patient–caregiver and for 10.1% of patient–physician comparisons. These are much lower than the findings of Sneeuw et al. using the Dartmouth COOP Functional Health Assessment charts/WONCA (3), who reported 17% and 18% respectively. The different study tools are probably the main reason for this discrepancy: COOP/WONCA is a more compact tool while QLQ-C30 is more in line with the Iranian culture.

In accordance with the findings of Sneeuw et al. (5), there was no significant association between patient–proxy differences and demographic variables using regression analysis.

It is important to point out that there is no gold standard for QoL estimations, therefore we cannot always consider the proxy estimation to be wrong and those of the patients to be correct. In fact, some of these differences were related to the differences between objective health, which was assessed as better by proxies, and subjective health, which was assessed as better by patients, and was considered the same as QoL.

Study limitations

One of the study limitations was that very sick patients could not participate in our study, although in practice we do need to use proxy QoL estimation for these patients.

The caregiver’s burden and health status are important variables which were not assessed in our study although they can affect the associations we measured.

Considering the the greatest patient–proxy discrepancy was on the emotional scale, it was suggested that patients with a wide range of mental health considerations be included in fu

ture studies. In addition, since our study population comprised patients who were referred to the medical oncology clinic of the Cancer Institute, the results should be generalized with caution.

From our findings it appears that home caregivers can be considered a more authentic source of information for patients’ QoL. Caution should be used in the interpretations of physicians’ reports on patients’ QoL, especially in the cognitive and role functional domains. Our findings could be used in the patient decision-making process, research and care planning when patients with breast cancer are unable to self-report.

Funding: This study was supported by Tehran University of Medical Sciences.

Competing interests: None declared.

References

  1. Montazeri A, Harirchi I, Vahdani M, Khaleghi F, Jarvandi S, Ebrahimi M, et al. The EORTC breast cancer-specific quality of life questionnaire (EORTC QLQ-BR23): translation and validation study of the Iranian version. Qual Life Res. 2000 Mar;9(2):177–84.
  2. Brow JP, McGee HM, O’Boyle CA. Conceptual approaches to the assessment of quality of life. Psychol Health. 1997;12(6):737–51.
  3. Sneeuw KC, Aaronson NK, Sprangers MA, Detmar SB, Wever LD, Schornagel JH. Evaluating the quality of life of cancer patients: assessments by patients, significant others, physicians and nurses. Br J Cancer. 1999 Sep;81(1):87–94.
  4. Higginson IJ, Gao W. Caregiver assessment of patients with advanced cancer: concordance with patients, effect of burden and positivity. Health Qual Life Outcomes. 2008;6:42.
  5. Sneeuw KC, Aaronson NK, Sprangers MA, Detmar SB, Wever LD, Schornagel JH. Comparison of patient and proxy EORTC QLQ-C30 ratings in assessing the quality of life of cancer patients. J Clin Epidemiol.1998 Jul;51(7):617–31.
  6. Skevington SM, Lotfy M, O’Connell KA; WHOQOL Group. The World Health Organization’s WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. A report from the WHOQOL group. Qual Life Res. 2004 Mar;13(2):299–310.
  7. The World Health Organization quality of life assessment (WHOQOL): position paper from the World Health Organization. Soc Sci Med. 1995 Nov;41(10):1403–9.
  8. Slevin ML, Plant H, Lynch D, Drinkwater J, Gregory WM. Who should measure quality of life, the doctor or the patient? Br J Cancer. 1988 Jan;57(1):109–12.
  9. von Essen L. Proxy ratings of patient quality of life–factors related to patient-proxy agreement. Acta Oncol. 2004;43(3):229–34.
  10. Addington-Hall J, Kalra L. Who should measure quality of life? BMJ. 2001 Jun 9;322(7299):1417–20.
  11. Kulpeng W, Sornsrivichai V, Chongsuvivatwong V, Rattanavipapong W, Leelahavarong P, Cairns J, et al. Variation of health-related quality of life assessed by caregivers and patients affected by severe childhood infections. BMC Pediatr.201313(1):122.
  12. Magaziner J. The use of proxy respondents in health surveys of the aged. In: Wallace RB, Woolson RF, editors. The epidemiologic study of the elderly.New York: Oxford University Press; 1992 (120–9).
  13. Sprangers MA, Aaronson NK. The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease: a review. J Clin Epidemiol. 1992 Jul;45(7):743–60.
  14. Pearcy R, Waldron D, O’Boyle C, MacDonagh R. Proxy assessment of quality of life in patients with prostate cancer: how accurate are partners and urologists? J R Soc Med. 2008 Mar;101(3):133–8.
  15. Bassett SS, Magaziner J, Hebel JR. Reliability of proxy response on mental health indices for aged, community-dwelling women. Psychol Aging. 1990 Mar;5(1):127–32.
  16. Sneeuw KC, Aaronson NK, de Haan RJ, Limburg M. Assessing quality of life after stroke. The value and limitations of proxy ratings. Stroke. 1997 Aug;28(8):1541–9.
  17. Janse AJ, Gemke RJ, Uiterwaal CS, van der Tweel I, Kimpen JL, Sinnema G. Quality of life: patients and doctors don’t always agree: a meta-analysis. J Clin Epidemiol .2004 Jul;57(7):653–61.
  18. Giesinger JM, Golser M, Erharter A, Kemmler G, Schauer-Maurer G, Stockhammer G, et al. Do neurooncological patients and their significant others agree on quality of life ratings? Health Qual Life Outcomes. 2009 Oct 9;7:87.
  19. Williams LS, Bakas T, Brizendine E, Plue L, Tu W, Hendrie H, et al. How valid are family proxy assessments of stroke patients’ health-related quality of life? Stroke. 2006 Aug;37(8):2081–5.
  20. Hilari K, Owen S, Farrelly SJ. Proxy and self-report agreement on the Stroke and Aphasia Quality of Life Scale-39. J Neurol Neurosurg Psychiatry. 2007 Oct;78(10):1072–5.
  21. Sneeuw KC, Aaronson NK, Osoba D, Muller MJ, Hsu M-A, Yung WK, et al. The use of significant others as proxy raters of the quality of life of patients with brain cancer.Med Care. 1997 May;35(5):490–506.
  22. Sneeuw KC, Aaronson NK, Sprangers MA, Detmar SB, Wever LD, Schornagel JH. Value of caregiver ratings in evaluating the quality of life of patients with cancer. J Clin Oncol. 1997 Mar;15(3):1206–17.
  23. Hays RD, Vickrey BG, Hermann BP, Perrine K, Cramer J, Meador K, et al. Agreement between self reports and proxy reports of quality of life in epilepsy patients.Qual Life Res. 1995 Apr;4(2):159–68.
  24. Safaee A, Moghim Dehkordi B. Validation study of a quality of life (QOL) questionnaire for use in Iran. Asian Pac J Cancer Prev. 2007 Oct–Dec;8(4):543–6.
  25. Fayers PM, Aaronson NK, Bjordal K, Grønvold M, Curran D, Bottomley A. The EORTC QLQ-C30 scoring manual, 3rd ed. Brussels: European Organisation for Research and Treatment of Cancer; 2001.
  26. Bartko JJ. The intraclass correlation coefficient as a measure of reliability. Psychol Rep. 1966 Aug;19(1):3–11.
  27. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977 Mar;33(1):159–74.
  28. Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull. 1959 Mar;56(2):81–105.
  29. Lee J, Koh D, Ong CN. Statistical evaluation of agreement between two methods for measuring a quantitative variable. ComputBiol Med. 1989;19(1):61–70.
  30. Marshall GN, Hays R, Nicholas R. Evaluating agreement between clinical assessment methods. Int J Methods Psychiatr Res. 1994;4:249–57.
  31. Cohen J. Statistical power analysis for the behavioral sciences, 2nd ed. San Diego: Academic Press; 1969.
  32. Jewell NP. Statistics for epidemiology.New York: Chapman and Hall/CRC; 2003.
  33. Blazeby JM, Williams MH, Alderson D, Farndon JR. Observer variation in assessment of quality of life in patients with oesophageal cancer. Br J Surg. 1995 Sep;82(9):1200–3.

Privacy policy

© WHO 2024