Quality of documentation of electronic medical information systems at primary health care units in Alexandria, Egypt

PDF version

M. Noureldin,¹ R. Mosallam ² and S.Z. Hassan³

جودة توثيق النُظُم الإلكترونية للمعلومات الطبية في وحدات الرعاية الصحية الأولية في الإسكندرية، مصر

مصطفى نور الدين، رشا مسلم، صفاء حسن

الخلاصة: يتوافر قدر محدود من البيانات حول تنفيذ النظم الإلكترونية للسجلات في البلدان النامية، وتهدف الدراسة الحالية إلى تقييم جودة التوثيق في السجلات الطبية الإلكترونية في وحدات الرعاية الصحية الأولية في الإسكندرية، مصر، وللتعرف على تعقيبات الأطباء حول العوائق وحول العوامل التي تسهل عمل النظام. وقد جمع الباحثون البيانات من 7 وحدات اختاروها عشوائياً من كل منطقة إدارية، وكان في كل وحدة 50 سجلاً ورقياً مع ما يقابلها من سجلات إلكترونية تم اختيارها للمرضى الذين زاروا الوحدة خلال الشهور الثلاثة الأولى من عام 2011. وقد كانت البيانات الإدارية مستكملة تقريباً في كل من السجلات الورقية والإلكترونية، إلا أن مدى اكتمال البيانات السريرية كان يتراوح بين %60 و%100 في جميع الوحدات وفي جميع أنماط السجلات. وقد كان معدل الدقة في التشخيص الرئيسي في السجلات الإلكترونية مقارنة بالسجلات الورقية يتراوح بين %44.0 و%82.0، أما العوائق الأكثر تكراراً والتي ذكرت على أنها تعرقل تنفيذ نظام السجلات الإلكترونية فهي العبء الثقيل من العمل ودرجة تعقُّد النظام.

ABSTRACT Limited data are available about the implementation of electronic records systems in primary care in developing countries. The present study aimed to assess the quality of documentation in the electronic medical records at primary health care units in Alexandria, Egypt and to elicit physician’s feedback on barriers and facilitators to the system. Data were collected at 7 units selected randomly from each administrative region and in each unit 50 paper-based records and their corresponding e-records were randomly selected for patients who visited the unit in the first 3 months of 2011. Administrative data were almost complete in both paper and e-records, but the completeness of clinical data varied between 60.0% and 100.0% across different units and types of record. The accuracy rate of the main diagnosis in e-records compared with paper-based records ranged between 44.0% and 82.0%. High workload and system complexity were the most frequently mentioned barriers to implementation of the e-records system.

Qualité de la documentation des systèmes électroniques d’information médicale dans des unités de soins de santé primaires à Alexandrie (Égypte)

RÉSUMÉ Les données disponibles sur la mise en œuvre de systèmes de dossiers électroniques en soins de santé primaires dans les pays en développement sont limitées. La présente étude visait à évaluer la qualité de la documentation des dossiers médicaux électroniques dans des unités de soins de santé primaires à Alexandrie (Égypte) et à recueillir les commentaires des médecins sur les obstacles et les leviers ayant un impact sur le système. Des données ont été recueillies au sein de sept unités sélectionnées aléatoirement dans chaque région administrative ; puis dans chaque unité, 50 dossiers au format papier et leurs dossiers électroniques correspondants ont été sélectionnés aléatoirement pour les patients qui avaient consulté dans l’unité de soins au cours de trois premiers mois de l’année 2011. Les données administratives étaient presque complètes dans les dossiers au format papier et au format électronique, mais l’exhaustivité des données cliniques variait entre 60 % et 100 % en fonction des unités de soins et du type de dossiers. Le taux d’exactitude du diagnostic principal dans les dossiers électroniques par rapport aux dossiers papiers était compris entre 44,0 % et 82,0 %. Une lourde charge de travail et la complexité du système étaient les obstacles les plus fréquemment cités à la mise en œuvre du système de dossiers électroniques.

¹Hamdan Bin Mohammed e-University, Dubai, United Arab Emirates.
²Department of Health Administration and Behavioural Sciences, High Institute of Public Health, University of Alexandria, Alexandria, Egypt (Correspondence to R. Mosallam: This email address is being protected from spambots. You need JavaScript enabled to view it.).
³Ministry of Health, Alexandria, Egypt.
Received: 09/01/13; accepted: 19/03/13
EMHJ, 2014, 20(2): 105-111

Introduction

Advances in electronic medical record (e-records) technology have made it possible to replace many functions of the traditional paper chart [1]. In primary health care (PHC), e-records have uses beyond simply retrieval of patient information; for example, warnings of allergies and drug interactions, developing management protocols for chronic illness, generating pre-appointment reminders and establishing communication links between different levels of care [2,3]. E-records are assumed to improve the quality of documentation over paper-based medical records via automatic reminders to health-care professionals of important data that are missing [4,5]. Some authors, however, have warned that inadequate computer skills or lack of training of health professionals together with limitations in e-records software could result in the data in e-records being truncated compared with paper-based records [6]. Others have raised the concern that parallel use of paper-based and e-records could result in inconsistencies between the systems, caused by failure to update both versions of the record [7].

The results of studies performed to compare the quality and completeness of documentation in paper-based and e-records have been contradictory. While some studies revealed that data recording in e-records compared favourably with paper-based records, others have demonstrated that e-records sacrificed the thoroughness of data recording that is inherent in paper-based records [7,8]. In the United Kingdom (UK), e-records scored favourably when compared to paper-records with respect to legibility, comprehensibility and completeness [6]. In Germany, the introduction of a hand-held computer was associated with improved patient assessment, coding and number of recorded diagnoses [8], while in the United States of America (USA) e-records were 40% more complete and 20% faster to retrieve than paper-based records [9]. In contrast with the previous results, 4%–13% of the documents in the electronic record were found to be missing compared with 1% missing from paper records in a Norwegian university hospital [7].

Since 1996 Egypt has adopted a health sector reform strategy aimed at achieving universal coverage with a basic package of PHC services with a special emphasis on women, children and poor people [10]. Currently, PHC units exist in 5 of Egypt’s 27 governorates, and the scheme is being rolled out across the country [11]. PHC units in Egypt deliver curative and preventive care services, with a performance-based incentive scheme for staff and an accreditation programme to ensure quality is maintained [12]. An e-records system has been implemented in PHC units to improve the quality and accuracy of documentation. The present study aimed to assess the quality of documentation in e-records compared with paper-based records at PHC units in Alexandria, Egypt, and to elicit physicians’ feedback on barriers and facilitators to the current electronic system.

Methods

Study setting

PHC units in Egypt are owned and operated by the Ministry of Health and Population and comprise family health units and family health centres. The family health units provide all first-encounter PHC services to a roster of families. Alexandria is divided into 7 administrative districts with a total of 75 PHC units. At the time of the study 45 units had functioning e-records and 30 had a non-functioning system.

Sample

In each of the 7 administrative regions, PHC units with a functioning e-records system were listed and by using tables of random numbers 1 unit was selected randomly. Inside each PHC unit, the master file registers was used to prepare a list of patients who visited the unit in the first 3 months of the year 2011. By using tables of random numbers, 50 paper-based records and their corresponding e-records were randomly selected from the prepared list. A total of 350 records were therefore examined for the 7 PHC units.

Development of data collection tool

Unstructured interviews were conducted with information technology personnel in one PHC unit and with the director of health information system in one of the administrative districts to explore items that should be completed in both paper and e-records and those that were not supported by the e-records. In addition to the unstructured interview, paper and e-records were reviewed to identify data items related to different forms. The review revealed that on the first patient encounter at the PHC facility a health family record is constructed. This record is a medical record that is unified across all family care units in Egypt. There are 3 forms: for general, dental and specific medical conditions. The general and dental forms must be completed for all patients at their first visit and data are added in follow-up visits. The specific forms must be completed if the patient has one the following conditions: diabetes, hypertension, pregnancy, referral, family planning or follow-up for a child aged < 5 years. PHC units use the International Classification of Diseases, 9th revision–clinical modification (ICD-9-CM) for classification of diseases, and physicians are trained to code the diseases themselves.

For this study a data collection form was developed to assess the concordance of information in paper-based and e-records and to determine the accuracy of diagnoses in e-records. The items in the data collection forms were coded as follows: item not documented in paper record and not entered in e-record (score 0); item documented in paper record and entered in e-record (score 1); item documented in paper record but not entered in e-record (score 2); item entered in e-record but not documented in paper record (score 3); item documented in paper record only because e-record does not support this information (score 4); and inaccurate diagnosis (score 5). In e-records there is no electronic entry for the physician signature so the presence of the physician name in the electronic form was used to denote signature completion. The coding scheme was converted into percentages for analysis.

Data collection

Records review

A review of both types of records (general medical records and dental records) was performed for patients’ initial visits to the unit and follow-up visits if both were in the first 3 months of the year 2011. Subsets of records were reviewed if the patient had any the following conditions: diabetes, hypertension, pregnancy, referral, family planning and follow-up for a child aged < 5 years. For each patient encounter the completeness of the paper-based and e-records were scored according to the coding system above. To determine the accuracy of recording diagnosis in e-records the final diagnoses were compared with that of the paper records and the percentage accuracy of diagnoses was calculated. For all analyses, the paper record was taken as the gold standard.

Physicians’ views

In order to elicit reasons for incomplete documentation of e-records and suggestions for improvement, semi-structured interviews were conducted with a sample of physicians working at family medicine units where auditing of paper and electronic records was done. The interview started with an explanation of the purpose of the interview and physicians were given the opportunity to elaborate on the reasons of incompleteness of e-records and how to overcome this problem. Physicians who agreed to participate in the interview (31 physicians) were approached and interviews were conducted at a convenient time for each physician. At least one physician was interviewed from each unit where the record review was conducted. Researchers grouped physicians’ feedback regarding barriers to successful implementation of the system into: physician-related factors; system-related factors; and patient-related factors.

Ethical considerations

Confidentiality of patients’ information in the paper and electronic medical information systems was maintained. As for physicians’ interviews, physicians were assured that their identity and responses were strictly anonymous and participation was on a voluntary basis.

Statistical analysis

Data management and analysis were performed using Microsoft Excel and SPSS, version 13.0. Descriptive statistics using frequency distribution tables were carried out. The z-test was used for comparing 2 proportions. The 5% level was used as a cut-off point for statistical significance.

Results

Records review

A total of 350 records were reviewed; the section on administrative data (11 items) had a total of 3850 data elements, while physician identification (1 item) totalled 350 data elements.

In medical forms, with the exception of the physician identification during the initial visit, all reviewed data sections showed higher or equal rates of completion in paper-based than e-records. In dental forms the pattern was more variable. There was great variability in completion rates across different PHC units, especially concerning completeness of e-records.

General medical records

In the general medical records, higher rates of completion of data elements were found in the paper-based records for items in the section on administrative data (85.1% versus 75.0%, P < 0.001) in the initial visits and the sections on complaint/examination (87.1% versus 42.1%, P < 0.001) and diagnosis and treatment (96.2% versus 81.0%, P < 0.001) in the follow-up visits, whereas e-records had a higher rate of completion compared with paper-based records for the section on physician identification (100.0% versus 62.1%, P < 0.001) in initial forms (Table 1).

Individual units varied greatly with regards to the level of completion of items in both paper and e-records. The highest variability was for the section on complaint/examination in e-records, whereby the best performing unit had a level of completion of 67.6% compared with 11.3% for the worst performing unit. As for the section on physician identification in paper-based records, the best performing unit had a level of completion of 98.0% compared with 15.0% in the worst performing unit (Table 1).

Dental records

Comparison of paper dental records with their electronic counterparts revealed a higher average rate of completed e-records items for the section on administrative data (100.0% versus 99.1%, P < 0.001) in the initial visits and the sections on date (72.8% versus 64.4%) and physician identification (100.0% versus 73.1%, P < 0.001) in the follow-up visits. On the other hand, paper-based records had higher rates of completed items for the sections on gum/dental examination (81.8% versus 73.0%, P < 0.001) and diagnosis/treatment (73.1% versus 63.0%, P < 0.004) (Table 2).

Variation between different PHC units in the rates of completion of e-records was most noticeable in the sections on gum/dental examinations in the initial visit and date and diagnosis/treatment in follow-up visits, whereby the best performing PHC units had a level of completion from 2 to 4-fold better than the worst performing units. A high level of variability also existed between individual units in item on physician identification in the follow-up sheet in the paper-based records, whereby the lowest performing units had a level of completion of 50.0% compared with 88.0% for the highest performing units (Table 2).

Specific disease records

Comparison of paper and e-records with regards to completion of items in specific disease forms showed marked variability across different diagnoses. Family planning and paediatric records had the highest rate of completed items in paper-based records (92.2% and 91.3% respectively), while diabetes and hypertension records had 100% of items completed in e-records (Table 3).

Accuracy of diagnoses

The percentage accuracy of diagnoses in e-records in the 7 different PHC units ranged from as low as 44.0% in unit no. 6 to as high as 80% in unit no. 2, with an average of 65.7% (Table 4).

Physicians’ views

The interviewed physicians mentioned 7 physician-related factors, 3 system-related factors and 1 patient-related factor as reasons for not recording data in e-records. Of these, high workload was the factor most frequently mentioned (96.8%) followed by complexity of the e-records system (67.7%). The only patient-related factor was patients’ reluctance to give the required information (29.3%).

In order to improve documentation in the e-records system, training on using the system was the factor most frequently cited by physicians (61.6%) as facilitating successful implementation. Additional suggestions about facilitating factors included physicians’ awareness about the importance of electronic documentation (51.6%), follow-up from the quality committee (48.4%), reduction of paperwork (48.4%) and initiation of incentives for electronic documentation (45.2%).

Discussion

This study aimed to assess the completeness and accuracy of documentation in electronic medical records at PHC units in Alexandria. The results showed that data recorded in e-records were significantly less complete when compared with paper-based records and that there were very wide ranges of completeness of e-records across different PHC units. The present study findings agree with studies performed in Norway and the USA [7,13] and disagree with other studies performed in Germany and the UK, where documentation in e-records compared favourably with paper-based records [6,8,9,14]. However, the level of completion and accuracy of documentation greatly lagged behind other studies. In a study performed in Norway, 4%–13% of e-records were incomplete [7], whereas the present study revealed a level of incompleteness as high as 57.9% in the section on complaint/examination in follow-up general visits and 37.0% in the section on diagnosis/treatment in follow-up dental sheets.

The high level of incompleteness of e-records in the present study could be attributed to several reasons. First, writing on paper could be easier for physicians than writing on a keyboard, especially if the computerized system has limited space for free text. Secondly, physicians may assume that the data in e-records is just a subset of the data in the paper records [7] especially if the e-records do not support all data elements in the paper records, as was the case in the present study. Thirdly, in PHC units e-records only support documentation and do not support other functions such as decision support, disease management protocols and alerts for drug interaction. Therefore physicians may consider e-records a waste of time by duplicating the paper-based records, with no added benefit to the practice. Fourthly, the level of documentation for some elements in both electronic and paper records were low (e.g. administrative and history and examination data in general forms, gum and dental examination data in dental forms) and pregnancy records. Therefore changing the tool without changing the process already in place for documentation may not result in improvements in documentation practices. Effective e-records implementation requires understanding and changing workflow to optimize e-records use in patient care [13].

The results of the present study also revealed that documentation of e-records was superior to that of paper-based records with regards to physician identification data at initial general medical visits (100.0% in e-records versus 62.1% in paper records) and in physician identification in follow-up dental visits (73.1% in e-records versus 62.1% in paper-records). It was also better in disease-specific records, in which documentation was 100.0% complete for both in diabetes and hypertension in e-records and only 82.2% and 82.8% complete respectively in paper-based records. The fact that parallel use of e-records and paper-based records results in missing data in both has been highlighted by a number of studies [7,14]. This is exacerbated in e-records of PHC units in the present study by the fact that many data elements in paper-based records were not supported in e-records. This could have a significant impact on the daily practice of the physician, who would not be able to rely on either type of record in his/her clinical decisions. If physicians were to go through both types of documentation to ensure that there are no missing data this would significantly impede their workflow [14].

Missing data in both types of records could also question the methodology of using paper records as a gold standard for assessing the quality of documentation in e-records. To overcome this defect, some studies used videotaped patient encounters as a gold standard for assessing the completeness and correctness of clinical encounter data [15,16]. Other studies highlighted the different presentation of e-records and paper records, with the former mainly composed of highly standardized data elements and the latter composed of a more or less unstructured free text. Thus, on comparing data, both types of records should be transformed into a common presentation [14].

The present study reviewed the accuracy of diagnosis in e-records as compared with paper records; the overall percentage of accuracy in diagnosis was 65.7%. This could be attributed physicians’ lack of interest in changing the type of diagnosis from provisional to final diagnosis for every case, resulting in 35% of patients having provisional diagnoses rather than final diagnoses. Simple training for physicians about the importance of documentation, including selecting the final diagnosis, is needed to overcome the problem of inaccuracy.

Our study agreed with the growing body of literature showing that high workload and complexity of the system are major barriers to using e-records [17–22]. The complexity associated with the use of e-records, especially if physicians are not well trained in their use, leads to more time allocated per patient, especially if the physician has to disrupt the clinical encounter to enter data [13,23]. Thus it is crucial to train physicians before implementing the system, and training was also the most frequently cited facilitating factor to successful implementation of the system. Additionally, physicians stressed the importance of improving awareness about e-records. This suggestion is supported by the literature stating that physicians may be sceptical about the benefits of e-records [18,20,24], which would subsequently create personal resistance to their adoption [23]. However, another study reported that following implementation of a new e-records system physicians were receptive to the new system within 1 year of implementation [25]. This would reassure managers that physicians’ initial resistance will decline by time. Physicians in our study agreed with their counterparts in other studies stating that they need a personal benefit to persuade them to switch from their traditional work procedures to a new system. The motivating incentive in most of the studies was found to be financial [17,18,23].

The present study aimed to evaluate e-records at the PHC level. This kind of evaluation is rarely done, despite its importance in capturing opportunities for improvement and in evaluating the feasibility of implementation of e-records in a country with economic constraints such as Egypt. Other strengths of this study were that all administrative regions in Alexandria were included and the study PHC units and medical records were sampled randomly. A limitation of the present study was that it did not assess the timeliness of documentation in e-records. This can be attributed to the fact that accessibility was provided to the user interface only and not to the administrator interface. Another limitation was the use of paper-based records as the gold standard to assess the completeness and accuracy of e-records. A third limitation was that the study was cross-sectional in its design, and therefore did not test trends over time.

Conclusion

In PHC units where the study was conducted, parallel use of e-records and paper-based records resulted in missing data from both types of record. The dual use of records should be transitional and limited to the implementation phase of e-records. Afterwards the paper-records should be completely replaced by the e-records. Before complete replacement, the e-records system should be able to support all data elements in the paper-based records. The capabilities of e-records systems should be widened from just supporting documentation to supporting clinical decisions, improving patient safety, acting as a reminder for patients and referral of patients between different levels of care. Physicians should receive education and training on the benefits and use of such capabilities to gain their support.

Acknowledgements

A tremendous debt of gratitude is owed to the IT personnel and physicians at primary health care units at Alexandria, Egypt for their generous support.

Funding: The study was self-funded by the researchers.

Competing interests: None declared.

References

1. Institute of Medicine, Committee on Data Standards for Patient Safety. Key capabilities of an electronic health record system: letter report. Washington DC, National Academy Press, 2003:7.
2. Tolar M, Balka E. Beyond individual patient care: enhanced use of EMR data in a primary care setting. Studies in Health Technology and Informatics, 2011, 164:143–147.
3. Tomasi E, Facchini LA, Maia MF. Health information technology in primary health care in developing countries: a literature review. Bulletin of the World Health Organization, 2004, 82:867–874.
4. Pourasghar F et al. What they fill in today, may not be useful tomorrow: lessons learned from studying Medical Records at the Women hospital in Tabriz, Iran. BMC Public Health, 2008, 8:139. doi.org/10.1186/1471-2458-8-139.
5. Farhan J et al. Documentation and coding of medical records in a tertiary care center: a pilot study. Annals of Saudi Medicine, 2005, 25:46–49.
6. Hippisley-Cox J et al. The electronic patient record in primary care—regression or progression? A cross sectional study. British Medical Journal, 2003, 326:1439–1443.
7. Mikkelsen G, Aasly J. Concordance of information in parallel electronic and paper based patient records. International Journal of Medical Informatics, 2001, 63:123–131.
8. Stengel D et al. Comparison of handheld computer-assisted and conventional paper chart documentation of medical records. A randomized, controlled trial. Journal of Bone and Joint Surgery, 2004, 86A:553–560.
9. Tsai J, Bond G. A comparison of electronic records to paper records in mental health centers. International Journal for Quality in Health Care, 2008, 20:136–143.
10. Sallam I. Health care in Egypt. Lancet, 1998, 352:1632.
11. Lie DA et al. An international physician education program to support the recent introduction of family medicine in Egypt. Family Medicine, 2004, 36:739–746.
12. Ward TR. Implementing a gatekeeper system to strengthen primary care in Egypt: pilot study. Eastern Mediterranean Health Journal, 2010, 16:684–689.
13. Hahn KA et al. Electronic medical records are not associated with improved documentation in community primary care practices. American Journal of Medical Quality, 2011, 26:272–277.
14. Stausberg J et al. Comparing paper-based with electronic patient records: lessons learned during a study on diagnosis and procedure codes. Journal of the American Medical Informatics Association, 2003, 10:470–477.
15. Pringle M, Ward P, Chilvers C. Assessment of the completeness and accuracy of computer medical records in four practices committed to recording data on computer. British Journal of General Practice, 1995, 45:537–541.
16. Logan JR, Gorman PN, Middleton B. Measuring the quality of medical records: a method for comparing completeness and correctness of clinical encounter data. Proceedings of AMIA Annual Symposium, 2001:408–412.
17. Miller RH, Sim I. Physicians’ use of electronic medical records: barriers and solutions. Health Affairs, 2004, 23:116–126.
18. Vishwanath A, Scamurra SD. Barriers to the adoption of electronic health records: using concept mapping to develop a comprehensive empirical model. Health Informatics Journal, 2007, 13:119–134.
19. Laerum H, Ellingsen G, Faxvaag A. Doctors’ use of electronic medical records systems in hospitals: cross sectional survey. British Medical Journal, 2001, 323:1344–1348.
20. Jha AK et al. Electronic health records: use, barriers and satisfaction among physicians who care for black and Hispanic patients. Journal of Evaluation in Clinical Practice, 2009, 15:158–163.
21. Meade B, Buckley D, Boland M. What factors affect the use of electronic patient records by Irish GPs? International Journal of Medical Informatics, 2009, 78:551–558.
22. Terry AL et al. Implementing electronic health records: key factors in primary care. Canadian Family Physician, 2008, 54:730–736.
23. Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Services Research, 2010, 10:231. doi.org/10.1186/1472-6963-10-231.
24. Simon SR et al. Physicians and electronic health records: a statewide survey. Archives of Internal Medicine, 2007, 167:507–512.
25. El-Kareh R et al. Trends in primary care clinician perceptions of a new electronic health record. Journal of General Internal Medicine, 2009, 24:464–468.