Report
K.M.I. Saeed,1,3 R. Bano 2 and R.J. Asghar 3
تقييم النظام الوطني لترصُّد السل في أفغانستان
خواجة مير إسلام سعيد، راشدة سيد بانو، رانا جواد أصغر
الخلاصـة: يوجد في أفغانستان نظامان لترصُّد السل؛ البرنامج الوطني لمكافحة السل، ونظام المعلومات للإدارة الصحية. وقد أجرى الباحثون تقييماً لهذين النظامين للترصُّد في الفترة كانون الثاني/يناير – شباط/فبراير 2010 للتعرُّف على مَوَاطن القوة ومكامن الضعف ولصياغة التوصيات. وتم تقييم مكونات كل نظام باستخدام الدلائل الإرشادية لمراكز مكافحة الأمراض والوقاية منها في الولايات المتحدة الأمريكية، ووجد الباحثون أن منافع ومرونة البرنامج الوطني لمكافحة السل جيدة، وأن استقراره وحُسْن تمثيله وجودة المعطيات فيه معتدلة، أما البساطة والمقبولية وملاءمة التوقيت فيه فكانت سيئة؛ إذ يتجاوز التأخُّر في إصدار التقارير ثلاثة أشهر، وكانت القيمة التنبؤية الإيجابية %11 والحساسية %70. كما وجد الباحثون أن نظام المعلومات للإدارة الصحية بسيط ومقبول ومستقر، تُعَدُّ تقاريره في الوقت المناسب، وقد كان إعداد التقارير والإمداد بالمعلومات الارتجاعية جيداً لأن هذا النظام كان يحظَى بدعمٍ حكومي جيد. وكانت المرونة وجودة المعطيات والتمثيل معتدلة، وكانت القيمة التنبؤية الإيجابية %10 والحساسية %68. ولم يكشف أيٌّ من النظامين أية فاشية. واتضح للباحثين أن النظامين يمثِّلان ازدواجية لجهود الترصُّد وأنهما لا يغطيان القطاع الخاص.
ABSTRACT Afghanistan has 2 tuberculosis surveillance systems, the National Tuberculosis Control Programme (NTP) and the Health Management Information System (HMIS). An evaluation of these surveillance systems in January/February 2010 was done to identify their strengths and weaknesses and to formulate recommendations. Attributes of the programmes were evaluated using US Centers for Disease Control and Prevention guidelines. Usefulness and flexibility of the NTP system were good; stability, representativeness and data quality were average. Simplicity, acceptability and timeliness were poor. Reporting delays regularly exceeded 3 months. Positive predictive value and sensitivity were 11% and 70% respectively. The HMIS system was simple, acceptable and stable, with timely reporting. Reporting and feedback were good, as this system has strong government support. Flexibility, data quality and representativeness were average. Positive predictive value and sensitivity were 10% and 68% respectively. No outbreaks were detected by either system. The NTP and HMIS surveillance systems are duplicative and neither covers the private sector.
Évaluation du système national de surveillance de la tuberculose en Afghanistan
RÉSUMÉ L'Afghanistan dispose de deux systèmes de surveillance de la tuberculose, le Programme national de lutte antituberculeuse et le système d'information pour la gestion de la santé. Une évaluation de ces systèmes de surveillance en janvier/février 2010 a été menée afin d'identifier leurs forces et leurs faiblesses et de formuler des recommandations. Les caractéristiques des programmes ont été évaluées à l'aide des recommandations des Centers for Disease Control and Prevention américains. L'utilité et la souplesse du programme national de lutte antituberculeuse étaient satisfaisantes ; la représentativité et la qualité des données étaient moyennes. La simplicité, l'acceptabilité et la ponctualité étaient médiocres. Le retard de transmission des notifications dépassait régulièrement trois mois. La valeur prédictive positive et la sensibilité étaient respectivement de 11 % et 70 %. Le système d'information pour la gestion de la santé était simple, acceptable et stable, et transmettait les notifications en temps voulu. Les notifications et les commentaires étaient satisfaisants, car le système bénéficiait d'un appui soutenu du gouvernement. La souplesse, la qualité des données et la représentativité étaient moyennes. La valeur prédictive positive et la sensibilité étaient respectivement de 10 % et 68 %. Aucune flambée n'a été détectée par ces deux systèmes. Le programme national de lutte antituberculeuse et le système d'information pour la gestion de la santé font double emploi mais aucun ne couvre le secteur privé.
1Afghanistan National Public Health Institute, Ministry of Public Health, Kabul, Afghanistan (Correspondence to K.M.I. Saeed:
2Communicable Disease Surveillance and Response Unit, World Health Organization Country Office, Kabul, Afghanistan.
3Field Epidemiology and Laboratory Training Programme, National Institute of Health, Islamabad, Pakistan.
Received: 29/08/11; accepted: 21/02/12
EMHJ, 2013, 19(2):200-207
Introduction
Tuberculosis (TB) is a public health problem which is more common in developing countries due to the disease’s close links to poverty, housing status and access to health services. Furthermore, the disease has been re-emerging in developed countries in recent decades. Based on World Health Organization (WHO) estimates, 9.27 million new cases of TB occurred in 2007 (139 per 100 000 population), compared with 9.24 million new cases (140 per 100 000 population) in 2006 [1].
Afghanistan is recovering from the aftermath of more than 2 decades of civil war; nevertheless, there is ongoing security instability and economic difficulties in the country. Control of communicable diseases is one of the highest priorities for the Ministry of Public Health (MoPH) because these account for 60%–80% of all curative outpatient visits and over half of all deaths in Afghanistan [2]. The country remains one of the TB high-burden countries with a high ratio of females to males infected (1.85:1). TB control services are an integral part of the package of services delivered through the primary health care system at district and provincial levels. The sustainability of activities is unclear, however, given the unstable security situation with its current reliance on donor support.
TB control is a priority for the country, being one of the main 7 components of the Basic Package of Health Services for Afghanistan [3]. TB care is provided free of charge in all public health facilities. It is very important to strengthen surveillance systems in order to find TB patients as early as possible and treat them properly. There are surveillance 2 systems which, in addition to other activities, are responsible for TB surveillance in Afghanistan: the Health Management Information System (HMIS) [4] and the National Tuberculosis Control Programme (NTP). This paper is an evaluation report which describes the national tuberculosis surveillance systems in Afghanistan, identifies strength and weakness of the systems and provides recommendations for improvement.
Methods
The review was undertaken during January to February 2010. In order to evaluate the TB surveillance systems at the national level the guidelines for the evaluation of public health surveillance systems developed by the United States Centers for Disease Control and Prevention (CDC) [5] were used. These guidelines include the following steps:
Engage the stakeholders in evaluation.
Describe the surveillance system to be evaluated.
Focus the evaluation design.
Gather credible evidence regarding the performance of the surveillance system.
Justify and state conclusions and make recommendations.
Ensure use of evaluation findings and share lessons learned.
Using the guidelines, the surveillance systems for one disease were taken into acount, the sampling was purposeful and the head of departments were approached to obtain necessary information. Both the Afghan NTP and HMIS were evaluated by applying these guidelines. We did a desk review before interviewing managers in both systems. The managers of other relevant departments were also interviewed. We did not use any statistical tests of significance; however, selected indices were used and calculated as described in the guidelines [5]. Ten attributes (usefulness, simplicity, flexibility, data quality, predictive value, sensitivity, timeliness, acceptability, representativeness and stability) were evaluated in both systems. Attributes were scored based on pre-defined criteria and ranked as poor, average or good. The main documents of each system (strategic plan, guidelines, manuals, annual reports and databases) were reviewed and consultation with stakeholders was carried out. In addition 5 key informants from each system were interviewed from among the directors, managers and officers. Furthermore 1 or 2 staff at departments of the MoPH (research, surveillance, preventive health, policy and planning and health care service delivery) were interviewed; they were selected based on their involvement in and relevance to the TB surveillance system.
Description of the TB surveillance systems
The Afghan NTP was established in 1954 with technical and financial support from WHO. The NTP is a technical department of the MoPH that leads and carries out prevention, detection, diagnosis and treatment of TB patients. Its vision is a TB-free country, with elimination of the disease as a public health problem by 2050 [6]. The major objectives of the programme are to reduce the risk of infection, morbidity and mortality due to TB by having 100% coverage of DOTS treatment by the end of year 2015 and ensuring the cure rate and detection rate of new sputum smear-positive pulmonary TB cases at over 85% and 70% respectively. There are 13 recording and reporting forms which are used by the surveillance department for data management. The programme claims that data are collected, analysed and disseminated quarterly in review meetings. Originating from public health facilities, the data flows to the provinces and then to regions and finally to the NTP at the national level, where multiple national indicators are calculated and reported to the MoPH and WHO Regional Office for the Eastern Mediterranean. The flow of information is presented in Figure 1. The system is supported by WHO, the United States Agency for International Development (USAID), the Japan International Cooperation Agency, the Global Fund to Fight AIDS, Tuberculosis and Malaria and some other donors.
The Afghan health information system was upgraded to HMIS in 2003 [4] and focuses on standard reporting forms containing the minimum information that needs to be collected, analysed and reported routinely. The system is computerized, which allows data on priority health problems at the local levels to be aggregated on a database at the provincial and national levels and rapid distribution of analysis copies to all levels. The main aims of the system are better management of health services and resources, facilitating supervision and assisting in planning, monitoring and evaluation. In HMIS data are collected about suspected TB cases and the number of slides examined. Data generated at the facility level are reported to provincial levels monthly for analysis and feedback and afterwards the reports go to the national level quarterly for feedback and dissemination. Figure 2 illustrates the flow of information. The system is supported by USAID and the World Bank.
Attributes of the systems
A summary of the evaluation of the 10 attributes from the CDC guidelines are presented in Table 1.
Usefulness
HMIS data are useful for planning and monitoring but are less useful for detecting TB outbreaks, whereas proper analysis of NTP data will detect and allow a response to outbreaks. TB data collected by HMIS are poorly linked to action, while NTP data are used by both national and international stakeholders. Based on research department statements at MoPH they have not used TB surveillance data for the development and conduct of studies; however, NTP surveillance data are used by the TB research department and WHO for various studies.
Simplicity
Standard case definitions are well-utilized in both systems and facility registries (logbooks) are the source of data. HMIS collects few variables whereas NTP uses consistent, standard forms to collect more variables regarding morbidity and mortality of TB including age, sex and geography. The HMIS database is linked and fully integrated with other databases at MoPH, while the NTP database is still in the pilot stage. Prior to sending to the national level, data are edited, analysed and shared with stakeholders at provincial levels and submitted electronically to the MoPH quarterly. The HMIS is simple, quick and easy to enter and analyse the data. The Microsoft Access database is familiar to staff and facilitated by a series of simple drop-down menus. However, data collection and flow of information is more complex. Taking into account the very high turnover of staff the officers are well-trained at all levels. All implementers at primary health care facilities, including government, national and international NGOs are involved in the reporting of data and timely quarterly feedback is provided to them from the national level.
Using multiple forms for data collection makes the NTP system more complex both in its structure and mode of operation. Data are sent as hard copies to all levels, with delays at each level. Vital TB indicators such as case notification rate, case detection rate, TB conversion rate and treatment success rate are calculated using available data in NTP. With proper feedback mechanisms the main vehicle for communication is discussing the key issues using data in the quarterly review meetings at provincial, regional and national levels. Staff are well-trained regarding TB surveillance but there is still a need to conduct training at the grass-roots level in health facilities for effective TB case management. Outbreak detection and investigation throughout the country is not recorded either by HMIS or NTP. Some other issues include the complex data collection (many forms and ways of reporting), flow of information (delays) and laboratory algorithms (different categories, not just positive or negative). There are also issues with the NTP conducting crosschecks of the national reference laboratory data and random samples of slides from regions which are using different indicators.
Flexibility
HMIS seems to be less flexible for accommodating changes when there is a need for additional information or modes of operation, however, removing questions (variables) is easier than adding new questions to the forms. Being supported by the government and having staff in the government salary system, the HMIS is sustainable, unlike a project based approach in which the funding is withdrawn when the project is finished. It also conforms with planned new software systems in which common files will link to other databases such as training, expenditure, pharmacy, human resources and others. Variations in funding have had little effect on the functioning of the system.
In contrast, the NTP system is flexible and able to adapt and accommodate changes when there is need for additional information or reporting in TB forms. However, the system is considerably donor-dependent and cessation of funding and support is a concern for its sustainability. For example, there was an interruption in funding for laboratory services and no reporting was done in late 2008.
Data quality
While the HMIS has progressed since it was set up in 2003 in terms of quality of data, some blank cells and illegal values are still observed in the forms; for instance, it is known that some facilities have reported that they are performing sputum smear examinations when in reality they do not have microscopes available. By and large the HMIS is collecting complete and valid data from registries by checking for missing and illegal values (out of a predetermined range) at different levels. Data management, including data collection, entry, editing, analysis and feedback, is good due to timely feedback and training.
The NTP has expanded rapidly but there are still blank and illegal values in the forms despite checks on the quality of data at provincial, regional and national levels in quarterly review meetings. It covers more comprehensive data compared with the HMIS. However, there are difficulties in data management at NTP at all levels, including reporting, analysis and feedback.
Acceptability
HMIS benefits from acceptability among all MoPH, NGOs and international agencies. Data are collected from registries and there is no need to interview and collect the data for routine cases. A high rate of data reporting is observed at all facilities and almost no refusals in participation are seen. Complete and timely data are available. However, HMIS is less acceptable by TB-related stakeholders due to problems with incomplete and duplicate data.
The NTP on the other hand is not fully acceptable at all levels of health workers due to the multitude of forms and high workload at primary level health facilities. The system even lacks acceptability within the TB programme among those working at grass-roots levels, such as heads of health facilities. Additionally private sector acceptability is another issue. WHO, however, is more supportive of NTP and uses the data for its regional and global reports. A weak point is that quarterly data sent to the NTP for analysis are delayed and incomplete. On the other hand, quality control practices are good as laboratory slides are collected from facilities randomly for crosschecking.
Sensitivity of the surveillance system
We defined sensitivity as the ability of the surveillance system to truly detect cases of TB in the country (sensitivity = cases detected/cases existing × 100). No survey has been conducted to establish the exact burden of TB in Afghanistan and the system is using WHO estimates for case detection and other indicators.
HMIS data
Based on HMIS case definitions for TB the patient is recorded as suspect and he or she is sent for sputum examination, but the HMIS is not able to trace the contacts. The following is a calculation of the system sensitivity using data from the HMIS database for 2008:
Total TB suspected cases reported: 146 981
Total sputum smear-positive cases estimated: 17 885
Total TB sputum smear-positive cases detected: 12 229
Sensitivity (12 229/17 885) × 100 = 68%
NTP data
The NTP system is collecting epidemiological and laboratory data from all facilities covered by DOTS. The contact tracing is another problem which is not done by the system and reduces its sensitivity. We calculated sensitivity just at the routine reporting level in 2008. We can calculate sensitivity at the level of all TB cases or at the level of sputum smear-positive cases.
Total TB suspected cases (prevalence): 40 000
Total TB suspected cases detected : 28 300
Sensitivity (first level) (28 300/40 000) × 100 = 71%
Using data from smear-positive cases shows almost the same sensitivity:
Total TB sputum smear-positive cases estimated: 17 885
Total TB sputum smear-positive cases detected: 13 136
Sensitivity (second level) (13 136/17 885) × 100 = 73%
Predictive value positive of the surveillance system
We defined predictive value positive (PPV) as the proportion of acid-fast bacilli (AFB)-positive slides out all slides examined by laboratory facilities (PPV = true positives/all positives × 100).
HMIS data
After a review of data and interviews with HMIS staff at the central level in 2008 data at HMIS we calculated the PPV in 2 ways. First, it was based on the proportion of AFB slides that were positive out of all slides examined for AFB:
Total AFB slides examined: 283 831
Total AFB slides positive: 28 951
PPV (28 951/282 831) × 100 = 10%
Secondly, we can calculate the proportion of TB cases starting treatment (i.e. true positives) out of those identified as suspected TB:
Total TB cases suspected: 146 981
Total TB cases started treatment : 21 487
PPV (21 487/146 981) × 100 = 15%
NTP data
There are possibilities of laboratory examination for AFB at facilities but that is not possible to do culture except at the NTP reference laboratory in Kabul. For each TB suspect a sputum sample is collected and 2 to 3 AFB slides are examined both for diagnosis and follow up. The following 2008 data from the NTP were identified:
Total AFB slides examined: 268 614
Total AFB slides positive: 30 447
PPV (30 447/268 614) ×100 = 11%
Representativeness
HMIS covers public facilities including government and NGOs; however, 15% of the country is not covered by public health facilities and a considerable proportion of the population are utilizing the private sector, are not seeking medical care and/or are seeking care through other means. In addition the system is not collecting sufficient information about the demographic and socioeconomic status of the population. It therefore cannot be said that the system is representative of the whole population. Nevertheless, HMIS covers fewer variables and is trying to cover the private sector and military health facilities as well in efforts towards improving the representativeness of the data.
NTP has expanded its DOTS coverage exponentially and is now collecting data from 1013 facilities, which are almost 90% of Basic Package of Health Services facilities. The system is collecting sufficient information regarding age and sex of cases along with their residential areas. There are 2 sources of information: basic management units (facilities) and laboratory registries. Similar to the HMIS, the NTP misses data from private sector and military health facilities.
Timeliness
Timeliness was quantified as delays more than 3 months. At the community level the amount of time it takes for clients in the community to seek medical care is very difficult to determine. In HMIS data are taken from health facilities and sent to the provinces monthly. At the provincial level data are analysed and shared with all stakeholders and after cleaning data are forwarded as soft copies to the national level at the end of each quarter without delays. More than 95% of facilities are sending their reports regularly. Collection and examination of sputum is also timely at facilities. Feedback is provided after the quarterly data collection from central HMIS to those facilities in which problems are identified in analysis.
NTP is collecting information quarterly using laboratory and facility quarterly formats. It is also reporting treatment outcome and conversion of sputum-smear positive cases. However, the quarterly reports are not sent on time from facilities to the provincial level or from the province to the regions or from the regions to the national level. Delay in reporting is therefore a concern. Furthermore, feedback is not provided timely and properly. All data are sent as hard copies from the lower level to the central NTP, which is not acceptable in the current challenging conditions of Afghanistan. Timeliness therefore is a concern for the NTP.
Stability
The HMIS is able to collect, manage and provide data properly without failure from all its facilities and its fully operational with its monthly reporting at the provincial levels and quarterly reporting at the central level. All stakeholders are entitled to receive the HMIS analysis folder for their own analysis of data including TB. The Microsoft Access database is simple and user-friendly, is well developed and few resources including human and non-human are required for its maintenance. It takes very little time to enter, edit, transfer, analyse, store and retrieve the data. Problems in extraction of information do occur, however, due to occasional breakdown in the electricity power supply.
Due to delays in reporting of data timely information is not available within the NTP. Data are collected at health facilities and laboratory facilities in hard copies for each quarter and are entered into a Microsoft Excel as well as a new Access database. These databases are very simple for data entry and analysis and few resources are needed for maintenance of the database. Corruption of databases due to viruses is always a concern. The system needs to be made electronic at all levels particularly at provincial and regional levels.
Discussion
HMIS is a very good system which provides timely information through daily recording and monthly and quarterly reporting and feedback. NTP is collecting comprehensive information but with delays in reporting and feedback. So timeliness, as a key surveillance system metric, is better in the HMIS than the TB surveillance system under the NTP. Unfortunately the information is only occasionally linked to action in HMIS while annual data is mostly used for planning at the national and international level at NTP. The HMIS system has a high degree of acceptability among stakeholders because it has been developed with the involvement and participation of various HMIS taskforces. The NTP surveillance system is less acceptable due to its complexity and multitude of forms.
HMIS is collecting fewer variables regarding TB compared with the NTP. There is poor coordination or sharing of information between the 2 surveillance systems regarding TB and this causes duplication of effort, wastage of resources and incomplete information. Both systems are collecting data from all public health facilities and both are missing data from the private sector facilities where many cases come and are managed.
Both systems have approximately equal sensitivity and PPVs. The low prevalence of smear-positive cases in the population has affected the PPV, lowering it to 10%. Both systems are able to show the seasonal and secular trend of TB cases, which generate hypotheses and trigger research questions.
Despite a review of the literature we could not find any similar evaluations for comparison, but some unpublished reports of public surveillance evaluations from the Field Epidemiology and Laboratory Training Programme in Pakistan provide similar results regarding TB surveillance systems.
The following points are recommended based on findings of this evaluation.
Attention should be given to integrate both systems in order to avoid duplication and to strengthen mechanisms for coordinating with related departments to use HMIS data for action.
Taking the lesson learnt from acceptability and sustainability of HMIS the surveillance system at NTP should search for mechanisms for improving its acceptability among health workers. There is a need to reconsider the variables in both the HMIS and NTP surveillance systems.
The system should assess the reliability of data sources or triangulate the data with other data sources. Refresher training of health workers at the lower levels on guidelines and case definitions of TB will increase their skills and improve the quality of data reported.
Better adherence to guidelines is needed in both of the systems in terms of collecting 3 samples to be sent to laboratories for confirmation. NTP should collect information on contacts and enhance contact tracing.
Both systems should try to strengthen public–private partnerships and incorporate private sector data into their systems in order to increase the utilization and representativeness of findings.
The concept of timeliness and prompt reporting should be applied equally to both systems and for dissemination of reports as well. The end users should receive the reports in hard and soft copies as soon as possible.
Establishment and protection of databases from power outrages and computer viruses will improve the stability of the systems. Double entry of data will enhance the data quality and prevent missing and erroneous data. More efforts are needed to ensure the stability and sustainability of the systems.
The general coordination, integration, decentralization and expansion of activities need to be strengthened in order to avoid duplication and provide information for action.
Acknowledgements
This study was supported by Field Epidemiology and Laboratory Programme, National Institute of Health, Islamabad, Pakistan. In addition the National TB Control Programme and Health Management Information System at Ministry of Public Health were supportive in the evaluation. I would like to thank all who were involved in this study in one way or another.
References
- WHO report 2009. Global tuberculosis control: epidemiology, strategy, financing. Geneva, World Health Organization, 2009 (WHO/HTM/TB/2009.411).
- Health Management Information System. Data analysis. Kabul, Afghanistan, Ministry of Public Health, 2009.
- Basic Package of Health Services (BPHS). Kabul, Afghanistan, Ministry of Public Health, 2006.
- National Health Management Information System procedures manual. Kabul, Afghanistan, Ministry of Public Health, 2006.
- Updated guidelines for evaluating public health surveillance systems: recommendations from the guidelines working group. Morbidity and Mortality Weekly Report, 2001, 50 No. RR-3.
- Annual report and statistics of tuberculosis in Afghanistan—2008. Afghanistan. Kabul, Afghanistan, National TB Control Program, Ministry of Public Health, 2008.
