Strengthening national health information systems: challenges and response

PDF version

Ala Alwan ¹, Mohamed Ali ¹, Eman Aly 1, Azza Badr ¹, Henry Doctor ¹, Ahmed Mandil ¹, Arash Rashidian ¹ and Olla Shideed ¹

A well-functioning national health information system (HIS) is a prerequisite for the provision of reliable and timely health-related information. This information is essential for: 1) policy development and evidence-informed decision-making; 2) proper health management and rational resource allocation; and 3) monitoring and evaluation of health systems and other related social services performance. National HIS draw upon multiple data sources, including civil registration and vital statistics systems, census, population-based surveys, routinely generated data from health facilities and administrative information systems. These are supported by governance structures, human and financial resources infrastructure and information, communications technology, quality assurance procedures, data management and standards, and clear plans for dissemination, use of the information and evidence generated from the system.

Learning from the experience of the Millennium Development Goals (MDGs), generation, availability and accessibility of timely and quality information for key health indicators is essential for monitoring the progress towards achieving the targets of the health-related Sustainable Development Goals (SDGs). Despite the progress made by many countries in monitoring health during the MDG era, the lack of reliable, timely and comparable information in low- and most middle-income countries remains an issue (1) and often hampers tracking and evaluation of progress. As a result, there has been reliance on estimates and modelling to substitute for unavailable key indicators. This process surely comes with important limitations, including observations from national authorities that argue the estimates do not always reflect the reality in countries. This situation calls for effective and sustained action to strengthen national health information systems, as well as reinforce the capacity of countries in generating, compiling, analyzing, disseminating and reporting reliable data for the monitoring of health situations in countries.

For the specific case of the SDGs, several SDG3 targets focus on different measures of mortality. These range from maternal mortality, nenonatal mortality, preventable mortality from noncommunicable diseases (NCDs), and mortality from environmental hazards, to road traffic accidents and other forms of violence (2,3). Consequently, strengthening national civil registration and vital statistics is imperative if countries are to have adequate data systems. This is especially important when 53% (4) of all deaths globally are not appropriately registered and only one in five deaths in the Region (5) is medically certified and coded using the International Classification of Diseases (ICD), which also suffers from non-ignorable proportions of ill-defined causes (6,7). As a result, the reporting of cause-specific mortality in the SDGs’ monitoring and accountability framework will face notable constraints if further improvements are not achieved. This area has seen very little progress over the past 25 years; in 2014, only 50% of Member States, globally, reported cause-of-death data to WHO, compared to 45% in 1990 (8). Regionally, only 11 countries regularly reported to WHO cause-specific mortality data during 2008–2012 (9).

Strengthening HIS in countries of the Eastern Mediterranean Region (EMR) has been given a special priority in WHO’s work in the Region during the past five years. Extensive work was done by the WHO Secretariat and a series of regional meetings – engaging multisectoral representatives of Member States, relevant United Nations (UN) agencies and regional organizations – were organized. The aim was to review the regional situation, identify constraints and collectively develop and implement strategic directions to address these, and support countries in reinforcing informed decision-making and strengthening their capacity to monitor national health development. The outcome of this work was regularly presented, discussed and endorsed at the highest level by Ministers of Health and other health policy-makers during the annual sessions of the WHO Regional Committee for the Eastern Mediterranean and other high-level meetings. Two Regional Committee resolutions were passed, which now represent a roadmap for countries and serve as the basis of WHO’s work (10). The objective of this paper is to review the current status, actions taken and present the priorities and strategic directions for both countries, WHO and other partners in this area of work.

The current status

Throughout WHO’s work with EMR Member States, a number of common gaps have been identified, which hinder development and strengthening of national HIS. There is often a lack of clarity on the essential health indicators that should be monitored as core in all countries. In many countries, political commitment and priority given to HIS is inadequate due to fragmentation of existing systems coupled with low capacity to collect, verify and disseminate data and information.

In 2012, efforts to strengthen health systems in the Region were initiated with detailed situation analysis of the status of different components of the health systems in Member States (11). For practical reasons, Member States were categorized into three groups based on population health outcomes, health system performance and health expenditure. A more in-depth analysis of the situation of national HIS across the three groups of countries was reported during the 61st Session of the WHO Regional Committee for the Eastern Mediterranean in 2014 (9). The conclusion drawn was that although countries across the three groups are at different stages of development, availability of skilled human capacity and financial resources, there are still major gaps that are shared across the three groups of countries.

All Group 1 countries have national HIS centres or units with dedicated staff as well as national plans. However, some plans are of limited scope in the areas of data collection, data analysis and capacity strengthening. In all countries, national HIS are computerized, with varying degrees of functionality and integration. With regards to basic data quality assessments and feedback, they are performed in all countries, although the assessment is not based on standardized tools and processes. All countries regularly publish annual health statistics reports in various formats (i.e. print, digital and World Wide Web), noting that dissemination through the World Wide Web has improved substantially in recent years.

A set of defined national indicators exists in most Group 1 countries with varying degrees of quality, completeness and frequency of reporting. All, except one country, collect cause-specific mortality data using ICD, but the quality of the collected data is an issue in all except one country. At the time of the review in 2014, the percentage of reporting of indicators ranged from 25% to 80%. Specifically, the reporting of: i) health determinants and risks indicators ranged from 25% to 90%; ii) health status indicators in all countries except one; and iii) health system response indicators ranged from 10% to 80%. A key challenge is regular updating of population data by age, sex, and nationality due to the high rate of in- and out-migration in these countries.

All Group 2 countries, except one, have national (and subnational) HIS centres with predominately information technology (IT) staff, as well as a national plan in the majority of the countries. Basic data quality assessments and feedback are limited; while all countries publish annual health statistics reports, only two disseminate these reports through the World Wide Web.

A set of core national and programme-specific indicators are defined in most Group 2 countries, and range from 52 to 152 indicators. Only one country reports complete cause-specific mortality data using ICD on a regular basis. Birth and death data are generally incomplete, as well as data on human resources for health. At the time of the review in 2014, the reporting of: i) health determinants and risks indicators ranged from 50% to 60%; ii) health status indicators at 70%; and iii) health system performance indicators at 50%.

All Group 3 countries, except two, have national HIS centres or units that lack capacity in most countries, especially those at the periphery, with only a few having a national plan, which is often outdated or incomplete. Data quality assurance and analysis through systematic methods are lacking. While all countries, except one, publish annual health statistics reports, there is a 2 to 3 year time lag. Across all countries, dissemination of health statistics through the web is either unavailable or inaccessible. Population-based and facility-based surveys are not conducted on a regular basis.

Most Group 3 countries lack a system of routine reporting, especially on cause-specific mortality data, with ICD coding not systematically used in all countries. At the time of the review in 2014, the reporting of: i) health determinants and risks indicators was at 50%; ii) health status indicators were few and largely based on global estimates; and iii) health system performance indicators were less than 50% in the majority of countries.

Other constraints include:

Weak coordination between national stakeholders and fragmentation of health information and civil registration and vital statistics systems, both within and across ministries. This also includes:

Limited access to routinely generated health care service data from the private sector, and often services by public health systems not under the ministry of health (e.g. insurance organizations, armed forces)

Lack of national comprehensive plans for health information systems including national population surveys and irregular conduct of such surveys

Limited governance or legal oversight from ministries of health and national statistics offices in obtaining the required data and ensuring their availability as part of use.

Weak and often incomplete reporting of deaths, cause-specific mortality data, the use of ICD and mortality statistics.

Out of 131 Member States, globally, with vital registration systems that provided summary data on cause-of-death statistics to WHO, only 60 countries had data that met certain criteria of completeness, accuracy, regular reporting, and disaggregated by sex and age (12).

Out of the 22 countries of the Region, seven have never reported to WHO causes of death using ICD. Eleven countries (50%) have major gaps in mortality data reporting and only four countries have continuously reported annual mortality data (13).

Weak reporting on key morbidity indicators: population-based cancer registries and integrated disease surveillances are weak or non-existent in most countries.

In the 2015 Global Survey on assessing national capacity for the prevention and control of NCDs conducted by WHO, less than 60% (14) of countries are reporting data on cancer incidence and type based on a population based cancer registry.

Ineffective leveraging of opportunities provided by information communication and technology (ICT) in data collection, analysis and dissemination.

Inadequacy of quality assurance procedures and lack of use or shared understanding of data collection and use standards and procedures.

Routinely collected data (e.g. health service data) often are not linked with established quality assurance procedures.

Shortage of skilled human resources in epidemiology, statistics, IT and disease and risk factors surveillance.

Preliminary results from assessments of essential public health functions in countries of the Region clearly demonstrate shortage of skilled human resource capacity in related health information disciplines. In many cases where capacity exists in the countries, retention and adequate distribution poses a challenge.

What is an effective health information system?

Health information systems include all activities and resources related to public health monitoring and reporting. This includes collecting data from the health sector and other relevant sectors, analysing the data and ensuring their overall quality, relevance and timeliness, and converting data into information for health-related decision-making (15). Health information systems provide the underpinnings for decision-making and have five key functions: data generation, compilation, analysis and synthesis, communication of information, and use (Figure 1).

The Health Metrics Network defines a national HIS as being “made up of all the data and records about the population’s health. The sources of data include civil and vital registration (recording births, deaths and causes of death), censuses and surveys, individual medical records, service records and financial and resource tracking information” (16) .

For policy-makers who recognize the major gaps that exist in their national HIS and the constraints encountered in addressing them, one practical question is to determine the essential health indicators that they need to monitor as part of their commitment to strengthen their health system and achieve universal health coverage. Because countries will vary in terms of socioeconomic development and health achievements, it is important to identify what is core for all countries irrespective of income and public health capacity and what can be added as an expanded list to cater for national priorities and programme-specific needs.

There is also agreement that good quality data on a small number of key indicators is more informative for policy, planning and prioritization than a larger number of indicators generated from unreliable data. With this understanding, the WHO Regional Office for the Eastern Mediterranean facilitated and led extensive discussions with Member States and partners through a series of technical meetings and expert consultations conducted between 2012 and 2014. The focus of the discussion was to have an in-depth assessment of the current status of health information systems including civil registration and vital statistics and identify the type of indicators that are considered essential in monitoring health trends and health development.

Two strategic initiatives to strengthen health information systems

It was clear from the in-depth assessment conducted in 2012 on the regional health challenges that reinforcing health information systems is one of the key priorities for WHO’s work with Member States . In this respect, two main strategic directions were adopted: a) reaching consensus on a practical framework for national HIS and establishing a core set of indicators that can be feasibly implemented in all countries; and b) strengthening civil registration and vital statistics as one of the most critical elements of HIS. Both strategic directions require the commitment and active engagement of other government sectors outside health.

Health Information Systems framework and regional core indicators

There have been several global frameworks proposed for HIS (17–19). The challenge policy-makers face is to adopt a framework that is practical, easy to understand, feasible to implement and covers the key areas necessary for health policy development and monitoring. Both WHO and Member States of the Region agreed that the three broad areas requiring action and monitoring are the essential determinants of health, health status, and health system capacity and response. This consequently led to consensus on the key components of a national HIS and identification of what needs to be monitored under each component (Figure 2). The three areas are: i) monitoring health risks and determinants; ii) assessing health status including morbidity and cause-specific mortality; and iii) evaluating health system performance.

To realize the Framework and ensure that national HIS are capable of effectively tracking health issues, it was agreed that a minimum dataset of core indicators should be prioritized and collected. Through a series of meetings, spanning 2013 and 2014, work continued to develop a list of core indicators (Figure 3) with a concise metadata registry. Every effort was made to ensure that the included indicators are valid, specific, relevant, feasible and affordable to generate. In consultation with Member States, stakeholders and experts, a total number of 68 core indicators were agreed, with an expanded (as relevant) list of 115 additional indicators for countries that need to have a wider coverage according to their capacity and needs. In this respect, since most countries are currently unable to report reliably on all the core indicators, it will be important for them to prioritize addressing the gaps before expanding.

An essential part of the initiative was the active engagement of stakeholders from ministries of health and interior, as well as central bureaus of statistics or national statistics offices; UN agencies; and international and regional experts.

The aim of the regional context-specific concise metadata registry (20), which draws upon the WHO global metadata registry, is to facilitate standardized data collection, analysis and reporting. A number of attributes are described for each indicator, including: the indicator name and abbreviated name, domain/subdomain, related/associated term(s), definition, measurement and estimation methods, primary/preferred and alternate/other possible data source(s), disaggregation and measurement frequency, numerator and denominator (see annex 1 for an example) (9,21).

Since its endorsement during the 61st Session of the WHO Regional Committee for the Eastern Mediterranean in 2014, WHO reports annually on the core indicators and verifies data with Member States (22). This has also served as the basis for the data that is included in the brief health system profiles that are updated annually and provide a snapshot covering key health system indicators, achievements, strengths, weaknesses and priorities for action. The health system profiles aim to help policy-makers focus on the assets and challenges within their countries.

The Regional Office has embarked on a process to assess countries’ capacity in reporting on the core indicators, through a rapid assessment. The assessment of country capacity to regularly report on the core indicators is a basic step to support countries to plan for their HIS strategy and monitor their progress towards achievement of SDGs targets. The first phase of the initiative has entailed the development of an assessment tool on the capacity to report on the core indicators, covering key domains of: existing data sources, frequency of data collection, data analysis, existing resources and use of appropriate standards for data collection, processing, and analysis. Member States undertook the rapid assessment, which was completed by most countries in August 2016 through a web-based questionnaire.

The initial results of 19 countries that participated in the survey reveal that although there is an improvement in reporting on the core indicators since their endorsement in 2014, none of the countries in the Region are able to report on all 68 core indicators in a timely manner. In 2016, reporting on the core indicators ranged from a minimum of 38 indicators and a maximum of 58 indicators (Figure 4). The results were discussed during a regional meeting of country delegations during August 2016. Discussions included key challenges and approaches to ensure that countries have functional health information systems, are able to conduct population-based surveys, improve reporting of the core indicators, and report on the SDG indicators that are not part of the core health indicators.

Improving civil registration and vital statistics systems

As mentioned before, assessing health status is one of the three basic components of HIS. It covers reporting and monitoring of cause specific mortality, which represents a major gap in HIS globally and in the EMR particularly. Addressing this gap requires strengthening of civil registration and vital statistics (CRVS).

Civil registration refers to “the continuous, permanent, compulsory, and universal recording of the occurrence and characteristics of vital events (live births, deaths, fetal deaths, marriages, and divorces) and other civil status events pertaining to the population as provided by decree, law or regulation, in accordance with the legal requirements in each country” (23,24).

A special priority has been given to CRVS by WHO as part of the regional initiative to strengthen national HIS in Member States. A starting point was to assess the current status in countries of the region. Rapid assessments of CRVS were carried out in 21 countries, using a standard assessment tool (25) in all countries from November 2012 to January 2013, through multi-stakeholder meetings. This was followed by comprehensive assessments (26) conducted in 21 countries from 2013 to 2016. Results of the assessments revealed that only 6 countries (29%) have satisfactory CRVS systems that produce data of sufficient quality to adequately cover the needs for policy decision-making (Figure 2). However, the systems in these countries cover only 5.3% of the EMR population. Eight countries (almost 40%) have either weak or dysfunctional CRVS systems (27). Additionally, the assessments revealed a major gap in birth and death registration, where more than six million (40%) births and more than three million (67%) deaths are unregistered (Figure 5). Furthermore, only 19% of deaths are registered and medically certified with adequate cause-of-death registration (Figure 6). Analysis of death registration figures by country indicated that almost half of the countries in the region (12 countries) attain registration figures equal to or greater than 80% while only 6 countries have registration figures lower than the regional average of 33%. These six countries are now the focus of WHO’s efforts to strengthen this area of work.

A regional strategy for the improvement of civil registration and vital statistics systems was developed by WHO, through intensive consultations with Member States and stakeholders. The strategy endorsed by the WHO Regional Committee in 2013, includes seven strategic domains (6):

Ensure a sound legal and regulatory framework for civil registration and vital statistics systems.

Strengthen registration infrastructure, resources and capacities.

Remove barriers at all levels to registration and the issuance of related legal documentation.

Improve mortality certification and coding practices.

Improve production, use and dissemination of vital statistics.

Improve intersectoral coordination and alignment among civil registration and vital statistics stakeholders.

Maintain and strengthen the existing regional and global partnerships in support of country strategies.

The strategy is designed to provide Member States with a list of interventions to implement, based on the level of development of their civil registration and vital statistics systems, country contexts, resources and capacities.

Based on the results of the rapid and comprehensive assessments conducted in countries over the past few years, WHO recently provided each Member State with a specific road map with a list of prioritized actions. Work has focused on providing technical expertise to help the relevant national authorities to take appropriate action and address the identified gaps.

The country-specific recommendations focus on a number of shared issues: 1) establishing national committees to oversee CRVS implementation; 2) establishing and updating electronic civil registration systems to follow international standards for recording births, deaths and causes of deaths; 3) developing a birth/death notification process to capture vital events that occur in communities; 4) enforcing decrees that forbid burials without death certificates; 5) expanding registration infrastructure by providing required resources; 6) capacity building in ICD-10 for physicians and training of medical students; and 7) capacity building in analysis and quality checks of vital statistics data.

The way forward and policy implications

The work conducted over the last five years has provided comprehensive information and a clear picture of the situation in each country with regards to their national HIS including civil registration and vital statistics. The joint work with countries and other stakeholders has also resulted in a clear vision and an evidence-based roadmap for both the Region and individual countries.

The HIS framework adopted in the Region has been appreciated by policy-makers as a well-defined, practical and focused approach, yet covering the essential data necessary for policy-making and monitoring of national health development. As mentioned before, a core set of 68 indicators under the three key components of the framework provides a comprehensive assessment of health situation in countries and the capacity and response of the health system. Some countries may argue for a higher number of indicators covering additional aspects and will use the expanded list. However, there is consensus among countries that the first priority is to address the gaps that currently exist in reporting on the core indicators. In this respect, it is encouraging to observe a significant improvement in the reporting of the core indicators since their endorsement in 2014.

The extensive work with Member States and other stakeholders has also laid the foundation for improving the completeness and accuracy of registration of birth and causes of death. Countries now have sound and clear guidance represented by the regional strategy for the improvement of civil registration and vital statistics 2014–2019. More specifically, using the rapid and comprehensive assessments, each country has been provided with an inclusive list of key gaps and a set of strategic actions to address them.

Several factors have contributed to the achievements made over the last five years. The wisdom of Member States in endorsing the five key priorities has resulted in a practical agenda for the joint work between WHO and Member States, which included a focus on strengthening HIS as an integral part of the strategic directions set for each of the five regional priorities. Within WHO, the leadership and commitment of all departments and the active engagement across the three levels of the Organization was a prerequisite for successful initiatives. An effective approach to improve HIS and civil registration and vital statistics will not be possible without the ownership and joint work with other key sectors within countries and other stakeholders and UN agencies at the country and regional levels. The participation of representatives of ministries of interior and national statistics offices has been key in designing the framework and development of rational and feasible multisectoral strategic actions.

Further and sustained progress is dependent on the commitment and leadership of the relevant sectors at the national level. Monitoring progress in implementing the framework and the recommendations for improving civil registration and vital statistics will motivate national action and is essential in guiding effective support from WHO and its partners.

Functional HIS will require sustained investments in strengthening national capacities in epidemiology, statistics, data collection, compilation, quality assessment, validation and adjustment, as well as analysis and review. Training is therefore a priority in all countries, irrespective of their healthcare status or socioeconomic development.

Finally, implementing the health information framework, the regional strategy for the improvement of civil registration and vital statistics system, and the WHO recommendations to countries in the Region –particularly low- and middle-income countries – will require adequate resources, both technical and financial. Global and regional support will be needed. Countries should be able to receive support from existing global initiatives (28,29) and efforts that provide a platform for collective and aligned action and the monitoring of progress towards the SDGs.

References

1. Health in 2015: from MDGs, Millennium Development Goals to SDGs, Sustainable Development Goals. Geneva: World Health Organization; 2015 (http://apps.who.int/iris/bitstream/10665/200009/1/9789241565110_eng.pdf?ua=1, accessed November 2016) .
2. Mikkelsen, L., et al. A global assessment of civil registration and vital statistics systems: monitoring data quality and progress. The Lancet. Vol 386(10001), October 2015.
3. Towards 2030: counting and accountability matter. The Lancet. Vol 386(1312), October 2015.
4. World health statistics 2016: monitoring health for the SDGs, sustainable development goals. Geneva: World Health Organization; 2016 (http://apps.who.int/iris/bitstream/10665/206498/1/9789241565264_eng.pdf?ua=1, accessed November 2016).
5. Civil registration and vital statistics [website]. Cairo: WHO Regional Office for the Eastern Mediterranean; 2016 (http://www.emro.who.int/civil-registration-statistics/assesment/crvs-assessments.html , accessed November 2016).
6. Regional strategy for the improvement of civil registration and vital statistics system 2014 – 2019. Cairo: WHO Regional Office for the Eastern Mediterranean; 2014 (http://applications.emro.who.int/dsaf/EMROPUB_2014_EN_1754.pdf?ua=1, accessed November 2016).
7. Regional strategy for the improvement of civil registration and vital statistics systems. Cairo: WHO Regional Office for the Eastern Mediterranean; 2013 (EM/RC60/10; http://applications.emro.who.int/docs/RC_Techn_paper_2013_10_15084_EN.pdf?ua=1, accessed November 2016).
8. Health in 2015: from MDGs, Millennium Development Goals to SDGs, Sustainable Development Goals. WHO, 2015.
9. Reinforcing health information systems. Cairo: WHO Regional Office for the Eastern Mediterranean; 2014 (EM/RC61/7; http://applications.emro.who.int/docs/RC_technical_papers_2014_7_15463_EN.pdf?ua=1, accessed November 2016).
10. WHO Regional Committee for the Eastern Mediterranean Region resolution EM/RC60/R.7 on a regional strategy for the improvement of civil registration and vital statistics systems 2014–2019. Cairo: WHO Regional Office for the Eastern Mediterranean; 2014 (http://applications.emro.who.int/docs/RC60_Resolutions_2013_R7_15140_EN.pdf?ua=1 , accessed November 2016).
11. Health systems strengthening in countries of the Eastern Mediterranean Region: challenges, priorities and options for future action. Cairo: WHO Regional Office for the Eastern Mediterranean; 2012 (EM/RC59/Tech.Disc.1; http://applications.emro.who.int/docs/RC_technical_papers_2012_Tech_Disc_1_14613_EN.pdf?ua=1, accessed November 2016).
12. WHO methods and data sources for country-level causes of death 2000-2012. Geneva: World Health Organization; 2014 (WHO/HIS/HIS/GHE/2014.7; http://www.who.int/healthinfo/global_burden_disease/GlobalCOD_method_2000_2012.pdf, accessed November 2016).
13. Progress report on the regional strategy for the improvement of civil registration and vital statistics systems 2014–2019. Cairo: WHO Regional Office for the Eastern Mediterranean; 2015 (EM/RC62/INF.DOC 8 Rev.2; http://applications.emro.who.int/docs/RC_technical_papers_2015_inf_doc_8_16497_EN.pdf?ua=1&ua=1, accessed November 2016).
14. Assessing national capacity for the prevention and control of noncommunicable diseases: report of the 2015 global survey. Geneva: World Health Organization; 2016 (http://apps.who.int/iris/bitstream/10665/246223/1/9789241565363-eng.pdf, accessed November 2016).
15. Toolkit on monitoring health systems strengthening: health information systems. Geneva: Wolrd Health Organization; 2008 (http://www.who.int/healthinfo/statistics/toolkit_hss/EN_PDF_Toolkit_HSS_InformationSystems.pdf , accessed November 2016).
16. The need for strong health information systems: rationale for the HMN framework. Geneva: World Health Organization (Health Metrics Network; http://apps.who.int/healthmetrics/documents/Rational_HIS_Capacity.pdf?ua=1, accessed November 2016).
17. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies. Geneva: World Health Organization; 2010 (http://www.who.int/healthinfo/systems/WHO_MBHSS_2010_full_web.pdf?ua=1, accessed November 2016).
18. Framework and standards for country health information systems. Geneva: World Health Organization; 2008 (Health Metrics Network; http://apps.who.int/healthmetrics/documents/hmn_framework200803.pdf?ua=1, accessed November 2016).
19. Global Reference List of 100 Core Health Indicators. Geneva: World Health Organization; 2015 (WHO/HIS/HSI/2015.3; http://apps.who.int/iris/bitstream/10665/173589/1/WHO_HIS_HSI_2015.3_eng.pdf?ua=1, accessed November 2016).
20. Eastern Mediterranean Region Indicator and Metadata Registry [website – registry]. Cairo: WHO Regional Office for the Eastern Mediterranean; 2016 (http://gis.emro.who.int/EMRIMR/ , accessed November 2016).
21. Eastern Mediterranean Region Framework for health information systems and core indicators for monitoring health situation and health system performance 2016. Cairo: WHO Regional Office for the Eastern Mediterranean; 2016 (http://applications.emro.who.int/dsaf/EMROPUB_2016_EN_19169.pdf?ua=1, accessed November 2016).
22. WHO Regional Committee for the Eastern Mediterranean resolution EM/RC61/R.1 on the annual report of the Reginal Director for 2013. Cairo: WHO Regional Office for the Eastern Mediterranean; 2014 (http://applications.emro.who.int/docs/RC61_Resolutions_2014_R1_15553_EN.pdf?ua=1, accessed November 2016).
23. Principles and recommendations for a vital statistics system (Revision 2). New York: United Nations; 2001 (http://unstats.un.org/unsd/publication/SeriesM/SeriesM_19rev2E.pdf, accessed November 2016).
24. Gouda et al. Health information priorities for more effective implementation and monitoring of noncommunicable disease programs in low- and middle-income countries: lessons from the Pacific. BMC Medicine (2015) 13:233.
25. Rapid assessment of national civil registration and vital statistics systems. Geneva: World Health Organization; 2010 (WHO/IER/HSI/STM/2010.1; http://apps.who.int/iris/bitstream/10665/70470/1/WHO_IER_HSI_STM_2010.1_eng.pdf, accessed November 2016).
26. Improving the quality and use of birth, death and cause-of-death information: guidance for a standards-based review of country practices. Geneva: World Health Organization; 2010 (http://www.who.int/healthinfo/tool_cod_2010.pdf?ua=1, accessed November 2016).
27. Civil registration and vital statistics - CRVS rapid assessments [website]. Cairo: WHO Regional Office for the Eastern Mediterranean; 2016 (http://www.emro.who.int/civil-registration-statistics/assesment/crvs-rapid-assessments.html, accessed November 2016).
28. More than numbers: how better data is changing health systems [website]. Geneva: World Health Organization; 2016 (http://www.who.int/features/2016/health-data-collaborative/en/, accessed November 2016).
29. Health data collaborative [website]. (https://www.healthdatacollaborative.org/, accessed November 2016).