Health needs and eHealth readiness assessment of health care organizations in Kabul and Bamyan, Afghanistan

PDF version

Research article

H. Durrani,¹ S. Khoja¹,² A. Naseem,³ R.E. Scott,⁴ A. Gul⁵ and R. Jan⁶

تقييم الاحتياجات الصحية والاستعداد للصحة للإلكترونية في مؤسسات الرعاية الصحية في كابول وباميان في أفغانستان

حماد دُرَّاني، شارق خوجا، عذرا نسيم، رتشرد أ سكوت، عادل غُلْ، جان رفت

الخلاصة: قيَّمَت هذه الدراسة مدى احتياج واستعداد مؤسسات الرعاية الصحية في كابول وباميان في أفغانستان لتطبيق تقنيات المعلومات والاتصالات في الرعاية الصحية (الصحة الإلكترونية). وقد تم اعتماد تصميم مؤلّف من مزيج من الطرق في مؤسستين ضمن شبكة أغاخان للتنمية في أفغانستان، وهما المعهد الطبي الفرنسي للأطفال في كابول ومستشفى المقاطعة في باميان. وقد جمع الباحثون المعلومات اللازمة لتقييم الاحتياجات من المقابَلات ومن مجموعات التركيز البؤرية، ثم قاموا بتقييم الاستعداد لتطبيق لصحة الإلكترونية باستخدام وسيلة مصدوقة للمسح. وقد صنَّف الباحثون احتياجات المؤسسات في شبكة أغاخان للتنمية في أفغانستان كما يلي: احتياجات تقديم الرعاية؛ واحتياجات التعلُّم؛ واحتياجات إدارة المعلومات. وكان معدَّل الاستعداد للصحة الإلكترونية أقل في باميان منه في كابول في جميع مجالات تقييم الاستعداد. وسوف يكون من الممكن لمؤسسات أخرى في أفغانستان أن تستفيد من تطبيق نموذج تقييم الاحتياجات والاستعداد الذي استخدمه الباحثون في مؤسسات شبكة أغاخان للتنمية.

ABSTRACT This study assessed the need and readiness of health care institutions in Kabul and Bamyan, Afghanistan for successful implementation of information and communication technology in health care (eHealth). A mixed methods design was adopted at 2 institutions in the Aga Khan Development Network in Afghanistan: the French Medical Institute for Children in Kabul and Bamyan Provincial Hospital, Bamyan. Information for the needs assessment was obtained from interviews and focus groups and eHealth readiness was assessed using a validated survey tool. The needs of institutions in the Aga Khan Development Network in Afghanistan were categorized as follows: provision of care needs; learning needs; and information management needs. eHealth readiness on average was lower in Bamyan compared with Kabul in all areas of the readiness assessment. Other institutions in Afghanistan may benefit from adopting the model of needs and readiness assessment used for Aga Khan Development Network institutions.

Évaluation des besoins en matière de santé et de la préparation à la cybersanté dans des établissements de soins de santé à Kaboul et Bamyan (Afghanistan)

RÉSUMÉ La présente étude a évalué le besoin et la préparation d'instituts de soins de santé à Kaboul et Bamyan (Afghanistan) pour une mise en œuvre réussie de la technologie de l'information et de la communication en matière de santé (cybersanté). Un modèle méthodique mixte a été adopté dans deux institutions du Réseau Aga Khan de développement en Afghanistan : l'Institut médical français pour l'enfant de Kaboul et l'hôpital provincial de Bamyan. L'évaluation des besoins a été réalisée à partir d'entrevues et de groupes thématiques et la préparation à la cybersanté a été évaluée à l'aide d'un outil d'enquête validé. Les besoins des institutions du Réseau Aga Khan de développement en Afghanistan ont été classés comme suit : besoins en fourniture de soins, besoins en connaissances, et besoins en gestion des informations. La préparation à la cybersanté en moyenne était plus faible à Bamyan qu'à Kaboul dans tous les domaines de la préparation. D'autres institutions en Afghanistan pourraient tirer avantage de l'adoption du modèle d'évaluation des besoins et de la préparation utilisé pour les institutions du Réseau Aga Khan de développement.

¹Department of Community Health Sciences; ³Open Learning, Institute for Educational Development; ⁵Information Technology Division; 6School of Nursing, The Aga Khan University, Karachi, Pakistan (Correspondence to H. Durrani: This email address is being protected from spambots. You need JavaScript enabled to view it.)

²Health Telematics Unit, Department of Community Health Sciences; ⁴Global eHealth Research and Training Programme, Health Innovation and Information Technology Centre, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada.

Received: 20/07/10; accepted: 16/03/11

EMHJ, 2012, 18(6): 663-670

Introduction

eHealth is the cost-effective and secure use of information and communications technology (ICT) in support of health and health-related fields, including health care services, health surveillance, health literature and health education, knowledge and research [1,2]. ICTs are currently being used in developed [3] and developing countries [4–6] and have been used to improve access to sources of knowledge for both patients and health care providers. However, the undersized health and ICT sector in a country such as Afghanistan limits its potential and wider benefits [7].

Afghanistan is a developing country with a huge diversity of languages and cultures. Long lasting wars, insecurity and difficult terrains have made Afghanistan one of the least developed countries in the world [8,9]. Most of the Afghan population does not have access to even basic health services. The most important constraint to improving health status is the lack of physical infrastructure in the country [10]. In Kabul, around half of the population has to travel less than 5 km and more than one-fifth of the population has to travel more than 10 km to reach their closest health centre [11]. In Bamyan province access to health care is more difficult for many people, with nearly three-quarters of the population having to travel over 10 km to get medical attention: 70% for access to health centres and 72% for dispensaries [12].

Three decades of continuous war has left the land-based communications networks of Afghanistan shattered. The Afghanistan telecommunications sector started its development activities in 2002, when the telephone penetration was less than 0.05% and telecoms infrastructures were much damaged. Currently Afghanistan has more than 12.5 million telephone subscribers (50% phone-density). Investment in the telecommunications sector, as per the reports provided at the end of the last quarter of 2009, exceeds US$ 1.3 billion, and about 80% of the country’s population has access to telecommunications services. There are 5 licensed mobile companies trying to increase coverage of mobile telephony in different provinces of Afghanistan, and around 75% of the population has mobile coverage, mostly GSM [global system for mobile communication], aside from Kabul and other major cities where there is GPRS [general packet radio system] facilities. Internet services are available, although mostly through satellite, which makes the service very costly. There are 23 small- and medium-sized Internet service provider licensees in the country, serving about 1 million internet users throughout the country. Efforts are also being made to lay down an extensive fibre optic skeleton throughout Afghanistan by the Ministry of Telecom [11–14]. Some ICT parameters of Afghanistan are shown in Table 1.

eHealth for Afghanistan

There is a severe shortage of health facilities and health human resources to provide even basic services to the population in Afghanistan. Several agencies and nongovernmental organizations are working to provide health care facilities to the population. Aga Khan Development Network (AKDN) is one of the few agencies actively working in Afghanistan providing health services at all levels of care through a group of development agencies [15] (Aga Khan Health Services, Aga Khan University, French Medical Institute for Children and the Aga Khan Foundation) working in close collaboration with Afghanistan’s Ministry of Public Health.

AKDN recognizes the role eHealth can play in bringing different institutions and providers in the network together, providing coordinated care to the population, ensuring a continuum of care at all levels, and minimizing the barriers of distance and time. These benefits inspire the network to explore the potential benefits of eHealth for the institutions and agencies working for it. However, the diffusion and adoption of eHealth in Afghanistan requires health care institutions to identify their needs and readiness, in order to prepare individuals and organizations for any organizational change. This process involves in-depth needs and readiness assessment, so that technologically appropriate and culturally sensitive eHealth solutions can be identified, aligned and prioritized in a manner that maximizes the efficiency and effectiveness of investment in any given setting.

Methods

The study was designed to identify and prioritize the needs of AKDN health care institutions working in Afghanistan, ascertain eHealth solutions to address those needs and also assess the eHealth readiness of these institutions in implementing such initiatives.

A research team was developed at AKDN to carry out this study with expertise in areas related to the study such as needs and eHealth readiness assessment, eHealth policy, nursing services and eLearning. The study was conducted between September 2007 and August 2008. Site visits to AKDN hospitals and health centres in Kabul and Bamyan were conducted by the research team to perform a detailed needs and readiness assessment. The following steps were taken.

Core stakeholder team. A team of key stakeholders comprising of health care providers, managers, and information technology staff was established at the French Medical Institute for Children, Kabul. The team worked closely with the AKDN eHealth team to monitor and make decisions on the process of needs and readiness assessment in the country.

Approach. A mixed methods approach [16] was used, including a case study (qualitative) for needs assessment and a survey for readiness assessment (quantitative).

Health needs assessment. The needs analysis involved focus group discussions and key informant interviews with stakeholders from all potential user-groups of the hospital (IT, hospital and nursing administration and services) within the selected AKDN institutions and partner agencies. Purposive sampling was done to identify these participants and data collection finished after reaching saturation. The key informant interviewees included officials of the Afghanistan Ministry of Public Health to include the views of the government. Separate guidelines for both the interview and focus groups were developed. The interviews/discussions were tape-recorded where permission was granted; notes were also taken in all interviews and focus groups.

Conducting readiness assessment of network institutions. For readiness assessment the interviewees, focus group participants and other health care providers and managers were asked to complete the validated tools for institutional eHealth readiness assessment [17–19]. Participants rated the levels of readiness of their institutions on a scale of 1–5 (1 being the minimum and 5 being the maximum) in 5 areas as follows;

Core readiness: questions related to the overall planning process for the proposed eHealth programme, along with assessing the knowledge and experience of planners with programmes using ICT.

Technological readiness: questions related to the availability and affordability of required ICT along with the hardware and software needed to implement the proposed programme. This category was included in the tool for managers only.

Learning readiness: questions related to the existence of programmes and resources to provide training to health care providers in using the technology suggested in the proposed eHealth programme. This category was included in the tool for health care providers only.

Societal readiness (ICT use and interaction): the existing interaction of the concerned institution with other health care institutions in the region and beyond.

Policy readiness (at institutional and government levels): the existence of policies at the government and institutional levels to deal with common issues such as licensing, liability and reimbursement.

For the needs analysis 10 key informant interviews and 8 focus group discussions were conducted (Table 2). The interview and focus group data were analysed by 2 research team members independently who coded and analysed the data [20]. The codes were clustered into broad themes, which led to the identification of the needs of the health care institutions in Afghanistan that can be addressed through eHealth.

eHealth readiness was studied in depth at 2 institutions: the French Medical Institute for Children and the Bamyan Provincial Hospital. Both hospitals are managed by agencies of Aga Khan Development Network, i.e. Aga Khan University and Aga Khan Health Services Afghanistan, respectively. These institutions were selected to initiate the eHealth programme in Afghanistan based on their cooperation and willingness to implement eHealth at their respective hospitals. The French Medical Institute for Children in Kabul is an 85-bed state-of-the-science tertiary care facility, providing services in paediatric inpatient and paediatric/adult outpatient services, i.e. in medical, surgical, cardiology and orthopaedics. Bamyan Provincial Hospital is the main secondary referral facility in Bamyan province and delivers health care to approximately 670 000 people in the province. Since Aga Khan Health Services, Afghanistan took over the hospital in 2004, the facility provides a range of health services such as acute hospital services, surgery, internal medicine, maternal and child services, dental, ophthalmology and diagnostic services.

Results

Key findings of health needs assessment

Based upon the information obtained through interviews and focus groups, the needs of the AKDN institutions in Kabul and Bamyan, Afghanistan were categorized as:

Needs in provision of care;

Learning needs;

Needs in information management.

Needs in provision of care

Participants identified several gaps where new ideas and technologies could help in providing better quality and timely care to the population. A number of key findings emerged:

Shortage of health human resources. The shortage of qualified physicians and nurses was a major problem in both Kabul and Bamyan, due to years of conflict and the poor security situation in the country. Most of the specialists working in the hospitals did not possess postgraduate degrees. Many of them also lacked experience working in peacetime conditions, and dealing with cases other than trauma. There was a severe shortage of nurses and midwives especially in Bamyan. Lack of quality institutions to train doctors, nurses and other health professionals also magnified the situation.

Difficulties in referral systems. Tough mountainous terrains and lack of transport systems made it extremely hard for patients to be transferred from Bamyan to cities. Participants gave emphasis to improving the capacity of the local health care providers and bring services closer to the communities as the only solution to reduce the burden of illness.

Government policies. Participants also criticized government policies for not allowing health care providers in rural areas to provide more than the approved list of services or dispensing more than the official list of drugs. This policy greatly limited the number of services these health professionals could provide, giving them no other choice than to refer the patients to next level of care.

Issues with service utilization. Some issues were also identified that may hinder the ability of population to utilize the services. The security situation in Kabul and lack of electricity at Bamyan were major barriers to providing medical services 24 hours a day, 7 days a week. Another issue was the traditional beliefs of people which led to them use home remedies and untrained health care providers before patients are brought to hospital. This can delay patients’ care to such an extent that it is difficult to treat them successfully.

There was also uncertainty about policies and procedures regarding consultations, referrals (between institutions), education, communication and information and knowledge transfer, and there was a perceived general lack of enabling strategies and policies for the adoption of eHealth solutions.

Learning needs

Participants identified training of new and existing health human resources to enhance the competence and credentials of health care providers as the most crucial and urgent need for improving health services in Kabul and Bamyan, Afghanistan. The following findings emerged:

Lack of continuing education programmes. Participants at Bamyan talked about the clear need for short and long-term courses for doctors, nursing and allied professions to enhance their learning on a regular basis.

Lack of access to current information and research. Another problem related to the learning needs of the providers was the lack of access to policies, procedures, guidelines and research databases. This was seen to lead to professional isolation of health care providers and to limit the capacity of managers to introduce best practice guidelines based on current research. Lack of access to research databases also limited the research capability among health care providers. There was limited access to computers at hospitals for staff to enhance their knowledge and communication. A need to improve the low level of ICT literacy among nursing and allied health staff, especially in Bamyan, was highlighted. Participants also identified the need for simple and comprehensive access to current specialized literature, such as guidelines and policies regarding best practices.

Needs in information management

Better management of information in hospitals and communities in Kabul and Bamyan was emphasized as a key to better planning of health care resources and planning for the future. Key findings were:

Paper-based medical record systems. Several participants from institutions in both Kabul and Bamyan pointed out the need for proper medical records to provide quality services and facilitate referral of patients. They said that most records were still in paper format, which lead to missing information, and also caused delays in access to records at times.

Communication between institutions and providers for information sharing. There was a need for timely access to patient’s information through appropriate medical record systems. Participants also identified a need for improved materials management and to place strategies or policies to permit seamless, interjurisdictional sharing of health information within and between institutions and different countries.

Key findings of eHealth readiness

The study used eHealth readiness assessment tools to collect information on the areas that needed more attention during the planning for eHealth programmes. Readiness assessment was conducted at the French Medical Institute for Children in Kabul and Bamyan Provincial Hospital, Bamyan . A total of 17 health care providers and 6 managers provided information on eHealth readiness. The demographic characteristics of these participants and their involvement in eHealth are shown in Table 3. Overall 65% of them had been involved in eHealth planning and 23% in implementing eHealth.

The results suggested that the health care providers at the 2 institutions rated their overall eHealth readiness nearly equally. However, managers at the Kabul hospital rated their readiness higher than the managers at Bamyan Provincial Hospital. Table 4 provides a summary of the results for the eHealth readiness categories described earlier (a higher score indicates greater readiness, to a maximum of 5).

Core readiness

Health care providers and managers at the Kabul hospital generally rated their core-readiness as high. Both the groups expressed the need to be involved in prioritization of eHealth related needs. Health care providers at the Bamyan hospital showed low levels of comfort with the use of technology. They also rated their involvement in the planning process as low, and wanted to be involved in prioritizing the needs for eHealth. Managers in Bamyan also rated their overall core-readiness as low, especially the process of needs identification, awareness and comfort with the technology

Technological readiness

Managers at the Kabul hospital rated technological readiness as high, except that they saw a need for improving hardware and software at the hospital. Managers in the provincial hospital in Bamyan, on the other hand, graded their technological readiness as low, and emphasized a need for improving the quality of Internet, availability and affordability of desired technologies and institutional access to ICT training.

Learning readiness

Health care providers at both institutions emphasized the need for using ICT to train health care providers. Health care providers in Bamyan highlighted the need for ICT related training and their involvement in the planning and implementation of eHealth programmes

Societal readiness

Health care providers and managers at the Kabul hospital rated their societal readiness as generally high, but identified a need for improving communication with other institutions. Both providers and managers at Bamyan Provincial Hospital rated their societal readiness as low, noting the need for communication with other organizations and health care providers for coordinated patient care. They also noted the need to consider sociocultural factors among staff and clients.

Policy readiness

Health care providers and managers at the Kabul hospital generally rated their institutional policy readiness as high, but felt a need to improve such readiness among politicians. In contrast, managers at the Bamyan hospital showed a need for better policies for licensure, liability and reimbursement when providing care through eHealth. Managers in Bamyan also showed a need for creating awareness among politicians, policy-makers and health care providers at the institutional level regarding eHealth, to acquire more support for eHealth programmes.

Discussion

The results of this survey explain the eHealth needs of 2 different health care institutions working in Kabul and Bamyan respectively under the AKDN. The results also give us a picture and degree of readiness of these institutions regarding eHealth implementation.

The eHealth needs assessment identified needs in 3 categories: care provision, learning and information management. In all these categories 2 main themes emerged prominently, first the lack of capacities (both in technical and infrastructural) and secondly lack of awareness and policies regarding eHealth. The institutions called for specialized human resources and to put in place structures to improve technical proficiencies in eHealth that would lead to a more sustainable implementation of eHealth. Though the study’s main focus was on AKDN institutions, considering the overall condition of Afghanistan, the results can be generalized to other parts of the country especially the rural parts of Afghanistan where conditions and capacities are more or less the same.

These findings are further corroborated by the results of the readiness assessment, which was considerably lower in Bamyan Provincial Hospital than in the French Medical Institute for Children. The difference of readiness levels between the 2 hospitals highlights the contrast in availability and use of technology in different provinces of Afghanistan, i.e. Kabul versus Bamyan. The readiness level of Bamyan health care managers was low in all categories, with policy and societal readiness at the lowest. In a broader perspective this indicates not only a danger of widening the digital divide in different provinces of Afghanistan, but connects to other issues, such as lack and retention of specialized health human resources, continuous capacity problems, lack of infrastructure, plus uncertain and undefined institutional and organizational policies.

The survey identified that these gaps could be addressed, however, through giving more importance to the areas of low score and designing culturally acceptable and technologically sustainable eHealth solutions. Linking eHealth planning and implementation to defining needs and eHealth readiness might first appear to be a longer path. On the contrary, such an approach can flag up problems and minimize challenges and yield more positive results, leading to a much smoother processes. An appropriate plan cannot be designed without accurate knowledge of what exists on the ground, i.e. the actual needs and skills of an organization to provide health care to their population.

In the final way forward, the drive to transform developing countries into knowledge-based societies will necessitate intergovernmental as well as private sector cooperation. AKDN as a private network is using the results of the study and have initiated few eHealth activities in collaboration with other public and private entities in Afghanistan. Recently, the teleradiology project between French Medical Institute for Children, Kabul and Aga Khan University Hospital, Karachi, Pakistan is one example where private and government bodies have come together successfully to provide diagnostic services for computerized tomography scanning studies and medical education. To establish a sustainable model, various organizations contributed technical, financial and logistic support. These organizations include the telecommunications company Roshan, the Afghanistan Ministry of Public Health and the networking specialist Cisco [10]. The project has recently been extended to Bamyan province, where the provincial hospital of Bamyan is connected to the French Medical Institute for Children in Kabul for exchanging advice on simple X-rays along with sessions on continued professional development of health care providers (doctors, nurses and other hospital support staff). According to one of the reports, more than 340 patients have benefited from this telehealth model and more than 231 Afghan medical personnel have participated in diagnostic and training opportunities facilitated by the technology between Aga Khan University Hospital, Karachi; French Medical Institute for Children, Kabul; and Bamyan Provincial Hospital, Bamyan [16].

Most of the other institutions in Kabul and Bamyan are trying to learn from and follow the eHealth model adopted by AKDN. Better understanding of the needs and readiness of these institutions will enable comparison with other institutions and modification of their eHealth programmes accordingly.

Conclusion

This study has shown that any eHealth programme in Kabul and Bamyan, or indeed other parts of Afghanistan, must consider and address the issues above-mentioned before embarking on a technological solution. Rushing into projects without an assessment of the range of needs and the priorities is a costly experiment, which poor countries cannot afford. Where resources are scarce, priorities have to be focused, and linkages have to be maximized to lay a solid base for future development of any organization. Institutions first need to analyse where health care providers and other users stand in respect to readiness levels, followed by defining their needs in terms of readiness outputs, so that the final implementation accurately reflects the ground reality of the institution. This model also allows eHealth to broaden the vision of the institutions and organization as a whole, supporting a relatively smooth eHealth diffusion and adoption process. The study has also shown that as we strive to use eHealth to provide larger benefit to institutions across the globe, each institution must adopt a “network” mindset as they address issues such as readiness, change management, health human resources and selection of different technology options. To address this array of issues, health care organizations need to develop, individually and collectively, proper protocols, policies and legislations to support networked eHealth implementation and application.

Acknowledgements

We acknowledge the Aga Khan Foundation, Geneva, for funding, and AKDN and public institutions in Afghanistan for their support. We also appreciate Ms Syeda Suha Tirmizi for her help in drafting and finalizing the manuscript.

References

1. Fifty-eighth World Health Assembly, Geneva, 16–25 May 2005. Resolutions and decisions. Annex. Geneva, World Health Organization, 2005 (WHA58/2005/REC/1) (http://apps.who.int/gb/ebwha/pdf_files/WHA58-REC1/english/A58_2005_REC1-en.pdf, accessed 20 April 2012).
2. Al-Shorbaji N. E-health in the Eastern Mediterranean Region: a decade of challenges and achievements. Eastern Mediterranean Health Journal, 2008, 14(Suppl.):S157–S173.
3. West SP et al. Goal setting using telemedicine in rural underserved older adults with diabetes: experiences from the informatics for diabetes education and telemedicine project. Telemedicine Journal and e-Health, 2010, 16:405–416.
4. Kommalage M. Use of the internet by patients attending specialist clinics in Sri Lanka: a cross sectional study. BMC Medical Informatics and Decision Making, 2009, 9:12.
5. Durrani H, Khoja S. A systematic review of the use of telehealth in Asian countries. Journal of Telemedicine and Telecare, 2009, 15:175–181.
6. Kirigia JM et al. E-health: determinants, opportunities, challenges and the way forward for countries in the WHO African Region. BMC Public Health, 2005, 5:137.
7. Chandrasekhar CP, Ghosh J. ICT and health in low-income countries: the potential and constraints. Bulletin of the World Health Organization, 2001, 79:850–855.
8. Sabri B et al. Towards sustainable delivery of health services in Afghanistan: options for the future, Bulletin of the World Health Organization, 2007, 85(9):712–718.
9. Newbrander W, Yoder R, Debevoise AB. Rebuilding health systems in post-conflict countries: estimating the costs of basic services. International Journal of Health Planning and Management, 2007, 22:319–336.
10. Health systems profile Afghanistan. Cairo, World Health Organization Regional Office for the Eastern Mediterranean, 2006.
11. Kabul provincial profile. World Food Programme [online factsheet] (http://foodsecurityatlas.org/afg/country/provincial-Profile/Kabul, accessed 20 April 2012).
12. Bamyan provincial profile. World Food Programme [online factsheet] http://foodsecurityatlas.org/afg/country/provincial-Profile/Bamyan, accessed 20 April 2012).
13. A glance at the telecom developments in Afghanistan. Telecommunications Regulatory Authority Afghanistan, 3 May 2010 [press release] (http://www.atra.gov.af/en/aglance_telecom_dev.html, accessed 20 April 2012).
14. Roshan announces expansion of Afghanistan’s first telemedicine project to Bamyan region. Aga Khan Development Network, 31 May 2009 [press release (http://www.akdn.org/Content/758/Roshan-Announces-Expansion-of-Afghanistans-First%20Telemedicine-Project-to-Bamyan-Region, accessed 20 April 2012).
15. Kashif MM et al. Teleradiology between Afghanistan and Pakistan: one year experience. Pakistan Journal of Radiology, 2008, 18:22–25.
16. Creswell JW, Fetters MD, Ivankova NV. Designing a mixed methods study in primary care. Annals of Family Medicine, 2004, 2:7–12.
17. Khoja S et al. e-Health readiness assessment tools for healthcare institutions in developing countries. Telemedicine Journal and e-Health, 2007, 13:425–431.
18. Khoja S et al. Reliability testing of eHealth readiness assessment tools. eHealth International Journal, 2007, 3(1).
19. Khoja S et al. Validating eHealth readiness assessment tools by using qualitative research methods. eHealth International Journal, 2007, 3.
20. Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health Services Research, 2007, 42:1758–1772.