Short research communication
Naeem Hassan Saleem1, Muhammad Naveed Asghar2, Hassanali Dalvi Shirazi2 and Sohail Chand3
1World Health Organization, Lahore, Pakistan (Correspondence to Naeem Saleem:
Abstract
Background: Health systems research and publication are vital for improving healthcare at all levels of care. They provide evidence for policy and for better service outcomes.
Aims: To assess published health systems research in Pakistan from 2011 to 2020 and to model and forecast the publication trend.
Methods: This cross-sectional study searched health systems research publications database for Pakistan in Scopus using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Tables and graphs were created using Microsoft Excel, visualization was produced on VoS Viewer, and SPSS version 29.1.0 was used for analysis, while R software was used to plot the time series data.
Results: A total of 697 articles with an average of 16.6 citations were published between 2011 and 2020. The highest number of publications (240) per single institution was from Aga Khan University (including Aga Khan University Hospital), Karachi. There was a significant difference between the number of publications before and after the midpoint (2015) of the bibliometric analysis (t = -3.08, P = 0.015, 95% CI -87.78–-12.61). We observed a strong relationship between publications and citations over the same period (Correlation coefficient 0.809, P = 0.002, CI 0.46–0.98) but there was no significant difference between the number of citations before and after the midpoint.
Conclusion: There was an acute dearth of health systems research publication at the beginning of the study period. A few medical institutes are now taking the lead in conducting and publishing health systems research. Technical and financial support is needed to strengthen the capacity of Pakistani medical institutions and researchers to contribute more to knowledge generation within the country.
Keywords: health systems, healthcare, research, publication, Pakistan
Citation: Saleem NH, Asghar MN, Shirazi HD, Chand S. A bibliometric analysis of health systems research in Pakistan. East Mediterr Health J. 2024;30(6):430–439. https://doi.org/10.26719/2024.30.6.430.
Received: 09/03/23; Accepted: 12/05/24
Copyright: © Authors 2024; Licensee: World Health Organization. EMHJ is an open access journal. All papers published in EMHJ are available under the Creative Commons Attribution Non-Commercial ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo).
Introduction
Health systems research and publication are critical for enhancing the effectiveness of health care delivery. They help generate evidence and new knowledge for policy and for improving service outcomes. However, there has been a gap in health systems research and publication in Pakistan, making new knowledge inaccessible to scholars, researchers and policymakers (1,2).
Health system comprises all the organizations, institutions and resources contributing to actions to improve healthcare (3). Health systems research helps gain understanding of the situation in order to plan for improvements (4). A bibliometric analysis will help quantify published research over a certain period with the aim of charting a forward path for the research agenda. Bibliometric research is a study that quantifies physical published or bibliographic units (5). Pirchard coined the term "bibliometrics" in 1969, and it gained recognition in the 1980s by providing a new perspective to published research (6).
In recent years, bibliometric studies have been used to broadly assess research and publication progress in different fields of interest (7). However, in Pakistan, bibliometric research in the health sciences has been limited to local journals. An analysis of local journals in the field of health sciences provides information about 749 original research articles, review articles and case reports from 2011 to 2015 (8), but it provides data only about the volume of publication, number of citations and authorship patterns. Other researchers extended their work by collecting data from various national journals, with one analysis presenting data from 6 e-journals from 2006 to 2015 (9). The uniqueness of this research lies in providing results as gender-segregated data of authors and collaboration among authors (10). Haq et al evaluated the publication capacity of researchers in Pakistan from 2001 to 2020, extending work from conventional bibliometric indicators to visualization output (11).
Earlier bibliometric analyses in Pakistan mainly provided time-bound, journal-specific and subject-based information (9,10,12-14). Our study is noteworthy for incorporating numerous time-bound, topic-specific elements and publications forecasting for a specified period. It adds to the existing body of knowledge by quantifying health systems research publications and would be very useful in strengthening health research for improving healthcare systems in Pakistan.
Objectives of the study
This study assessed the differences in health systems research and publication in Pakistan from 2011 to 2020, it modelled and forecasted publication trends using data gathered from the assessment. We hypothesized no difference between the number of publications before and after the mid-point between 2011 and 2020. Alternatively, we hypothesized that there was a difference between the two periods.
Methods
Study design and eligibility criteria
This cross-sectional study was conducted in March 2022 to analyse secondary data obtained from Scopus from 1 January 2011 to 31 December 2020. It included peer-reviewed articles, empirical researches and review articles on health systems in Pakistan (15). Any publication by a Pakistani author as a first or lead author or co-author was included. Foreign authors who published their work about Pakistan with or without Pakistani authors were also included. Key terms to build the relevant queries were ascertained from PubMed database and included medical subject headings (MeSH). Syntax of keywords containing “AND”, or the specific keywords used independently, or with alternative terms, were exhausted for this purpose and all potential variants were included by applying the asterisk.
Data and analysis
We searched the Scopus electronic databases as a primary source to retrieve the publication data. The data were examined for duplication and compatibility with the eligibility criteria. In the forefront, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) provided an outline of a process for the identification, screening, eligibility, and final inclusion of studies retrieved from the database for final addition in the bibliometric analysis (Figure 1). Tables and graphs were created using Microsoft Excel, visualization was produced on VoS Viewer, and SPSS version 29.1.0 was used to analyse the data, while R software was used to plot the time series data.
Main outcomes
As our variable of interest was discrete, comprising the number of publications and number of citations, we applied the independent samples t-test as 2 groups were unrelated (to compare the means of the publications before and after the midpoint). We applied Pearson’s correlation to the publications and citations. The year was an independent variable and the number of publications was a dependent variable for the forecasting modelling.
Ethical consideration
There was no need for ethical approval because we used publicly available published research and secondary data. The study adhered to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cross-sectional studies.
Results
A total of 697 records met the inclusion criteria, out of which 605 (86.8%) were original studies and 92 (13.2%) were reviews. All the 697 publications were cited 11 602 times, with an average of 16.6 citations per document. Beginning from 2011, there were 8, 2, 6, 13, 4, 9, 12, 10, 9, and 19 published reviews for each year from 2011 to 2020, while there were 43, 35, 37, 48, 60, 73, 79, 74, 92, and 156 original research articles in each year (Table 1). The highest number of publications was in 2020, with 156 (22.3%) manuscripts referenced by 2623 sources and the lowest number of publications (35) was in 2012, accounting for 5% of all publications over the decade. Only 642 citations, the fewest, were recorded in 2015.
We rejected the null hypothesis. There was a significant difference between the number of publications before and after the midpoint (2015) of the bibliometric analysis (t = -3.08, P = 0.015, 95% CI -87.78–-12.61). We observed a strong relationship between publications and citations over the same period (Correlation coefficient 0.809, P = 0.002, CI 0.46–0.98) but there was no significant difference between the number of citations before and after the midpoint. Although there was an increasing trend for the number of publications, there was none for citations (Table 1).
Prolific authors, institutions, funders, and frequent publishers of health system research
The most prolific author of health system and health policy research in Pakistan was ZA Bhutta, with 52 publications and 3391 citations. The highest number of publications (240) per single institution was from Aga Khan University (including Aga Khan University Hospital), Karachi (Table 2). Among studies funded by foreign partners, 27 were sponsored by the National Institutes of Health of the United States, followed by Bill and Melinda Gates Foundation. Journal of Pakistan Medical Association was the leading journal in publishing health systems research related to Pakistan. The journal published 51 articles with 318 citations and a 6.24 citation impact.
Authorship patterns
As an additional analysis, the number of authors on each health system research publication were assessed. Publications with 4 authors were the highest, 122 with 1260 citations, followed by 3 authors, 103 with 992 citations.
Keyword analysis
We considered keywords mentioned 4 or more times. The most common keywords were COVID-19, patient satisfaction, health systems, immunization, primary health care, developing countries, and tuberculosis (bottom part of Table 2). The most common keywords were represented by using different circle sizes and colours to indicate various coupling clusters and keyword occurrence levels (Figure 2).
Global collaboration and bibliographic coupling among nations
The most significant publication collaboration was between Pakistan and the United States, accounting for 135 research articles authored by researchers from both countries. This was followed by collaborations with United Kingdom (110), Canada (86) and China (47) (Figure 3).
Publications forecast for 2021–2022
Models were fitted to the yearly trend of the number of publications from 2011 to 2020. The analysis indicated a regular, continuous shift in the trend over the observed period. There was generally an increasing trend, representing a long-term growth. The trend was modelled using linear and quadratic models (Figure 4). The figure and model performance measures showed that the quadratic model was a better fit for number of publications (adjusted R-squared: 0.84, P = 0.00064). To evaluate the out-of-sample prediction performance of the fitted models, the number of publications were forecasted for 2021 and 2022 (not considered in developing the model). A linear model for 2021 indicated 127 publications, quadratic model indicated 162, compared to actual 180 publications retrieved from the Scopus database. Similarly, for 2022, the quadratic model forecasted 192 publications, while the linear model predicted 137, as opposed to the actual 203 publications obtained from the Scopus database. The quadratic model had a smaller forecast error than the linear model.
Discussion
This study evaluated health systems research and publication in terms of the most productive institutions, prominent authors, citations, use of keywords, and networking. Besides predicting future publications by using a forecast model, our time series model can predict upcoming publications and has been tried in forecast and bibliometric studies (16,17). Globally, bibliometric studies have provided insight into institutional collaborations, visual mapping and assessment of health system research productivity (18). Academics have evaluated health system policy research using comparable indicators to reveal the strengths and health outcomes of health systems in low- and middle-income countries (19). At the country level, bibliometric research has been used to understand health research output and geographic research disparities. It has been used in identifying top research scholars and donors funding health systems research, as well as research capacity-building, which subsequently helped improve health outcomes (20)(21, 22).
We observed a significant difference between the number of publications on health systems research in Pakistan. From 2011 to 2015, the number of publications was 223 and the next 5 years after 2015, the number increased to 474. Research studies in all other disciplines across the country had an average increase from the year 2000 to 2011 (11). This shows that the number of publications in the health sector steadily increased, although it remained low by international standards, and predominantly published in unindexed local journals (2). Unfortunately, only 4 of more than 100 medical journals in Pakistan were indexed in Medline (23,24).
To improve the quality and quantity of health research publications, several initiatives have been implemented, including an evaluation system that sets minimum standards for journals and provides guidelines for editors, reviewers and authors. Since 2015, professional organizations like the Pakistan Association of Medical Editors have been offering training to journal editors, reviewers and authors (23) through workshops, seminars and conferences, independently and in coordination with the Eastern Mediterranean Association of Medical Editors (25,26). Most of the departments of medical education in Pakistan were established during the middle of the preceding decade and their curriculums promoted medical research. Although there were no obvious observed relationships, there appears to be some contributions from these initiatives (27). Between 2012 and 2018, a private institution reported an annual increase of 88% in publications (25). A similar trend was observed in the neighbouring India between 2009 and 2016 (28).
Pakistan made considerable efforts in the later part of the last decade to advance health systems research publications but achieved marginal results when compared to international standards. Pakistan ranked 44th in the SCImago journal and country rank for 2020 (29).
We found 697 peer-reviewed health systems focused publications from 2011 to 2020, with an average of 16.6 citations per publication. It was plausible that the initial low publication output for health systems research in Pakistan during the previous 10 years was due to certain factors common in low- and middle-income countries, such as low health research budgets, low research capacity, insufficient health data, and underutilisation of research in policymaking (30,31). This is the reason that low-income countries contributed only 2% to global health research (32,33). Sadly, Pakistan had only 0.58% of all categories of global research undertaken in 2019 (7).
Our research revealed that Aga Khan University was by far the highest contributor to health systems research publication in Pakistan. The university has evolved as a premier research institution in Pakistan during the past 20 years (7). A research metrics conducted on Aga Khan University publications from January 2010 to December 2019 revealed it as the most dominant research institution in Pakistan (34). It was the second highest contributor to medical research productivity for 2007 to 2010 (2). Our finding on the most prolific author in health research has been validated by another study, which indicates that the same author generated 686 manuscripts between 2001 and 2020 (7). Also, a bibliometric analysis of published medical research revealed that the Journal of Pakistan Medical Association was the topmost journal with 4989 papers from 2001 to 2020, followed by the Journal of College of Physicians and Surgeons (7).
Based on our findings, 19% of the published articles were written by a single author while the remaining 81% were written by more than one author; the highest number of manuscripts were published by 4 authors. An analysis published in Rawal Medical Journal shows that the majority of the articles (77%) were written by more than one author and only 23% were by one author (35). Bibliometric analysis of an indexed Pakistani journal from 2010 to 2014 reveals that the 4-author pattern was the most common (26.1%), followed by the 3-author pattern (21.3%), and single authorship (6.4%) was the least common, which correlates with our findings (36). Latif reported 4-author patterns as the most frequent (19.9%) in medical science literature (37).
None of the top 10 funding institutions that supported health systems research and publication in Pakistan was domestic. This is contrary to the recommendations of the National Commission for Health Research that 2% of the national health budget and 5% of donor funds should be allocated to research (38). Public health sector research in Pakistan is predominantly funded by 2 national organizations, the Pakistan Medical Research Council and the Higher Education Commission. An increase in the annual budget of the Research Council was observed from 2007 to 2012, but it was mainly used for employees' salaries and not for research (39). During the past 5 years, the Higher Education Commission allocated PKRs. 957 million (US$ 34.30 million) for health sector research and awarded 139 research grants (40).
Another important finding from our research is the health systems research collaborations between Pakistan and the United States, the United Kingdom, Canada, and China. A bibliometric analysis by Latif revealed that major health sector research projects in Pakistan were implemented in collaboration with authors from the United States (8.6%), followed by the United Kingdom (4.8%) and Saudi Arabia (3.0%) (41).
Study limitations
This quantitative analysis provides a snapshot of health research publications in Pakistan. Analysis was limited to the publication and citation data provided by Scopus, therefore, the findings should be extrapolated with caution. Other databases may yield a different set of records with the same search criteria. Thus, some of the articles not included in the citation database may have been missed during analysis. The key strengths of the study are the strict inclusion criteria and selection by 2 reviewers, which may have reduced the inherent biases.
Conclusions
This bibliometric analysis provides insight into the status and trends of health systems research and publications in Pakistan. It helps in understanding the existing research and identifying directions for future reform. There was an acute dearth of health systems research and publication at the beginning of the previous decade, but a few medical institutes are now taking the lead in health systems research in the country. With capacity-building support, medical institutions that are underrepresented may be motivated to contribute to health knowledge generation. District public health managers should be incentivised to publish local research.
The analysis shows that local authors collaborate more with scholars from the “West” instead of local researchers and experts. There is a need to encourage collaborative work among regional expert groups by strengthening linkages between institutions and partners. There is also a need to promote the use of regional journals to enhance the visibility of local researchers. To improve the quality of published articles, national level journals should invite high-profile writers to contribute regularly.
Funding: None.
Competing interests: None declared.
Analyse bibliométrique de la recherche sur les systèmes de santé au Pakistan
Résumé
Contexte : La recherche sur les systèmes de santé et les publications associées sont essentielles pour améliorer les soins de santé à tous les niveaux. Elles permettent de fournir des données probantes pour l'élaboration de politiques et l'amélioration de la qualité des services.
Objectifs : Évaluer les articles de recherche sur les systèmes de santé publiés au Pakistan entre 2011 et 2020, ainsi que modéliser et prévoir les tendances en matière de publication.
Méthodes : La présente étude transversale a interrogé la base de données Scopus sur les publications relatives à la recherche sur les systèmes de santé au Pakistan à l'aide des directives PRISMA (Éléments de notification préférés à des fins d'examen et de méta-analyse systématique). Des tableaux et des graphiques ont été créés à l'aide de Microsoft Excel et une visualisation a été générée sur VoS Viewer. Le logiciel SPSS version 29.1.0 a été utilisé pour l'analyse, tandis que le logiciel R a permis de tracer les données de séries temporelles.
Résultats : Au total, 697 articles, avec une moyenne de 16,6 citations, ont été publiés entre 2011 et 2020. Le nombre le plus élevé de publications (240) par institution a été atteint par l'Université Aga Khan de Karachi, y compris l'Hôpital universitaire Aga Khan. Une différence significative a été constatée entre le nombre de publications parues avant et après la date correspondant au point médian (2015) de la période couverte par l'analyse bibliométrique (t = -3,08 ; p = 0,015 ; IC à 95 % -87.78 – -12.61). Nous avons observé une forte corrélation entre les publications et les citations au cours de la même période (coefficient de corrélation 0,809 ; p = 0,002 ; IC 0,46-0,98), mais il n'y avait pas de différence significative entre le nombre de citations avant et après 2015.
Conclusion : Il existait une pénurie aiguë de publications relatives à la recherche sur les systèmes de santé au début de la période d'étude. Quelques institutions médicales prennent désormais l'initiative de mener et de publier des recherches dans ce domaine. Un soutien technique et financier est nécessaire pour renforcer la capacité des institutions médicales et des chercheurs pakistanais à contribuer davantage à la production de connaissances dans le pays.
تحليل بيبليومتري لبحوث النظم الصحية في باكستان
نعيم حسن سليم، محمد نويد أصغر، حسن علي دلفي شيرازي، سهيل شاند
الخلاصة
الخلفية: لبحوث النظم الصحية وما نشر منها أهميةٌ بالغة في تحسين الرعاية الصحية على جميع مستويات الرعاية. فهي توفر الدلائل اللازمة لوضع السياسات وتحسين مخرجات الخدمات.
الأهداف: هدفت هذه الدراسة إلى تقييم بحوث النظم الصحية المنشورة في باكستان في الفترة من عام 2011 إلى عام 2020، ووضع نماذج وتوقعات لاتجاه المنشورات.
طرق البحث: بحثت هذه الدراسة المقطعية في قاعدة بيانات منشورات بحوث النظم الصحية لباكستان في منصة Scopus باستخدام بنود التبليغ المفضلة للاستعراضات المنهجية والتحليلات التلوية. وأُعدت الجداول والرسوم البيانية باستخدام برنامج Microsoft Excel وأُنتج العرض البصري على برنامج VoS Viewer، واستُخدمت النسخة 29.1.0 من برنامج SPSS للتحليل، فيما استُخدم برنامج R لرسم بيانات السلسلة الزمنية.
النتائج: نُشر ما مجموعه 697 مقالة بمتوسط 16.6 تنويهًا في الفترة من عام 2011 إلى عام 2020. وكان أكبر عدد مما نشر (240 مؤلفامنشورًا) لمؤسسة واحدة من جامعة الآغا خان (وهذا يتضمن مستشفى الآغا خان الجامعي)، في مدينة كراتشي. وكان ثمة فارق كبير بين عدد ما نشر قبل نقطة الوسط (عام 2015) للتحليل البيبليومتري إحصائية الاختبار (t = -3.08, P = 0.015, 95% CI= -87.78 – -12.61) وبعدها. ولاحظنا أيضًا علاقة قوية بين ما تم نشره والتنويهات خلال الفترة نفسها (Correlation Coefficient = 0.809, P = 0.002, 95% CI= 0.46–0.98) لكن لم يكن هناك فارق كبير بين عدد التنويهات قبل نقطة الوسط وبعدها.
الاستنتاجات: كانت ثمة ندرة شديدة في منشورات بحوث النظم الصحية في بداية الفترة المشمولة بالدراسة. أما في الوقت الحالي، فتأخذ بضعة معاهد طبية زمام المبادرة بإجراء بحوث النظم الصحية ونشرها. ويلزم توفير الدعم التقني والمالي لتعزيز قدرة المؤسسات الطبية والباحثين الطبيين من باكستان على الإسهام بقدر أكبر في توليد المعارف داخل البلد.
References
1. Mahboob N, Abbas M, Lodhi A, Rab M, Jones.C..S. The university research system in Pakistan. Islamabad: British Council in Pakistan; 2019.
2. Mushtaq A, Abid M, Qureshi MA. Assessment of research output at higher level of educaton in Pakistan. JPMA The Journal of the Pakistan Medical Association. 2012;62(6):628-32, https://europepmc.org/article/med/22755361https://europepmc.org/article/med/22755361;
3. WHO. The world health report 2000. Geneva: World Health Organization; 2000. Report No.: 924156198X.
4. Barron PB, G; Edwards,J; Makhanya, N; Palmer, N. Health System Reaserch. Report. Durban, S Africa: Health Systems Trust; 1997 April 1997. Report No.: 1-919743-02-2
5. Rochester MK. Professional communication through journal articles. Istanbul, Turkey: 61st IFLA General Conference - Conference Proceedings 1995 [cited 2021 4 July]. Available from: https://archive.ifla.org/IV/ifla61/61-rocm.htm.
6. Pritchard A. Statistical Bibliography or Bibliometrics? Journal of Documentation. 1969;25(4):348-9, https://www.researchgate.net/publication/236031787_Statistical_Bibliography_or_Bibliometricshttps://www.researchgate.net/publication/236031787_Statistical_Bibliography_or_Bibliometrics;
7. Haq IU, Rehman ZU. Medical Research in Pakistan; A Bibliometric Evaluation from 2001 to 2020. Library Philosophy and Practice. 2021, https://digitalcommons.unl.edu/libphilprac/5294/:1-13; https://digitalcommons.unl.edu/libphilprac/5294/;
8. Saleem MM, Pervaiz M, Baig B, Khan WA, Malik BA. Pakistan Armed Forces Medical Journal: Five Year Bibliometric Analysis from 2011 to 2015. PAFMJ. 2018;68(4):1037-41; https://pafmj.org/index.php/PAFMJ/article/view/85; https://pafmj.org/index.php/PAFMJ/article/view/85;
9. Baladi ZH, Haq IU. Comparative Bibliometric Analysis of Six Health Science Journals Published in Pakistan 2006-2015. Library Philosophy and Practice (e-journal). 2018, http://digitalcommons.unl.edu/libphilprac/2001 http://digitalcommons.unl.edu/libphilprac/2001
10. Lundberg J. Bibliometrics as a research assessment tool: impact beyond the impact factor [thesis]. Stolkholm: Karolinska Institutet (Sweden); 2006. Available from: https://www.proquest.com/openview/6370a7ac946d0e1743ca2a301c1637d4/1?pq-origsite=gscholar&cbl=2026366&diss=y.
11. Ahmed I. Medical research in Pakistan. Isra Med J. 2018;10(6):325-6, http://www.imj.com.pk/2018/12/21/medical-research-in-pakistan/http://www.imj.com.pk/2018/12/21/medical-research-in-pakistan/;
12. Memon AR. Bibliometric analysis of the Journal of Pakistan Medical Association during the period from 1965 to 2018. JPMA The Journal of the Pakistan Medical Association. 2019;69(8):1150-8, https://jpma.org.pk/article-details/9279?article_id=9279https://jpma.org.pk/article-details/9279?article_id=9279;
13. Ullah M, Butt IF, Haroon M. The Journal of Ayub Medical College: a 10-year bibliometric study. Health Information & Libraries Journal. 2008;25(2):116-24; https://doi.org/10.1111/j.1471-1842.2007.00757.x; https://onlinelibrary.wiley.com/doi/10.1111/j.1471-1842.2007.00757.x;
14. Haq IU EG, Dana I. Research Publications on Medical Microbiology in Pakistan during the period 2013-2017. Library Philosophy and Practice (e-journal). 2019;1(2253), https://digitalcommons.unl.edu/libphilprac/2253https://digitalcommons.unl.edu/libphilprac/2253;
15. Moed HF. Differences in the construction of SCI based bibliometric indicators among various producers: A first over view. Scientometrics. 1996;35(2):177-91; https://doi.org/10.1007/BF02018476; https://link.springer.com/article/10.1007/BF02018476 - citeas;
16. Ghalehkhondabi I, Ardjmand E, Weckman GR, Young WA. An overview of energy demand forecasting methods published in 2005–2015. Energy Systems. 2017;8(2):411-47; https://doi.org/10.1007/s12667-016-0203-y; https://doi.org/10.1007/s12667-016-0203-y;
17. Singh S, Dhir S, Das VM, Sharma A. Bibliometric overview of the Technological Forecasting and Social Change journal: Analysis from 1970 to 2018. Technological Forecasting and Social Change. 2020;154:119963; https://doi.org/10.1016/j.techfore.2020.119963; https://www.sciencedirect.com/science/article/pii/S0040162519305220;
18. Yao Q, Chen K, Yao L, Lyu P-h, Yang T-a, Luo F, et al. Scientometric trends and knowledge maps of global health systems research. Health Research Policy and Systems. 2014;12(1)https://doi.org/10.1186/1478-4505-12-26; https://doi.org/10.1186/1478-4505-12-26;
19. English KM, Pourbohloul B. Increasing health policy and systems research capacity in low- and middle-income countries: results from a bibliometric analysis. Health Research Policy and Systems. 2017;15(1):64; https://doi.org/10.1186/s12961-017-0229-1; https://doi.org/10.1186/s12961-017-0229-1;
20. Keshri VT, NS. Where there is no research: a bibliometric analysis of health research output from the resource-constrained Bihar state of India. Journal of Global Health Reports. 2020;4:e2020086; https://doi.org/10.29392/001c.16633; https://www.joghr.org/article/16633-where-there-is-no-research-a-bibliometric-analysis-of-health-research-output-from-the-resource-constrained-bihar-state-of-india;
21. Boyce R, Rosch R, Finlayson A, Handuleh D, Walhad SA, Whitwell S, et al. Use of a bibliometric literature review to assess medical research capacity in post-conflict and developing countries: Somaliland 1991–2013. Tropical Medicine & International Health. 2015;20(11):1507-15; https://doi.org/10.1111/tmi.12590; https://onlinelibrary.wiley.com/doi/abs/10.1111/tmi.12590;
22. Sweileh WM. A bibliometric analysis of global research output on health and human rights (1900–2017). Global Health Research and Policy. 2018;3(30):1-10; https://doi.org/10.1186/s41256-018-0085-8; https://doi.org/10.1186/s41256-018-0085-8;
23. Sherin A. Role of regulatory bodies in improving the quality of medical journals of Pakistan. Khyber Medical University Journal. 2015;7(4):145-6; http://orcid.org/0000-0003-1173-9671; https://www.kmuj.kmu.edu.pk/article/view/145.;
24. PakMediNet. Medical Journals of Pakistan Rawalpindi: PakMediNet 2001 [Available from: https://www.pakmedinet.com/page/aboutus.
25. Sherin A. Medical journalism in Pakistan: Where do we stand? Journal of Postgraduate Medical Institute. 2010;24(3), https://jpmi.org.pk/index.php/jpmi/article/view/1221/1129https://jpmi.org.pk/index.php/jpmi/article/view/1221/1129;
26. PAME. Pakistan Association of Medical Editors 2016 [cited 2023 25 April ]. Available from: https://www.pame.org.pk/objectives.htm.
27. Latif MZ, Wajid G. Reforming Medical Education in Pakistan through strengthening Departments of Medical Education. Pak J Med Sci. 2018;34(6):1439-44; https://doi.org/10.12669/pjms.346.15942;
28. CHAMAN Sab M; Dharani Kumar P; Biradar SB. Medicine Research in India: A Scientometric Assessment of Publications during 2009–2018. Library Philosophy and Practice (e-journal). 2018, https://digitalcommons.unl.edu/libphilprac/2186/https://digitalcommons.unl.edu/libphilprac/2186/;
29. SCImago Journal & Country Rank [Internet]. SCImago Lab. 2022 [cited JUNE 2022]. Available from: http://www.scimagojr.com/.
30. Salager-Meyer F. Scientific publishing in developing countries: Challenges for the future. Journal of English for academic purposes. 2008;7(2):121-32; https://doi.org/10.1016/j.jeap.2008.03.009; https://www.sciencedirect.com/science/article/abs/pii/S1475158508000271;
31. Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M. The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Research Policy and Systems. 2003;1(1):2; https://doi.org/10.1186/1478-4505-1-2; https://doi.org/10.1186/1478-4505-1-2;
32. Rahman MM, Ghoshal UC, Ragunath K, Jenkins G, Rahman M, Edwards C, et al. Biomedical research in developing countries: Opportunities, methods, and challenges. Indian Journal of Gastroenterology. 2020;39(3):292-302; https://doi.org/10.1007/s12664-020-01056-5; https://doi.org/10.1007/s12664-020-01056-5;
33. Adam T, Ahmad S, Bigdeli M, Ghaffar A, Røttingen J-A. Trends in health policy and systems research over the past decade: still too little capacity in low-income countries. PloS one. 2011;6(11):e27263; https://doi.org/10.1371/journal.pone.0027263;
34. Yusuf M, Gatiti P. Research metric analysis of the Aga Khan University Medical College, Karachi on Scopus database 2010-2019. Library Philosophy and Practice (e-journal). 2021, https://ecommons.aku.edu/libraries/57https://ecommons.aku.edu/libraries/57;
35. Javed M, Shah SS. Rawal Medical Journal-An analysis of citation pattern. Rawal Medical Journal. 2008;33(2):254-7, https://pesquisa.bvsalud.org/portal/resource/pt/emr-90010https://pesquisa.bvsalud.org/portal/resource/pt/emr-90010;
36. Ullah S, Jan SU, Jan T, Ahmad HN, Jan MY, Rauf MA. Journal of the College of Physicians and Surgeons of Pakistan: Five Years Bibliometric Analysis. Journal of the College of Physicians and Surgeons Pakistan. 2016;26(11):920-3; https://doi.org/10.32593/jstmu/Vol3.Iss1.67; https://www.jcpsp.pk/archive/2016/Nov2016/12.pdf;
37. Latif A, Haq IU. Bibliometric research productivity analysis: A case study of Shifa Tameer-e-Millat University. Journal of Shifa Tameer-e-Millat University. 2020;3(1):49-55; https://doi.org/10.32593/jstmu/Vol3.Iss1.67; https://j.stmu.edu.pk/ojs/index.php/jstmu/article/view/67;
38. Evans J. Commission on Health Research and Development. NEW YORK: Commission on Health Research for Development; 1990. Contract No.: 0-19-520838-2.
39. Ghaffar AZ, S. Qureshi,H. Hafeez,A. Medical education and research in Pakistan. The Lancet. 2013;381(9885):2234-6; https://doi.org/10.1016/S0140-6736(13)60146-4; https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(13)60146-4/fulltext;
40. Awais N. Support by HEC for Promotion of Health Research in Pakistan. 2021.
41. Meo S, Almasri A, Usmani A. Research Productivity of Pakistan in Medical Sciences during the period 1996-2012. European review for medical and pharmacological sciences. 2013;17(21):2839-46, https://www.europeanreview.org/article/5779https://www.europeanreview.org/article/5779;
