Schools of public health in Europe: common mission—different progress

Introduction

The wide range of public health challenges in Europe requires a sufficient and qualified public health workforce to implement evidence-based interventions, tackling piled up specific problems and health inequities at the population level (1,2). Therefore, boosting up education and training for public health research and practice obtains the attention of both scientists and public health practitioners. The primary institutional setting, which provides a competent public health workforce, is the School of Public Health (SPH) (3-5). Institutionalized Schools of Public Health emerged from the late 19^th century in the United States with rapid growth in number and manifestations during the early 20^th century (6). SPHs are culturally and socially contextualized in local, national, and regional perspectives on health and health needs (7). They share, however, many standard features. Besides having multidimensional missions concentrated on education, research, and practice targeting the health of populations, contemporary SPHs are striving to engage in knowledge brokering and support evidence-based public health policies (8-10). Like other advanced training and teaching institutions, SPHs attempt to contribute to sustainable development (11-13) and are confronted with digitalization (14), additionally boosted by the Corona pandemic since early 2020. Besides, worldwide, SPHs have to prepare for the next decades by upgrading their curricula to correspond more to real life, embrace practice-based and student-centered education, and their capabilities to respond to permanent change (15).

Today, we observe frequent reflections highlighting the different profiles of SPHs’ as mostly academic or state institutions, such as in infrastructure, types of educational programs, methods and quality of teaching, ratios of students’ enrolment and completion, and the existence of accreditation as well as proximal and distal learning outcomes. Such explorations are regular and frequent among academic communities of North American SPHs (10,16,17) while recently appearing also elsewhere (18-23). Quantitative analysis is a typical approach in these reviews and the qualitative elaboration of accreditation procedures, usually multidisciplinary content areas, and competencies transferred in broad public health sectors. In general, there is a marked trend increasing the establishment of new SPHs and enrollment and completion of the core academic, public health program—the Master of Public Health (10,22).

As a result of the long history of development (4,24), the variation of SPHs’ contextual factors formed different models of their institutional organization. Only at the beginning of the new millennium, worldwide, deans and directors of SPHs opened a debate about delays and progress in development and related determinants of training, research, and practice in real-life situations (25). Simultaneously, the deans and directors call for more robust networks between SPHs, having in mind the impressive benefits from academic partners’ interaction. This trend induced closer attention to the variation between SPHs.

Looking at the European region, when in 1996 one of the first publications on the institutional landscape of education and training appeared, the variation of the profiles of SPHs became obvious (7) as regard types of institutional settings, perspectives of establishing new SPHs, and upgrading current entities. In 2009, ten years later, Evans (4)—in his tribute to SPHs “Learning from history, looking to the future”—pointed to the absence of an evaluation of SPHs in terms of outcomes, apart from those done in the United States. Therefore, in 2010, the Association of Schools of Public Health in the European Region (ASPHER) established a working group to collect better evidence about European SPHs and overcome the lack of information for the sake of advancement in the professionalization of public health (26). One of the first well-documented variations between countries relates to the accreditation process of SPHs in Europe (27). Besides the examples from the United Kingdom (28), an excellent success story comes from Switzerland and their model of merging SPHs and different institutional departments—taking benefit from variation across the country to establish one operational national SPH (29). Also, this example underlines the importance of networking among SPHs, both within and between countries (5,30,31). It triggered even more attention to the exploration of different pathways. Variation of institutional settings can also help develop beneficial partnerships uncovering factors, which contribute to progress for some SPHs and delay others.

The ASPHER surveys of its members—the European SPHs between 2011 (32) and 2016 (33) revealed well-known variations. Various progressive intentions were indicated regarding almost all contextual factors—institutional structure, process, and outcomes. However, on average, significant differences between the two surveys—with a few exceptions—could be identified for neither the organization and teaching areas nor the assumed graduates’ performance. The shared mission of all SPHs in the European Region is to provide qualified bachelor, master, and doctoral degrees in public health-related fields (34). The required competence profile of graduates is defined by the ten European Essential Public Health Operations (EPHOs) (35). Therefore, it opens trajectories for variations between SPHs and the performance of the public health workforce and its contribution to population health.

Building on the first publications in 2013 (32) and 2019 (33), the primary goal of this analysis is to find patterns in the dynamics between the two surveys. This paper tries to find explanations of progress and apparent standstill or even regression between the two surveys in 2011 and 2015/6. We present this article in accordance with the SURGE reporting checklist (available at http://dx.doi.org/10.21037/jphe-21-4).

Methods

Out of 96 SPHs eligible in Survey II (2015/16), 78% or 81.3% participated, 48 of them also in Survey I (2011) (32,33). We identified 36 SPHs or 75% with complete data sets for both surveys, corresponding to a prospective cohort design (https://cdn.amegroups.cn/static/public/jphe-21-4-1.pdf gives an overview). The data collection was based on a standardized questionnaire (https://cdn.amegroups.cn/static/public/jphe-21-4-2.pdf), filled in under the responsibility of the head of the respective SPH. The information obtained has been grouped into clusters. To find an optimal grouping of the clusters, observations had to be organized so that within the clusters, they were as similar as possible, and the various clusters dissimilar to each other as much as possible. The study conforms to the provisions of the Declaration of Helsinki (as revised in 2013). This analysis does not involve individual data. Therefore, ethical approval is not required.

Statistical analysis

Several techniques are available to cluster multidimensional observations. However, there is no clustering technique, which is universally applicable (36). From the variety of techniques, we decided to apply a k-means approach because it allows combining both continuous [EPHO ratings (35)] and categorical variables (i.e., participating SPHs, respectively, the cities they are located in). The basic idea of this approach is unpretentious. Given a fixed number (k) of clusters, one has to assign the observations to clusters so that the means across clusters (for all variables) are as different from each other as possible (37). Central issues in the analysis are considerations of how an acceptable number of clusters should be determined, how similarity or dissimilarity of the clusters should be quantified, and finally, how the validity of the classification could be evaluated. For an interpretation of the results, it is necessary to clarify in advance whether the clusters of the two surveys are equivalent. Accordingly, any movement of cluster members between the clusters can be easily identified as improvement, worsening, or persistence of the initial status of Survey I.

This assessment of the particular clusters of the surveys requires a more detailed understanding of the ranking process applied here. The allocation of the SPHs to clusters is based on vectors. Those vectors consist of the variables, i.e., here the ten EPHOs. To compare, e.g., cluster 1 in Survey I with cluster 1 in Survey II, all ten variables would have to be considered or compared simultaneously, e.g., with the aid of lexicographic preferences (38-40). Alternatively, the variables should have to be converted to a single number by amalgamation or aggregation. Aggregation methods are types of calculations that are used to group values into a single metric. Wan et al. (41) give an overview of the various ways to aggregate multiple criteria. The simplified lexicographical ordering focuses primarily on the first variable. It does not take note of any possible exchange relationship (indifferences) between the variables (EPHOs). Another approach recommends combining the Analytic Hierarchy Process (AHP) (42) with lexicographical ordering (43). However, under the given conditions, we preferred the most feasible aggregation approach, which does not require weights to reflect the exchange relationships (indifferences) between the variables. To elude the determination of subjective weights, which could be scrutinized, we referred instead to the characteristics of vectors.

To aggregate the ten variables’ values into a single number, we calculated the magnitude (or length) of the position vectors. The magnitude of a position vector is a specific measure to describe the position (44) of a cluster member in an n-dimensional space. Here ten dimensions due to 10 EPHOs. The EPHOs are integrated into one number by applying the Pythagorean formula, i.e., calculating the root of the sum of squared individual EPHO values (45).

The dissimilarity of observations and clusters are assessed through Euclidean Distances then (46). Alternatives like Chebychev Distances measures are useful when defining two objects as “different” if they are different in any dimension (47). Squared Euclidean Distances puts progressively greater weight on objects that are further apart. The outcomes of City-block (Manhattan) Distances are in most cases similar to the simple Euclidean Distance. However, in this measure, the effect of single large differences (outliers) is dampened. Euclidean Distances are perhaps the most commonly chosen type of distance. It is the geometric distance in a multidimensional space (46). During the statistical analysis, continuous variables are standardized to prevent that different scaling might have an impact on the cluster analysis. If only one scale is used (e.g., the Likert scale throughout the measurement of the EPHOs), standardization would not be necessary. However, the routine used here standardizes the measured values automatically when using Euclidean Distance measurement with continuous variables.

The clustering process is sensitive to the initial choice of the seeds, i.e., starting values of the calculation (44). That means that the clustering process’s outcomes can depend to some extent on the initial configuration (cluster means or centers). To be on the safe side, we decided in favor of the option “choose observations to maximize initial between-cluster distances.” In this case, the statistical software will set observations or objects as the initial cluster centers; the choice of the object follows the rules to maximize the initial cluster distances. Specifically, (I) the software will select the first N (number of clusters) cases to be the respective cluster centers; (II) subsequent cases will replace previous cluster centers if their smallest distance to any of the cluster centers is larger than the smallest distance between clusters; if this is not the case, then (III) subsequent cases will replace initial cluster centers if their smallest distance from a cluster center is larger than the distance of that cluster center from any other cluster center. The effect of this selection procedure is to maximize the initial distances between clusters (46).

As mentioned above, it is of great importance how the appropriate number of clusters should be determined. There are two options: Firstly, theory-driven, i.e., there is a clear understanding of how many clusters should be appropriate or expected, e.g., based on literature and earlier research; secondly, cross-validation (44) and the automatic determination of the number of clusters by the statistical software. We choose the latter option: the data are divided randomly into two subsets, then. The cluster analysis is carried out separately on both subsets. If the clusters are valid, the result, i.e., the number of clusters recommended, should be the same (44).

Because we used Euclidean distance measures, as explained above, the variables are automatically standardized. The number of clusters is determined based on V-fold cross-validation of the statistical software (46).

Results

To completely represent the data from both surveys, three clusters were considered sufficient. In Figure 1, their position along the ten EPHOs is shown with an overlap of curves at EPHO 4.1. None of the improvements or worsening between the two surveys in each cluster is significant.

Figure 1 Means of continuous variables from Survey I and Survey II.

The most apparent difference between the two surveys is the ordering of the clusters and their shapes. Within Survey II, cluster 3 ranks higher than cluster 2 and partially ranks higher than cluster 2 of Survey I (see also the length of position vectors in Figure 2). The pattern of the clusters in Survey II is more distinct and separated than in Survey I. In Survey I, cluster 2 and cluster 3 overlap at EPHO 4.1.

Figure 2 Comparison of the length of the position vectors (magnitude) by cluster.

The integration of the ten EPHOs into single measurement numbers for each SPH, using position vectors according to the Pythagorean Theorem, allows analyzing the movements of the SPHs between the two surveys. Their assignment to the three clusters has changed frequently. The first cluster in Survey I comprises 15 SPHs compared to 13 SPHs in Survey II; cluster 2 in Survey I contains 12 SPHs, in Survey II 11 SPHs; cluster 3 comprises 9 SPHs in Survey I but 12 SPHs in Survey II. Altogether, 20 SPHs changed their cluster and only 16% or 44.4% stayed in the same cluster, moving up and down within it.

In summary, 16 of 36 SPHs (44.4%) shown in Table 1 have improved between 2011 and 2015/16 according to the increased magnitude of their position vector, with 9 SPHs (25%) showing significant progress at P<0.05. From the 20 SPHs (55.6%) that decreased their performance, 11 SPHs had a significant reduction in outcomes of Master of Public Health programs at P<0.05, which implies that N=20 or 55.6% of the participating SPHs evidenced significant changes.

In Figure 2, the three clusters of Survey I are arranged in descending order. The corresponding clusters in Survey II show a lower-ranked cluster 2, which is in line with the findings of Figure 1. A cross-comparison indicates that between the two surveys, some boxes (clusters) overlap. In these cases, the medians lie in the interquartile ranges, which means that no significant differences between the surveys can be expected in these cases. The arrows show possible but non-significant movements between Survey I and Survey II, which a Wilcoxon Matched Pairs Test (Table S1) has confirmed.

Moreover, the individual movements of SPHs starting from the 3 clusters in Survey I towards the clusters in Survey II are to be evaluated separately. A test of significance (see Table S2) based on all cluster members shows that movements between clusters are not significant. Looking at the individual members changes the picture a bit. Two kinds of movements have to be recognized: Movement between clusters and movements within clusters. The cluster in which a member is located may be higher ranked, thus indicating an improvement; however, based on comparing the individual length of vectors, its situation has worsened. The cluster as a whole has moved upwards. However, the individual SPH slides down within the cluster box, indicated by a smaller vector and a less favorable position. The opposite is possible also. The final evaluation of the movements of individual SPHs reflects the changes in the length of vectors.

Finally, we analyzed whether descriptive variables of relevance as published earlier [Laaser 2019 (33), Table 1] influence the importance of EPHOs for clustering. We selected “University-based SPH, Involvement in other programs, Lecturers from other programs, Active methods of learning, Regularly updated websites, Presentation at any social network, Strong practice links established, Research training of students, Alumni surveys executed, Ready to share experience, and Interested in student mobility.” As all descriptors are nominal, regression analyses are not applicable. Figure 3 together with its description in Table 2 confirms the analyzes presented above (Figures 1,2, Table 1) in that there are no significant differences between clusters 1, 2, and 3 according to their descriptors (for details, see Table S1).

Figure 3 Influence of descriptive variables on the importance of EPHOs for clustering.

Discussion

This paper shows that the average standstill of the European SPH as analyzed earlier (33) does not provide the full picture: Almost every SPH changed its ranking up- or downward in the first half of the 2010s, only 9 SPHs improve significantly. Thus, we see the composite of a small group of well-established and progressive SPHs, whereas a larger group is characterized by standstill and even regression. These movements occur between the two surveys and as well within the clusters, especially within the second and third clusters of the first Survey, leading to an exchange of their position in the second Survey. It explains why none of the clusters is significantly different, and the available descriptive variables have no significant influence. It seems that progress may be determined by personal motivation and engagement, which changes with the personalities involved. No structural differences could be identified.

Uncoordinated and non-directed movements seem to be a primary characteristic of the European public health scene: a lack of agreement and harmonization towards common, well-defined objectives as well as a related measurement of progress over the years. However, in the previous paper, we stated that positive intentions—described in qualitative terms—might lead to more progress in the second half of the decade; on the other hand, 20 SPHs decreased their performance up to 2015/16, 11 even significantly. It remains questionable whether this situation could have been repaired within the few years up to now.

Our education systems reflect the societies they are grounded in rather than aspirations of the societies we hope to create and leave to our children. A vision of integrated, coordinated, and ambitious public health, equitable to all, seems still far away, despite decades of calls to prioritize health over disease and public health over medicine, which may change shortly due to the lessons learned from the COVID-19 pandemic (48). The situation reminds us of the adage that medical curricula have seen a century of ‘reform without change’ (49,50). However, the field and profession of public health have always been about bettering society and the human lot. Therefore, the question begs how standards that reflect such an ambition would drive improvement and evolution rather than confirm the status quo or even a regression to more disconnected realities. Contrary to our assumption one year ago, the identified negative trend might coincide with a loss of prestige and relevance of public health and health sciences in Europe despite the successful organization of the big European Public Health Conferences (51). A look into that should be a central task of ASPHER, taking the new chances originating from the world’s experience with COVID-19 (52).

Strengths and limitations: a unique strength of this paper and the preceding one (33) is that the two surveys on SPHs in Europe—together with the Survey on employers of public health professionals (53)—still constitute the only database to provide research-based information on the status and development of the European SPHs. Proposed scientific surveys on graduates have not been operationalized up to now. Limitations are imposed by the incomplete participation with only 48 SDPs taking part in both surveys (12 of them with incomplete data sets) and the extended time lag until publication due to low capacity and lack of financial support.

Conclusions

Nevertheless, certain conclusions and recommendations can be safely formulated, based on the lack of convincing overall progress and our own decades-long experience in the field of the public health. We suggest that connecting the public health mission and its standards more explicitly and visibly to local, national and transnational policies and politics is the only way out of what is potentially a vicious circle (54). Education for public health and its infrastructure is, first of all, polity; however, it is not separate from policies and political considerations: it is politics—Rudolph Virchow at his best!

To better understand these mechanisms, it would be worthwhile to investigate the political environment where decisions are made, its processes, and the relevant decision-making with its outcomes. Decisions here are politically motivated, demonstrated by the partly erratic decision-making processes in the context of the COVID-19 pandemic. They are not primarily the result of scientific (rational) problem-solving. Analyzing and finally, understanding decision outcomes (policies) have to consider the specific organizational structure (polity) and the initiated processes (politics). However, the decisions usually will emerge from the competing interests of decision-makers, or in other words, actors (e.g., politicians, stakeholders, etc.) and special interest groups. Under the heading “policy field analysis,” several theories, approaches, and tools are available (55). Decision-makers are not acting isolated. They are embedded in various networks. Thus, network analysis could reveal if and how particular individual or institutional positions in a cooperating network influence the decision-making procedures (56), thus bringing in their views and priorities.

Our institutions need to accept this and act on it. We, therefore, agree with Czabanowska et al. (57) that the multidisciplinary public health profession urgently needs an autonomous basis in terms of an own chamber, a formal acknowledgment in the EU, and an agreed ethical code for public health professionals (58,59). It would make the public health engagement in the political discourse justified and legitimate and drive an accountable development and implementation of the full suite of public health training standards at every level of education.

Acknowledgments

We would like to thank all members of ASPHER for their commitment to responding to the survey questionnaire.

Funding: None.

Footnote

Reporting Checklist: The authors have completed the SURGE reporting checklist. Available at http://dx.doi.org/10.21037/jphe-21-4

Data Sharing Statement: Available at http://dx.doi.org/10.21037/jphe-21-4

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/jphe-21-4). The authors have no conflicts of interest to declare.

Ethical Statement: The authors declare that they are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This analysis does not involve individual data. Therefore, ethical approval is not required. The study conforms to the provisions of the Declaration of Helsinki (as revised in 2013).

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Mackenbach JP, Valverde JR, Bopp M, et al. Determinants of inequalities in life expectancy: an international comparative study of eight risk factors. Lancet Public Health 2019;4:e529-37. [Crossref] [PubMed]
2. World Health Organisation. Healthy, prosperous lives for all: the European Health Equity Status Report. Copenhagen, Denmark 1919. Available online: http://www.euro.who.int/en/publications/abstracts/health-equity-status-report-2019 (accessed 11 January 2020).
3. Laaser U. The institutionalization of public health training and the health sciences. Public Health Rev 2002;30:71-95. [PubMed]
4. Evans D. The role of schools of public health: learning from history, looking to the future. J Public Health (Oxf) 2009;31:446-50. [Crossref] [PubMed]
5. Foldspang A. Towards a public health profession: the roles of essential public health operations and lists of competences. Eur J Public Health 2015;25:361-2. [Crossref] [PubMed]
6. Winslow CEA. The untilled fields of Public Health. Science 1920;51:23-33. [Crossref] [PubMed]
7. de Leeuw E. European schools of public health in state of flux. Lancet 1995;345:1158-60. [Crossref] [PubMed]
8. Jessani NS, Siddiqi SM, Babcock C, et al. Factors affecting engagement between academic faculty and decision-makers: learnings and priorities for a school of public health. Health Res Policy Syst 2018;16:65. [Crossref] [PubMed]
9. Lawrence LM, Bishop A, Curran J. Integrated Knowledge Translation with Public Health Policy Makers: A Scoping Review. Healthc Policy 2019;14:55-77. [Crossref] [PubMed]
10. Leider JP, Plepys CM, Castrucci BC, et al. Trends in the Conferral of Graduate Public Health Degrees: A Triangulated Approach. Public Health Rep 2018;133:729-37. [Crossref] [PubMed]
11. United Nations. The Sustainable Development Goals Report 2019. New York 2019.
12. Kioupi V, Voulvoulis N. Education for Sustainable Development: A Systemic Framework for Connecting the SDGs to Educational Outcomes. Sustainability 2019;11:6104. [Crossref]
13. Findler F, Schönherr N, Lozano R, et al. The impacts of higher education institutions on sustainable development. IJSHE 2019;20:23-38. [Crossref]
14. Odone A, Buttigieg S, Ricciardi W, et al. Public health digitalization in Europe. Eur J Public Health 2019;29:28-35. [Crossref] [PubMed]
15. Sullivan L, Magaña L, Galea S. Peering into the future of public health teaching. Lancet Public Health 2018;3:e268 [Crossref] [PubMed]
16. Hemans-Henry C, Blake J, Parton H, et al. Preparing Master of Public Health Graduates to Work in Local Health Departments. J Public Health Manag Pract 2016;22:194-9. [Crossref] [PubMed]
17. Krasna H, Kornfeld J, Cushman L, et al. The New Public Health Workforce: Employment Outcomes of Public Health Graduate Students. J Public Health Manag Pract 2021;27:12-9. [Crossref] [PubMed]
18. Zwanikken PAC, Huong NT, Ying XH, et al. Outcome and impact of Master of Public Health programs across six countries: education for change. Hum Resour Health 2014;12:40. [Crossref] [PubMed]
19. Dlungwane T, Voce A, Searle R, et al. Master of Public Health programmes in South Africa: issues and challenges. Public Health Rev 2017;38:5. [Crossref] [PubMed]
20. Manyara AM, Buunaaisie C, Annett H, et al. Exploring the multidisciplinary extent of public health career structures in 12 countries: an exploratory mapping. J Public Health (Oxf) 2018;40:e538-44. [Crossref] [PubMed]
21. Tiwari R, Negandhi H, Zodpey S. Current status of master of public health programmes in India: a scoping review. WHO South East Asia J Public Health 2018;7:29-35. [Crossref] [PubMed]
22. Hou J, Wang Z, Liu X, et al. Public health education at China's higher education institutions: a time-series analysis from 1998 to 2012. BMC Public Health 2018;18:679. [Crossref] [PubMed]
23. Joshi V, Joshi NK, Bajaj K, et al. Undergraduate public health degree courses in India: selected overview. Int J Community Med Public Health 2019;6:4224-31. [Crossref]
24. Frenk J, Chen L, Bhutta ZA, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet 2010;376:1923-58. [Crossref] [PubMed]
25. Petrakova A, Sadana R. Problems and progress in public health education. Bull World Health Organ 2007;85:963-5; discussion 966-70. [Crossref] [PubMed]
26. ASPHER Working Group on Innovation and Good Practice in Health. What Do Schools of Public Health and Employers of Public Health Professionals Think about Performance? Brussels: ASPHER, 2010.
27. Goodman JD. The history of European public health education accreditation in perspective. South Eastern European Journal of Public Health 2015;III: (SEEJPH). [Crossref]
28. Paccaud F, Weihofen A, Nocera S. Public Health Education in Europe: Old and New Challenges. Public Health Rev 2011;33:66-86. [Crossref]
29. Künzli N, Crivelli L, Sprumont D, et al. Does the Swiss School of Public Health exist? Int J Public Health 2015;60:873-5. [Crossref] [PubMed]
30. Otok R, Czabanowska K, Foldspang A. Public health educational comprehensiveness: The strategic rationale in establishing networks among schools of public health. Scand J Public Health 2017;45:720-2. [Crossref] [PubMed]
31. Otok R, Richardson E, Czabanowska K, et al. The public health workforce. In: Rechel B, Jakubowski E, McKee M, et al. eds. Organization and financing of public health services in Europe. Copenhagen, Denmark: WHO Regional Office for Europe 2018. Available online: https://www.ncbi.nlm.nih.gov/books/NBK535724/pdf/Bookshelf_NBK535724.pdf
32. Bjegovic-Mikanovic V, Vukovic D, Otok R, et al. Education and training of public health professionals in the European Region: variation and convergence. Int J Public Health 2013;58:801-10. [Crossref] [PubMed]
33. Laaser U, Bjegovic-Mikanovic V, Vukovic D, et al. Education and training in public health: is there progress in the European region? Eur J Public Health 2020;30:683-8. [Crossref] [PubMed]
34. Association of Schools of Public Health in the European Region2020. Available online: https://www.aspher.org
35. World Health Organisation. Strengthening public health services and capacity, an action plan for Europe. Copenhagen: WHO Regional Office for Europe 2012. Available online: http://www.euro.who.int/en/health-topics/Health-systems/public-health-services/publications2/2012/european-action-plan-for-strengthening-public-health- capacities-and-services
36. Jain AK, Murty MN, Flynn PJ. Data clustering. ACM Comput Surv 1999;31:264-323. [Crossref]
37. Nisbet R, Elder JF, Miner G. Handbook of statistical analysis and data mining applications. Amsterdam, Boston: Academic Press/Elsevier, 2009.
38. LimHSJunDBKimDSAn Economic Choice Model for Lexicographic Preferences. SSRN J2015. doi:.10.2139/ssrn.2693388
39. Goswami MP, Manipushpak M, Sen D. A characterization of lexicographic preferences. Munich Personal RePEc Archive. Munich 2018. Available online: https://mpra.ub.uni-muenchen.de/id/eprint/90552
40. Encarnacion J. A Note on Lexicographical Preferences. Econometrica 1964;32:215. [Crossref]
41. Wan M, Wan R, Lazim A. Aggregation methods in group decision making: A decade survey. Informatica (Slovenia) 2017;41:71-86.
42. Dyer RF, Forman EH. An analytical approach to marketing decisions. Englewood Cliffs, NJ: Prentice-Hall, 1991.
43. Salas-Molina F, Pla-Santamaria D, Garcia-Bernabeu A, et al. A Compact Representation of Preferences in Multiple Criteria Optimization Problems. Mathematics 2019;7:1092. [Crossref]
44. Rencher AC. Methods of Multivariate Analysis, 2nd ed. New York, NY, USA: John Wiley & Sons, Inc 2002.
45. Nykamp DQ. Magnitude of a vector definition n.d. Available online: https://mathinsight.org/definition/magnitude_vector
46. TIBCO Software Inc. Statistical version 13 2017.
47. Nermend K. Employing Similarity Measures to Examine the Development of Technical Infrastructure in Polish Counties. Folia Geogr Ser Geogr Oecon 2008;7:87-97. [Crossref]
48. Martin-Moreno JM. Facing the COVID-19 challenge: When the world depends on effective public health interventions. Available online: https://www.seejph.com/index.php/seejph/article/view/3442
49. Bloom SW. Structure and Ideology in Medical Education: An Analysis of Resistance to Change. J Health Soc Behav 1988;29:294-306 [Crossref] [PubMed]
50. De Leeuw E. The politics of medical curriculum accreditation: thoughts, not facts? IJUDH 2012;2:53-69. [Crossref]
51. European Public Health Association (EUPHA). 16th World Congress on Public Health 2020.
52. Alderslade R, Kokeny M, Tsouros A. The health of the public: What has gone wrong? Available online: https://ubib-ojs-dev.ub.uni-bielefeld.de/index.php/seejph/article/view/3996
53. Vukovic D, Bjegovic-Mikanovic V, Otok R, et al. Which level of competence and performance is expected? A survey among European employers of public health professionals. Int J Public Health 2014;59:15-30. [Crossref] [PubMed]
54. De Leeuw E, Peters D. Nine questions to guide development and implementation of Health in All Policies. Health Promot Int 2015;30:987-97. [Crossref] [PubMed]
55. Parag Y. A System Perspective for Policy Analysis and Understanding: The Policy Process Networks. Systemist 2006;28:212-24.
56. Wenzel H, Bjegovic-Mikanovic V, Laaser U. Piloting an advanced methodology to analyse health care policy networks: The example of Belgrade, Serbia. South Eastern European Journal of Public Health 2015;IV: (SEEJPH). [Crossref]
57. Czabanowska K, Laaser U, Stjernberg L. Shaping and authorising a public health profession. South Eastern European Journal of Public Health 2014;II: (SEEJPH). [Crossref]
58. Laaser U, Schröder-Bäck P. Towards a Code of Conduct for the European Public Health Profession. South Eastern European Journal of Public Health 2016;V: (SEEJPH). [Crossref]
59. Laaser U, Schröder-Bäck P, Eliakimu E, et al. A code of ethical conduct for the public health profession. South Eastern European Journal of Public Health 2017;IX: (SEEJPH). [Crossref]