KEY POINTS:

• Policy effects are defined as changes that are attributed to the use of policy instruments. 

• The study of policy effects assumes a causal model between policy instruments and policy effects. There are several reasons for caution in causal reasoning: the non-existence of a causal relationship; the risk of ignoring context; the multiplicity of policy instruments; and the openness of policy instruments. 

• Health policy output must be distinguished from health policy outcomes. Policy output corresponds with the intermediate goals of health policy and policy out­comes with its ultimate goals. 

• Political effects constitute a specific type of policy effect. They relate to the political construction of policy effects

• The effectiveness of health policy is defined as the degree to which the instruments of a policy have contributed to the achievement of the stated policy goals. 

• Policy failure can be due to theory, implementation, and compliance failures.

• Health policy can have side effects (balloon effect or waterbed effect and fill-effect).

• Two specific policy effects are counterproductive effects and distributive effects.

• Ranking health systems is a new trend in measuring health system perform­ance. 

• A specific aspect of health system performance is health system resilience which can be defined as the health system’s ability to prepare for, manage (absorb, adapt, and transform) and learn from a sudden and extreme disturbance. 

• Health policy failure can elicit a blame game (political effect)

• Political trust can be conceptualized as a political system effect. There is evidence that political trust has declined. 

Box 7.1 Success and failure of the annual global budget cap in Dutch health care An important instrument of the Dutch government to control healthcare expenditure growth is to set an annual global budget for health care to restrict expenditure growth to a predetermined level (budget cap). Up to 2012, the Minister of Health determined the budget. Ever since, it has been part of the framework agreements on the annual growth of healthcare expenditures between the government and the national organizations of insurers and providers. How did the budget cap work in practice? Figure 7.1 demonstrates that budget caps were not effective until 2012. Except for 2007, healthcare expenditure outstripped the cap each year. How­ever, since 2013 the picture has reversed. The framework agreements have proven effective in controlling healthcare expenditure growth. Source: Jeurissen & Maarse, 2021.

7.1 Introduction

Health policy is no goal of itself but a strategy to foster health system performance. Central in the study of policy effects is whether state intervention has been effective. Have the policy instruments contributed to the achievement of the stated policy goals? Referring to box 7.1, has the budget cap proven effective in keeping health­care expenditu­re in check? At first sight, this seems indeed to be the case since 2013. Nevertheless, it is always possible that factors other than the budget cap explain the observed underspending. More questions need answers to get a good picture of the policy effects. For instance, which side effects (external effects) have occurred? What are the policy’s long-term effects, and how do they compare to its short-term effects? What do the results mean for the capacity of health systems to achieve their objectives? Each of these questions fits an instrumental perspective on health policy: health policy is conceptualized as a problem-solving activity.

Health policies may also have political effects. These effects concern the political construction of the effects achieved and the impact of this construction on health policymaking. For instance, health policy failure may elicit a blame game, contribute to declining public trust in the government and science, or have electoral consequences (Bovens et al., 2011).

This chapter consists of three parts. The first part starts with briefly exploring the concept and measurement of policy effects. The second part discusses the non-political effects of health policy. It starts with an analysis of the effectiveness of health policy and the causes of policy failure. Next follows a discus­sion of the financial effects of health policy, the occurrence of side-effects, counterproductive effects, and distributive effects. These effects have in common that they are connected to a single policy. An alternative approach is to analyze the compound effect of health policymaking. What is the impact of health policymaking on health system performance and health system resilience? The political effects of health policy are central in the third part of the chapter. 

7.2 The concept of policy effect

The effect of a policy is defined as a change which is attributed to the use of policy instruments. Policy effects can be intended or unintended, expected or unexpected, direct or indirect, known or unknown, become manifest immediately or later, and so on. The concept of policy effect presupposes a causal model: the (non)occurrence of an observed change is attributed to the use of a specific policy instrument or combination of policy instruments: X (instrument) is viewed as the cause of Y (effect). There are four reasons for caution in assuming a causal relationship between instrument and effect. First, it is uncertain to what extent an observed change can indeed be attributed to the instrument used to achieve this change. Goal attainment does not guarantee policy effectiveness because other factors than the policy instrument(s) used may explain the observed change. Policymakers very seldom have the opportunity to carry out a policy experiment with a control group to investigate policy effects because they are expected to act, preferably as soon as possible. It is also uncertain whether an experiment would yield complete insight into the causal relationship between instrument and effect because of measurement problems and the impossibility of a comparable control group.

Second, simple causal models ignore context. The occur­rence of an intended change seldom results from a single factor (policy instrument). Contextual factors always influence policy effects. Policy instruments may only work under some con­ditions but not under other conditions. For instance, the state’s call for social distancing worked well in the first stage of COVID-19 but lost some of its effectiveness in later stages of the pandemic. Decentralization of health policymaking (policy instrument) to regional authorities may work in countries with a tradition of decentralized policymaking but not in countries missing such an experience. Ignoring the impact of contextual factors is an important cause of policy failure. Conversely, policy­makers sometimes benefit from the luck of a favorable con­text. However, the success of today does not guarantee the success of tomor­row. Drawing policy lessons from policy success and failure must, for this reason, include an analysis of the impact of contextual factors on the observed effects.

The third problematic aspect of simple causal models is that most health policies include a broad repertory of instruments to achieve the stated policy goals. For instance, the instruments used in cost control may consist of price controls, global budgets, co-payment regimes, hospital planning, blunt expenditure cuts, etc. A combination of instruments makes it difficult to disentangle the effects of each distinct instrument. Sometimes, lack of information on the effects of distinct instruments becomes a political issue. An example is the political controversy over some policy instruments' effect­iveness in managing the coronavirus outbreak in the Netherlands. The instrumentation of the government’s policy prompted a political debate on the (added) effectiveness of wearing face masks and freedom-restricting instru­ments such as the lockdown, QR code, and curfew. Critics requested the government to present a sound foundation of these instruments' effectiveness, but the government could only give a calculated guess of their impact on the course of the pandemic.

The fourth problem concerns the ambiguity or ‘openness’ of many policy instruments. Policy effects are influenced by how they are applied in practice. Variety in implementation practices makes it difficult to draw hard conclusions on the effects of a policy instrument. An example: the effectiveness of decentralizing policy tasks to regional or local authorities to foster the efficiency of service provision (chapter 6) is contingent on how these authorities use their discretionary power in service provision. The investigation of policy effects requires an in-depth study of policy implementation. A pocketful of money for a stated policy goal contains no information on how to spend this money. 

Policy output and policy outcomes 

Health policy effects can be divided into two main categories: policy output and policy outcomes. Policy output refers to the immediate effect of a policy instrument and policy outcomes to its ultimate effect. The intended policy output is an instrumental goal to achieve the ultimate goal. The following example illustrates the distinction between policy output and policy outcome. Over the last few decades, governments have introduced various policy instruments to discourage the use of tobacco products (Willemsen, 2018). The ultimate goal of tobacco control policy is to decrease the prevalence and incidence of tobacco-related diseases. Assume that the number of smokers has reduced by x% compared to a predetermined baseline year. This per­centage is the policy output of tobacco control policy and the observed decrease in the prevalence and incidence of tobacco-related diseases the policy outcome (Figure 7.2). The attainment of the intended policy outcome is contingent on the effectiveness of the achieved policy output. 

The distinction between policy output and policy outcomes is usually complex. Most health policies include several intermediate and ultimate goals. For instance, the government makes a budget available to improve healthcare quality, defined as shorter waiting times and fewer medical accidents. One part of the budget is spent on extension of staff and the other part on training programs. Policy output is measured as the extension of staff and attendance to training programs. The measured shortening of waiting times and decline in the number of medical accidents are policy outcomes. Another problem is that the path from policy instrument to policy outcomes may include first-order outputs, second-order outputs, and so on. For instance, concerns about food safety are the reason for the government to raise the budget of the food inspectorate (instrument) to intensify its control of food safety. The extension of the food inspectorate is the first-order output, the number and intensity of extra inspections the second-order output, and the impact of these inspections on food safety is the policy outcome.

The distinction between first-order output and second-order output demon­strates that the path from instrument to policy outcomes can consist of several consecutive steps. Each output is a crucial link in the causal chain between instrument and intended outcome. The distinction between policy output and policy outcome also contains an important lesson: the achievement of an intended output does not guarantee the accomplishment of the intended outcome. The achievement of the ultimate policy goal depends on the effectiveness of the realized policy output. In the above example, the realized extension of staff (policy output) does not guarantee food safety (policy outcome). Nevertheless, it frequently happens that the realized output is presented as evidence of the policy’s effectiveness. Policy output is used as proxy for policy effectiveness. It speaks for itself that this practice can easily lead to wrong conclusions. 

The measurement of policy effects

Many textbooks have been written about measuring policy effects and the metho­dological problems that arise in measuring these effects (Patton, 2017; Pawson & Tilly, 1997; Fischer, 1995; Cook & Reichardt, 1979). Important methodological pro­blems are the content of the analytical model and its underlying assumptions for the measurement of policy effects, the ambiguity of policy goals and policy preferen­ces used as the normative framework to assess policy effects, the operationalization of policy output and policy outcomes, the availability, validity, completeness, and reliability of the data, and the period that is taken in consideration to measure policy effects. Consequently, information about policy effects is manufactured information (see next chapter). Using other data or an alternative measurement model may produce another picture of policy effects. Because methodological choices influence the results, the results can become the object of political dispute. While some actors claim success, others are skeptical or even speak of policy failure. Material interests may also play an important role. For instance, in the years of budget overruns (Box 7.1), government and hospitals in the Netherlands frequently struggled about their actual magnitude. Hospitals rejected the government’s calculation of overspending and filed a lawsuit or threatened to do so to annul the recoup of the assumed budget overrun. 

7.3 The effectiveness of health policy

The effectiveness of health policy is defined as the degree to which the policy instru­ments have contributed to the achievement of the stated policy goals. Health policy is called successful to the extent these goals have been achieved and a failure to the extent they have not been achieved. Policy success and failure may go together: while some goals have been achieved (or only in part), other goals have not been achieved (or only in part). Whether policy effects are considered a success or failure depends upon political preferences: what one actor sees as a success, another actor may frame as a failure (Chapter 4).

As spelled out in Chapter 1, the history of health policy is a history of success and failure. Some parts of health policy have proven quite successful. The introduction of public financing based on the ability-to-pay principle has considerably contributed to universal access to health care. The eradication of smallpox and other communicable and non-communicable diseases by mass vaccination programs has also proven success­ful. Social legislation has improved working and living conditions. At the same time, there are plenty of examples of policy failure. Healthcare cost control has been less successful than policymakers hoped and has remained a great concern. Attempts to reduce health disparities have largely failed so far. The success of policy instruments to tackle the problem of overweight and obesity turns pale compared to the relative success of tobacco control instruments.

The measurement of the effectiveness of health policy is a complicated exercise because of uncertainty about what would have happened, had no policy action been undertaken. Another problem concerns the ambiguity of health policy goals. Many goals only indicate the direction of change aimed at. Enhancing the quality of health care is an example of an aspirational goal. Without clearly stated goals, the effectiveness of policy instruments cannot be determined.

Furthermore, policy instruments that work in the short may fail in the longer run. For instance, the effectiveness of cost control in health ­care has proven only temporary. In this respect, Schwartz (1987) spoke about ‘the inevitable failure of current cost-containment strategies’ because they have little or no influence on three key factors explaining real expenditure growth: population growth, higher input prices, and technological innovation and diffusion. The results achieved are small and, for the most part, only temporary. Cost control may also be a reason for the postponement of investments and lead to a cost explosion later to catch up. What also may happen is that interventions lose some effectiveness just because of their effectiveness. The effecti­veness of childhood vaccination against measles has made some parents believe that the disease had disappeared and that there was no good reason anymore to vaccinate their children.

Policy success and policy failure

Many policymakers fail to resist the temptation of dealing asymmetrically with policy success and failure. If a policy instrument has proven effective, they claim policy success. That today’s success does not guarantee tomorrow’s success is not questioned. The situation is different for policy failure. Policy failure is often ascribed to external factors such as misfortune or sabotage. Failure is not the policymaker’s fault.

Both policy success and policy failure require an explanation. Why has a policy been a success? Why did an instrument work or not work? To what extent was the success due to favorable circumstances? The causes of policy failure can be manifold. The first cause of failure is that the stated policy goals were unrealistic or only paid lip service. The choice of policy instruments may also rest upon false assumptions. Sometimes, the optimism of policymakers on the effectiveness verges on naiveté. For instance, reliance on self-regulation by the industry to stop the production of products that harm public health has, in many cases, proven naïve. More than two decades of experience with outsourcing and privatization of the production of public goods and services should have policymakers learned that these organization-based instruments may prove less successful than claimed by their advocates (Pollock, 2004; NAO, 2011).

A second explanation of policy failure is implementation failure. Policies are not implemented as intended because of a shortage of staff, lack of expertise, information problems, failing instructions, regulatory inconsistencies, lack of effective central steering, organizational rivalries, or other factors. Implementation failure may also be due to the policymakers’ ignorance, underestimation, or plain disinterest in imple­mentation issues. Political compromises often appear as a source of implementation problems.

The third explanation is compliance failure: the target population (policy sub­jects) did not respond to policy instruments as expected. For instance, the failure and success of voluntary mass childhood vaccination programs are con­tingent on the parents’ willingness to have their children vaccinated. Other causes of compliance are lack of information and lack of bureaucratic competence. In practice, most implementation and compliance failures ensue from policy failure.

7.4 Side effects 

Many policy instruments have unintended side effects (external effects). They can be foreseen or unforeseen and assessed as either positive or negative. Negative side effects or the risk of negative side effects such as precedents is an argument for policymakers to abstain from certain instruments or take measures to minimize their occurrence.

There are many types of side effects. While some are immediately visible, such as the economic and social consequences of lockdowns (for instance, the closure of bars, restaurants, museums, or the ban on sports matches with spectators), others become manifest only in the long run. An example of a side effect that became manifest later is the occurrence of mental problems in the aftermath of COVID-19 (Bourmistrova et al., 2022; RIVM, 2022a). Some side effects are directly visible but taken for granted because of the urgency of other problems. The Netherlands Institute for Public Health and the Environ­ment has calculated that, due to the priority given to COVID-patients, about 305,000 non-urgent operations had been post­poned in 2020 and 2021 (about one-sixth of the expected number of operations). The loss of life-years in good health as a consequence of postponed operations was estimated at 320.000 (RIVM, 2022b).

A well-known dilemma in health policymaking is the occurrence of a conflict between collective and individual interests. The announcement of a lockdown served a collective interest (arresting the spread of the coronavirus) but restricted personal freedom (negative side-effect). Health promotion programs may simultaneously im­prove public health (intended effect) and enlarge health disparities. The explanation for this unintended side effect is that persons with higher education tend to be more responsive to these programs than persons with lower education.

Some side effects wipe out the intended policy effects. An example is when policy interventions directed at cost control in a specific healthcare sector generate higher costs at a later moment or in another field of health care. This effect is known as the balloon effect or waterbed effect. Co-payments lower healthcare expenditures (intended effect) in the short but may cause higher expenditures later because sick people have abstained from necessary care for financial reasons (Van Esch et al., 2017). The introduction of a co-payment regime for mental care in the Netherlands caused a drop in the demand for mental care (intended effect) and simultaneously an increase in expensive crisis interventions and compulsory admissions (Rave­steijn et al., 2017). The introduction of a co-payment for prescription medicines for hypertension in 1983 in Dutch health care had a double effect: the number of prescriptions per patient dropped by 20 percent in the year of introduction but the number of medicines per prescription increased by 12 percent (Starmans, 1998).

The fill effect is another type of side effect that may cancel out the intended effect. An example is when health professionals start new activities to compensate for the loss of revenues (Klink et al., 2017) or when the tobacco industry responds to tobacco control policies by exploring new markets.

The risk of unintended effects has been a reason to criticize competition in health care from a moral perspective. The commodification of medical care, the argument goes, will invoke value drifting. Money-making will get priority over good treatment. These developments undermine the trust relationship between doctor and patient (Berenson & Cassel, 2009; Pellegrino, 1999). Competition is not a morally-free zone. As Sandel (2012) formulated this problem in his study on the moral limits of markets: ‘Putting a price on every human activity erodes certain moral and civic codes worth caring about’ (p. 121). 

7.5 Counterproductive effects

Sometimes, policy instruments have effects that are opposite to their intended effects. An example of counterproductive effects is an experiment at six daycare centers in Haifa which struggled with the problem that parents were late picking up their children at the end of the day. The solution was sought in imposing a fine on these parents. The dis­incentive failed to work and had even a counterproductive effect because parents reacted to the fine by doubling the time they arrived late. After the centers had decided to revoke it, the parents’ enhanced tardiness persisted (Gneezy & Rustichini, 2000).

De Bruijn (2007) has described various dysfunctions of performance measu­rement. This instrument is used to inform organizations about their performance compared to other organizations and motivate them to (further) improve their performance. Performance measurement can be used to reward well-performers and punish poor-performers. The risk is that the instrument induces organizations to develop strategic adaptive behavior, for instance, by brushing up their performance or giving priority to measured activities while neglecting other ones. Another potential dysfunctional effect is that performance measurement undermines profess­ionality. Each of these effects obfuscates the effectiveness of performance measurement and may lead to disasters and scandals (Box 7.2).

7.6 Distributive effects

Policy effects may differ per group and region. Some groups may benefit more from an instrument than other groups. The enhanced freedom of choice of consumers in Dutch health care after the introduction of the market reform in 2006 illustrates the occur­rence of a distributive effect. The new Health Insurance Act allows consumers to switch to another insurer by the end of each calendar year. Insurers are obligated to accept each applicant without restriction. There is evidence that persons in the age category 18-39, persons with higher education, and persons perceiving their health as good have switched rela­tively more frequently than older persons, persons with low education, and persons perceiving their health as poor (De Jong et al, 2015). This result indicates that the first category has benefitted relatively most from their increased freedom of choice. There is also some evidence that voluntary deductibles elicit adverse selection. Persons with higher previous and future healthcare costs are less likely to choose a €500 deductible. Some groups suffer more from policy instruments than other groups. The impact of a co-payment regime to discourage the use of unnecessary care is likely to be stronger in low-income groups than in high-income groups. Distributive effects can also occur as a consequence of post-code rationing. A potential effect of decentralization of the provision of health services is that some local governments follow a less strict need assessment procedure than other governments. 

7.7 Administrative costs

Nowadays, the attention to the administrative costs of health policy is growing. Contract negotiations with multiple insurers, complex regulations, procurement proce­­dures, activi­ty-based funding models and recurrent revisions of these models, com­plex accounting procedures, risk reduction, supervision, and the detection of in­appro­­priate care or fraud are frequently mentioned as factors pushing up administrative costs. Unfortunately, the measurement of administrative costs is problematic because they are not only made by typical administrative bodies (e.g. Health Depart­ment, regulatory agencies, advisory bodies, or the administrative department of healthcare providers) but also by caregivers at the shop floor who must record their activities, fill in forms, follow instructions, and so on. Administrative costs have a multi-level structure. While some of these costs are visible and easily measurable, others remain obscured (Hage­naars, 2021).

The size of administrative costs is a research topic in international comparative studies. In their comparison of the gap in health administrative spending between the United States and Canada, Himmelstein and his co-authors (2020) conclude that this gap reflects the inefficiencies of the United States’ market-based healthcare system. While Canada’s administrative costs in 2017 amounted to 551 dollars per capita, insurers and providers in the United States spent more than four and a half times more per capita (2497 dollars). The fraction of administrative costs in US health spending was 34.2% and in Canada only 17%. The prices of medical care in the United States comprise a substantial surcharge to cover their administrative burden. The authors argue that the gap in administrative costs widened between 1999 and 2017. They ascribe this gap to the efficiency of Canada’s single-payer system and the inefficiency of the multi-payer system in the United States.

High administrative costs are a source of frustration among caregivers, not only because administrative activities crowd out the time for patient care but also because they perceive many of these activities as of low value. Health professionals in two Dutch academic hospitals and one teaching hospital said to spend 52.3 minutes daily on quality registrati­ons. The average number of quality measures was 91, with 1380 under­lying variables. Only 36% of these measures were perceived as useful (Zegers et al., 2021). 

7.8 Health system performance

An alternative strategy to investigate health policy effects is to investigate the com­pound effect of policy instruments on health system performance. Health system performance can be described as the degree to which health systems achieve the stated policy goals. The purpose of performance measurement is ‘to monitor, evaluate and communicate the extent to which various aspects of the health system meet key objectives’ (Smith et al., 2009). They mention the following list of headings under which these objectives can be summarized: population health, patient-reported outcomes, clinical quality and appropriateness, financial protection, health systems responsive­ness, equity of access to health care, and finally, productivity and efficiency (p. 8). Performance measurement is intended to inform health systems as well as health organizations about their performance and how their performance compares to the average or best performance.

Table 7.1 Average healthcare consumption and burden of finance per person and education, Netherlands, year 2011

Source: CPB, 2013.

Table 7.1 is an example of how health policymaking affects health system performance. The table exhibits how Dutch health policymaking plays out in terms of per capita consumption of health services per level of education and the per capita distribution of the financial burden of health care. The consumption of health services is influenced, among others, by the benefits catalog of statutory health insurance and long-term care legislation, and entitlement criteria. The distribution of the burden of finance results from the complex set of regulations concerning premium setting, social contributions, subsidy instruments, and co-payments. The table demonstrates, not surprisingly, that persons with low education consume on average more health services than persons with high education. Conversely, the burden of finance is highest for persons with high education. The regulation of the burden of finance has a redistributive effect. This effect is strongest for long-term care. Notice that the distribution of consumption and finance is not only influenced by health insurance legislation but also by various contextual factors, in particular the distribution of health and illness across the population.

Ranking the performance of health systems

A prominent research strategy in performance measurement is to compare the per­formance of health systems to find out which system performs best and learn from the best performers. Comparative health system performance research and system ranking has become a new trend in health policy analysis. An example of measuring and ranking the performance of 191 countries was undertaken by the World Health Organization in its report ‘World Health 2000’ (WHO, 2000). The researchers used five indicators to measure system performance: health status, health distribution, responsiveness level, responsiveness, and fair financing. The scores on each indicator were based on available statistical data per country and a non-representative internet-based questionnaire among WHO staff and people who had visited the WHO website. The critical step in the measurement procedure was the construction of a composite index to calculate a performance score per country. The report published two scores: overall health score and overall health system performance score. France scored best in terms of overall system performance and fourth best in overall health performance.

The report has been heavily criticized. The main points of critique were: the choice of indicators, the quality of data used, and the opaque construction of the composite indexes. The performance scores were utterly artificial. In a critical review of the report, Williams (2000) concluded that the report was ‘not robust enough to support the flimsy structure that has been created from it. The underlying database is skimpy and of dubious quality (p. 10). He did not believe that the report had any policy-learning potential.

An alternative attempt to construct a league table of health systems is the Euro Health Consumer Index, published by the Health Consumer Powerhouse since 2005. The ranking is based upon six groups of indicators: patient rights and information (10 indicators), accessibility/waiting time (6 indicators), outcomes (9 indicators), range and reach of services (8 indicators), prevention (7 indicators) and pharmaceuticals (6 indicators). There are three possible scores for each indicator: good (three points), so-so (two points), and not-so-good (one point). The relative weight per group of indicators varies from 100 points (pharmaceuticals) to 300 points (outcomes). The maximum score a country can attain is 1000. Figure 7.3 presents the ranking of countries for 2018. 

Though the EHCI uses more indicators than the World Health Report 2000, the methodological pitfalls of constructing league tables are similar. The list of indicators is biased toward medical care, and only one indicator explicitly refers to long-term care.

In their critical assessment of using composite indicators to measure the perform­ance of healthcare systems, Goddard and Jacobs (2009) identify serious methodological problems with composite indicators to measure healthcare system performance. Major problems are the choice of units to assess and organizational ob­jectives to encompass; the choice of indicators (data availability, type of indicators, collinearity between indicators, and combining indicators to create a composite); the trans­formation of individual indicators (weighting, decision rules to assign scores). They underscore that a single indicator has some advantages because it gives quick insight and probably captures policy attention more quickly than measuring the performance level by many diverse indicators and facilitates communication about performance issues with the public. Nevertheless, a single score remains an oversimplification of the complexity of healthcare systems and possibly masks serious shortcomings in health care. Moreover, a single score is not helpful from the viewpoint of policy learning because it does not inform policymakers of the source of failures and the remedial action required.

A final example of a cross-national comparison of healthcare system perfor­mance is presented in the report ‘Health at a Glance’ published yearly by the Paris-based Organization of Economic Co-operation and Development (OECD). In its reports, the organization abstains from constructing composite indexes to measure system performance. Instead, the report presents a number of ‘country dashboards’ in which coun­tries per indicator are classified as better, worse, or in close distance with the OECD average (measured by the standard deviation from the average). The 2017 report presents dashboards on five classes of indicators (table 7.2).

Table 7.2 OECD-dashboards for health system performance measurement

Health system resilience

An alternative way to investigate health system performance is to focus on health system resilience. Health system resilience has been defined as the ‘health system’s ability to prepare for, manage (absorb, adapt, and transform) and learn from a sudden and extreme disturbance’ (Sagan et al., 2022). The focus here is on the ability of health systems to respond effectively to sudden crises and how health system resilience can be strengthened.

The attention of health policy analysts to health system resilience has strongly increased in the aftermath of the COVID-19 pandemic. In its study of how governments have responded to COVID-19, the European Observatory on Health Systems and Policies has presented a scheme for evaluating the resilience of health systems. Based upon an extensive review of the strategies from Europe and beyond, the Observatory identified twenty key strategies to enhance resilience during COVID-19. Nine strategies relate to leading and governing the COVID-19 response; three strategies to the financing of COVID-19 services; three strategies to mobilizing and supporting the health work­force; three strategies to strength­en­ing public health interventions; and two strategies to transforming the delivery of health and social care services to address COVID-19 needs (Sagan et al., 2022). 

7.9 Political effects

All effects discussed so far fit in an instrumental perspective on health policy. The leading question was: has the policy worked as intended, and is there evidence of unintended effects? The investigation of policy effects should provide knowledge on how to strengthen the problem-solving capacity of health policy and how its potential negative side effects can be averted. These effects must be distinguished from political effects, which relate to the political construction of policy effects and its consequences for policymaking.

The political effects of health policymaking can take on many forms. An example is the impact of the influenza pandemic in 1918 on the rise of fascism in Italy in the early 1920s. Using a multivariate regression model, Galofré-Vilà and his colleagues (2022) found a remarkable correlation between the number of influenza deaths per capita in 1918 and the vote share of the Fascist Party in 2024 in seventy cities. The researchers also presented some historical evidence based on a qualitative archival analysis of the newspaper Il Populo d’Italia from June 1, 1918, until July 31, 1919, to underpin their conclusion. The rise of fascism suggests that voters held the government politically responsible for the dramatic consequences of the pandemic on social and economic life. Note, however, that methodological limitations make the researchers cautious in drawing firm conclusions about the impact of the influence of the pandemic on the rise of fascism. They claim no hard evidence for causation.

In their study ‘Deaths of Despair’, Case and Deaton (2019) also suggest a relationship between health and voting behavior. Using observational data, they conclude that ‘the fraction of people in an area who voted for Donald Trump in 2016 is also strongly correlated with the fraction in pain’ (p. 87). The more people reported pain in an area, the higher the probability that Donald Trump won in that area. For obvious reasons, the correlation is no hard evidence for a causal relationship between pain and voting behavior. Voting behavior is influenced by many factors. However, the research­ers draw attention to the fact that pain correlates with many distressing factors, joblessness, broken families, addiction to pain killers, and little perspective toward a better life. It is despair that influences voting behavior.

Political effects are sometimes closely associated with policy scandals and crises. Examples are the fall of the government, broad media coverage, public outrage, blame games, and legal action (Bovens et al., 2001). An instructive example is the political investigation of the blood scandal in France that occurred in the 1980s and drew much public and media attention. Eventually, the Penal Court investigated whether three former ministers could be held accountable for the scandal (Box 7.3).

Political effects are always influenced by the political context. Political opponents may frame policy failures as just another manifestation of the overall incompetence of the incumbent government to resolve public problems.

Policy failures in a polarized political atmosphere are a reason for a political hunt on policymakers and other office-holders who are held personally accountable for what has gone wrong. The political mechanism consists of four elements: (a) policy failures have causes; (b) these causes are traceable to individuals; (c) these individuals must be held accountable for policy failures; (d) they must be punished personally for their failures by resignation or prosecution if their actions were unlawful or exhibited gross neglect of duty (as in the contaminated blood scandal in France). Boin and ‘Hart (2009) speak about the rise of an ‘inquisition democracy’ in which personal attacks and blame games have become the new normal. However, the rise of an Inquisition democracy as new political culture is not without risks. The focus on the accountability of individual persons can undermine a serious investigation of the structural causes of policy failures and, consequently, policy learning. Another risk is the erosion of political trust in the state and public policy. 

7.10 Political trust

The level of political trust indicates how citizens evaluate the performance of political institutions. Van der Meer (2017) defines the concept as ‘citizens’ support for political institutions as government and parliament in the face of uncertainty or vulnerability to the actions of these institutions’. He conceptualizes trust as ‘a relational concept that links the subject (who trusts) to the object (that is trusted)’. Trust has four elements: ‘(a) trust in the objects competence to act in the subject’s interest; (b) trust that the object is benign to the subject; (c) trust that the commitment of the object can be enforced by the subject or that the object can be otherwise held accountable; (d) an trust that the of the object is predictable.’ Van der Meer emphasizes that ‘the absence of trust should not simply be equated to the presence of distrust. A crucial middle category is made up by the category of skepticism, the attitude to suspend judgment awaiting additional information. Political cynicism, by contrast, is the attitude that assumes the worst of the nature of political objects (actors, institutions) as reflected in their per­ceived incompetence and selfishness’.

An interesting question concerns the degree of political trust in the government’s policy to manage the coronavirus outbreak. The picture is diverse. A cross-country survey in June 2021 in Europe found that 46% of Europeans were very or fairly satisfied with how their government handled the pandemic; 49% said to be dissatisfied. Satisfaction was highest in Malta (75%), Portugal (75%), and Ireland (68%) and lowest in the Czech Republic (40%), Slovakia (40%), France (36%), and Germany (33%) (Flash Euro­barometer Survey, June 2021). Public support for the UK govern­ment’s handling of the pandemic showed a persistent gradual decline throughout 2021. The inability to sustain the elevated political trust at the onset of the pandemic had made the management of the pandemic increasingly challenging (Davis et al., 2021).

According to Krastev and Leonard (2021), Europeans were divided over what they believed to be the government’s motivations behind restrictions to control the pan­demic. They observed a generational divide, with the young more likely to blame governments for the impact of the pandemic than the elderly. They also reported notable cross-country differen­ces in how the population judged the government’s motivations behind its lockdown measures (Figure 7.4). In this respect, they dis­tinguished between the trustful who have faith in government, the suspicious who believe rulers want to cover up their failings, and the accusers who believe that governments seek to increase their control over people. In most countries, the percentage of persons trusting the government’s motivations was more than 60%; the lowest percentages were found in France, Bulgaria, and Poland. Conversely, these countries had the highest percentage of suspicious persons. 

Empirical research indicates a partisan divide in how people assess the government’s policy measures to manage COVID-19. International comparative research found that approval ratings of the government’s policy measures correlate to differences in political support and pre-pandemic approval ratings (Chen & Fan, 2022). Likewise, the Pew Research Center found wide differences between supporters of the Republican Party and the Democratic Party in the United States over the threat to public health from the coronavirus outbreak. While 82% of the ‘Democrats’ considered the outbreak in February 2021 a significant threat, only 41% of the Republicans agreed (Dean et al., 2021). 

The (declining) level of political trust in the government’s handling of COVID-19 is no isolated phenomenon. It must be understood as part of a broader political development that can be described as declining trust in overall government policy. Figure 7.5 shows an even split between those who trust the government and those who distrust the governance. 

7.11 Conclusion and suggestions for health policy analysis

The investigation of policy effects forms an important part of the task of health policy analysts. The central question is to what extent the goals of state intervention have been achieved or, put differently, to what extent state intervention has been effective. However, the analysis of policy effects should go beyond an analysis of effectiveness and include other effects as well, such as costs, side effects, long-term effects, distributive effects, and counterproduc­tive effects. Knowledge of these ‘other’ effects may throw a different light on the results. Health policy analysts should also wonder why observed changes are assessed as positive or negative (or a combination of positive and negative). Another issue concerns the assumed causal relationship between inter­vention and observed change. How to judge the validity of the assumed causal relationship, and which uncer­tainties exist in this respect? Which (accidental) contextual factors have influenced the observed results? To what extent are the observed changes an artifact of the measurement model? Answers to these fundamental questions should protect policymakers against false conclusions.

The study of the effectiveness and other effects of policy instruments fits in the instrumentalist approach to health policymaking. Health policymaking consists of interventions directed at achieving a desired situation, but these interventions can also have unexpected and undesired effects. Another category of effects of interest for health policy analysts is political effects. How is state intervention appreciated by policy clients or the population? Do health and health policymaking influence people’s trust in government and science? What is its impact on voting behavior? Does policy failure have political consequences? Policy analysts must make policymakers aware of the potential political effects of (the absence of) health policy interventions.

References Chapter 7

Berenson N, Cassel K (2009). Consumer-Driven Health Care May Not Be What Patients Need -- Caveat Emptor. JAMA, January 21; 301(3): 321-323. doi: 10.1001/jama.2008.994.

Boin A, ‘t Hart P, McConnell A (2009). Crisis Exploitation: Political and Policy Impacts of Framing Contests. European Journal of Public Policy, 16(1): 81-106. doi.org/10.1080135017608 024 53221

Bourmistrova N, Solomon, T, Braude Ph, Strawbridge R, Carter B (2020). Long-term Ef­fects of COVID-19 on Mental Health: A Systematic Review. Journal of Affective Disorders. Feb 15;299:118-125.doi: 10.1016/j.jad.2021.11.031

Bovens M, ‘t Hart P, Peters B (2001). Success and Failure in Public Governance: A Comparative Perspective. Edward Elgar.

Bowles S (2016). The Moral Economy. Why Good Incentives Are No Substitute For Good Citizens. Yale University Press.

Case A, Deaton A (2019). Deaths of despair in a capitalist society. Princeton University Press.

Chen C, Fan T (2022). Public Opinion Concerning Governments’ Response to the COVID-19 Pandemic. PLoS One. Mar 2; 17(3):e0260062.doi: 10.1371

Cook Th, Reichardt Ch (1979). Quantitative and Qualitative Methods in Evaluation Research. SAGE.

CPB (2013). De Prijs van Gelijke Zorg [The price of Equal care]. The Hague

Croes R, Schut F, Varkevisser M (2019). Is Adverse Selection Effectively Mitigated By Consumer Inertia? Empirical Evidence from the Dutch Health Insurance Market. In: Croes R, editor. Market Power in Hospital Markets and Selection in Health Insurance Markets. Rotterdam: Erasmus University Rotterdam; 151–173.

De Bruijn (2007). Managing Performance in the Public Sector. Routledge

De Jong J, Brabers A, Bouwhuis S, Bomhof M, Friele R (2015). Het Functioneren van de Zorg­ver­zekeringsmarkt: Een Kennissynthese [The Functioning of the Health Insurance Market: A Knowledge Synthesis] Utrecht: NIVEL

Dean C, Parker K, Kramlich J (2021). A Year of U.S. Public Opinion on the Coronavirus Pan­de­mic. https://www.pewresearch.org/2021/03/05/a-year-of-u-s-public-opinion-on-the-corona virus -pandemic

Esch T van, Brabers A, van Dijk C, Gusdorf L, Groenewegen,P, De Jong J (2017) Increased Cost Sharing and Changes in Noncompliance With Specialty Referrals in The Nether­lands. Health Policy, 121(1): 180-188. doi: 10.1016/j.psychres.2021.113819.

Evans S, Alkan E, Bhangoon J, Tenenbaum H, Ng-Knight T (2021). Effects of the COVID-19 Lockdown on Mental Health, Wellbeing, Sleep, and Alcohol Use in a UK student Sample. Psychiatry Research. Apr;298:113819.doi: 10.1016/j.psychres.2021.113819

Fischer F (1995). Evaluating Public Policy. Nelson-Hall Publishers.

Galofré-Vila G, McKee M, Gomez-Leon M, Stuckler D (2022). The 1918 Influenza Pandemic and the Rise of Italian Fascism: A Cross-city Quantitative and Historical Text Qualitative Analysis. American Journal of Public Health, 112(2):242–247. doi: 10.2105/AJPH.2021.306574.

Gneezy U, Rustichini A (2000). Pay Enough or Don’t Pay At All. Quarterly Journal of Economics, 115(2): 791-810. http://www.jstor.org/stable/2586896

Goddard M, Jacobs R (2009). Using Composite Indicators to Measure Performance in Health Care. In: Smith P, Mossialos E, Papanicolas I, Leatherman S (eds), Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects. Cambridge University Press: 339-368.

Hagenaars L (2021). Reducing Paper and Sugar for Fiscally Sustainable Health Care Systems: Studies on Administrative Costs and Junk Food Tax Policies. Radboud University (dissertation).

Himmelstein D, Campbell T, Woolhandler S (2020). Health Care Administrative Costs in the United States and Canada, 2017. Annual Internal Medicine, January 21, doi: 10.7326/M19-2818. 172(2): 134-142. doi: 10.7326/M19-2818.

Krastev L, Leonard M (2021). Europe’s InvisibleDivides: How COVID-19 is Polarizing European Politics. European Council on International Relations (policy brief).

Jeurissen P, Maarse H (2021). The Market Reform in Dutch Health Care. European Observatory on Health Systems and Policies.

NAO (National Accounting Office) (2011). Lessons from PFI and other projects: Report by the Comptroller and Auditor General. NAO (report)

Patton M (2017). Utilization-focused Evaluation. SAGE (4^th edition).

Pawson R, Tilly N (1997). Realist Evaluation. SAGE.

Pellegrino E (1999). The Commodification of Medical and HealthCare: The Moral Consequences of a Paradigm Shift from a Professional to a Market Ethic. Journal of Medicine and Philosophy, 24(3): 243-266. doi.org/10.1076/jmep.24.3.243.2523

Pollock A (2004). NHS Plc: The Privatisation of our Health Care. Verso.

OECD (2020). Health at a Glance 2020. Paris: OECD.

OECD (2022). Building Trust to Reinforce Democracy. Paris

Ravesteijn B, Schachar E, Beekman R, Janssen R, Jeurissen P (2017). Association of Cost Sharing With Mental Health Care Use, Involuntary Commitment and Acute Care. JAMA Psychiatry, 74(9): 1713-1723. doi: 10.1001/jamapsychiatry.2017.1847.

RIVM (2022a). De Gevolgen van de Coronapandemie voor de Gezondheid en het Welzijn van deJeugd. Een Systematische Literatuurstudie [The Consequences of the Corona Pandemic on the Health and Well-being of Youngsters. A literature Study]. Bilthoven.

RIVM (2022b). Operations Postponed Due to Pandemic Result in 320,000 Healthy Life Years Lost. Bilthoven.

Roemer M (1961). Bed Supply and Utilization: A Natural Experiment. Hospitals: 35: 36-42.

Rose G (1981). Strategy of Prevention: Lessons From Cardiovascular Disease, British Medical Journal, 282: 1847-1851. doi: 10.1136/bmj.282.6279.1847.

Sagan A, Webb E, Azzopardi-Muscat N, de la Mata I, McKee M, Figueras J. (eds) (2021). Health Systems Resilience During COVID-19: Lessons for Building Back Better. Copen­hagen: WHO Regional Office for Europe on behalf of the European Observatory on Health Systems and Policies.

Sandel M (2012). What money can’t buy: the moral limits of markets. Farrar, Strauss and Giroux.

Schakel Chr (2020). Fiscal rules for Health. Nijmegen: Radboud University (dissertation).

Schwartz F (1987). The inevitable failure of cost containment strategies. Why they can provide only temporary relief. Journal American Medical Association doi:10.1001/JAMA.1987. 03390020086032

Smith P, Mossialos E, Papanicolas I, Leatherman S (200). Performance measurement for health system improvement. Experiences, challenges and prospects. Cambridge Univer­sity Press.

Starmans B (1998). The use of health care facilities in the Netherlands: Dutch investi­gations and international evidence. Maastricht University (dissertation)

Steffen M (2001). Crisis governance in France: the end of sectoral capitalism. In: Bovens M, ‘t Hart P, Peters B (eds), Success and failure in public governance. Edward Elgar: 470-488.

Van der Meer T (2017). Political trust and the ‘crisis of democracy ’ https://doi.org/ 10.1093/ acrefore/9780190228637.013.77

Williams (2001). Science or marketing at WHO? A Comment on World Health 2000. Health Economics, 10: 93-100. doi: 10.1002/hec.594.

Willemsen M (2018). Tobacco control policy in the Netherlands. Palgrave.

WHO (World Health Organization) (2000). The World Health Report. 2000. World Health Organiza­tion.

Zegers M, Veenstra GL,Gerritsen G, Verhage R, Hoeven JG van der, Welker GA (2020). Perceived burden due to registrations for quality monitoring and improvement in hospitals: a mixed methods study. International Journal of Health Policy and Management, https://dx.doi.org/ 10.34172