1.

Policy formulation: where knowledge meets power

in the policy process
Michael Howlett and Ishani Mukherjee

INTRODUCTION: DEFINING POLICY FORMULATION

Public policies emanate from societies’ efforts to affect changes in their own institutional
or public behaviour in order to achieve some end goal key policy actors consider to be
important. Such policies are determined by governments but involve other actors and
institutions – private, commercial, family and others – in often complex governance and
governing arrangements and relationships (Howlett & Ramesh, 2016).
Policy formulation is part of the process of developing public policies and involves
governments and other policy actors asking and answering questions about how societies
can deal with various kinds of problems and conditions affecting citizens and organiza-
tions in the pursuit of their goals. These questions vary in range and scope, but address-
ing them typically involves deliberations among a wide range of actors about what kinds
of activities governments can undertake, and what kinds of policy instruments or levers
they can employ, in crafting solutions for the public and private dilemmas they identify,
or consider to be, policy problems. Some problems may defy solution, such as poverty or
homelessness in many countries and jurisdictions, and others may be more easily resolv-
able. But whatever solutions emerge from formulation activity are the basis of what, once
adopted, becomes a public policy.
The exercise of matching policy goals and means is thus central to the tasks and
activities of policy formulation. This is not a neutral or ‘objective’ or technical process,
of course, although it may sometimes be viewed in this way. As one of the earliest pro-
ponents of the policy sciences, Harold Lasswell, stated in 1936, it is a political activity
thoroughly immersed and grounded in questions about ‘who gets what, when and how’
in society (Lasswell, 1936).
In this light, the formulation of policies, or the matching, and often mis-­matching,
of goals and means, or policy aims and instruments, occurs through the interplay of
knowledge-­based analytics of problems and solutions with power-­based political consid-
erations. It emerges through the interaction of technical analyses of goals and instruments
and the political assessment of the costs and benefits to particular actors, the partisan
and electoral concerns of governments, and the realm of ideas and beliefs held by political
actors as governments attempt to articulate feasible policy options capable of resolving
problems and meeting social goals with at least a modicum of social and political support.
That is, all of this activity occurs within the context of the need to meet and placate
the diverse interests of the public, social actors and their administrations. As a result,
this process often ends in complex assemblages or mixes of policy aims and policy tools
that are somewhat unique to each jurisdiction and may or may not embody much in the
way of ‘technical’ merit (Howlett & Cashore, 2009). However, in this formulation process

3
4  Handbook of policy formulation

and in the subsequent implementation or evaluation of policies, governments can and do

learn from their own and others’ experiences and can often improve their performance to
more effectively attain their aims and goals. Such trial and error, as well as the ability of
governments and formulators to learn directly or indirectly from other governments and
societies’ experiences, allows them to engage in emulation and other forms of interaction
that can serve to reduce the differences found between jurisdictions and can lead to large
similarities in policy design and content among even the most diverse societies (Bennett,
1991; Bennett & Howlett, 1992).

POLICY FORMULATION AS A (SUB-­)PROCESS OF

POLICY-­MAKING

In general, ‘policy formulation’, or the activity of finding, devising and defining problem
solutions, takes place once a public problem has been recognized as warranting govern-
ment attention. Formulation thus follows an initial ‘agenda-­setting’ stage of policy-­
making and entails the various processes of generating options about what to do about an
identified and prioritized problem. During this period of policy-­making, policy options
that might help resolve issues and problems recognized at the agenda-­setting stage are
identified, refined and formalized. Formulation activities are thus distinct from other
aspects of policy-­making that involve authoritative government decision-­makers choos-
ing a particular course of action, or the actual implementation of the policy on the ground
(Schmidt, 2008).
This provision of solutions to problems is a complex matter and in practice the
development of options and alternatives to specific kinds of problems facing societies
and governments in think tanks, research institutes and other venues often precedes
the articulation of problems by a particular government. Hence, governments often
find themselves in the position of being either leaders or laggards in recognizing and
addressing problems and discussing or implementing possible solutions (Gunningham
et al., 1998). While there are some advantages to being leaders, there is a greater risk
of failure with innovating problem definitions and solutions. Laggards can benefit
from both the positive and negative experiences of leaders and often can inherit an
already well-­discussed and elaborated set of policy solutions when they do eventually
turn to address a particular problem already dealt with in other jurisdictions (Béland
& Howlett, 2016). Thus, a major component of the policy formulation activity under-
taken by governments is the monitoring of events in other jurisdictions, and even other
branches of the same government, to see how various efforts and tools aimed at provid-
ing solutions to problems have fared.

POLICY FORMULATION AS A FIELD OF POLICY STUDY

Policy scholars have always been interested in how policy instruments fare and how suc-
cessful a government has been in their creation and deployment, but the literature on
policy formulation has thus far remained quite rudimentary and fragmented (Sidney,
2007).
 Policy formulation  5

The study of policy tools and their design has been one major venue for building
­ nowledge about policy formulation (Salamon, 1989, 2002). Policy instrument studies
k
have over the last several decades been concerned with what Cochran and Malone (1999)
deem to be the substantive ‘what’ questions of policy-­making. That is, ‘What is the
plan for dealing with the problem? What are the goals and priorities? What options are
available to achieve those goals? What are the costs and benefits of each of the options?
What externalities, positive or negative, are associated with each alternative?’ (Cochran &
Malone, 1999, p. 46). In parallel with this effort, the study of policy design has also dealt
with the ‘how’, or the procedural and process-­oriented questions about how best to for-
mulate policy solutions and how such solutions have evolved over time and spread over
space (Considine, 2012; Howlett, 2000; Linder & Peters, 1990; Schneider & Ingram, 1997).
Much useful knowledge about processes and designs has emerged from these studies
(Howlett et al., 2015).
Key questions regarding the different actors involved in the process of policy formula-
tion and the capacity and extent of their involvement, on the other hand, until recently
have remained relatively less well explored and their findings more poorly integrated
(Howlett & Lejano, 2013). This Handbook makes a novel contribution in exploring both
the existing strong and weakly addressed subjects of policy formulation by examining the
analytics as well as the politics entailed in these processes, and by bringing together for
the first time a wide range of research findings on the subject.

ORGANIZATION OF THE HANDBOOK

Using the various questions about policy formulation posed above as a guide, this
Handbook unites scholarship on policy tools and design with that examining policy
actors and the roles they play, why and with what effect, in the formulation process.
The contributions in Part I situate policy formulation in the greater policy process
in order to set the context for the remaining chapters and parts. Part II then deals with
the ‘what’ and ‘how’ components of formulation, detailing the substantive and process-­
oriented considerations concerning what goals are defined during formulation and how
policy artefacts – such as policy instruments and their combinations in policy mixes and
programmes – are created to address them. Part III continues the focus on the ‘how’ ques-
tions and picks up on the operational facets of formulation and its modes of analyses,
including chapters on the participation of non-­state actors in policy creation. This focus
on the agency or ‘who’ of policy formulation is further developed in Part IV, where several
chapters discuss the various actors who supply information, knowledge, policy advice and
enterprise during the formation of policy. The contributions in Part V then look more
closely at the role that experts of different kinds play in defining policy problems through
their collective activities as epistemic actors who, among other things, influence how
policy targets are defined and considered during formulation.
In Part VI, the contribution of various actors in proposing policy solutions is explored.
This part examines the role of ‘instrument constituencies’, think tanks and research
organizations in shaping and disseminating policy alternatives, while also discussing the
institutional and behavioural aspects of their activities that influence their ability to shape
formulation.
6  Handbook of policy formulation

Delving deeper into the politics of policy formulation, Part VII includes a discussion
of policy paradigms and the political economy that inform and influence the nature of
formulation activities including the kinds of ideas and beliefs that shape policy alterna-
tives and considerations and determinations of their acceptability or political as well as
technical ‘feasibility’. The influence of political parties as well as interest groups is also
examined in this part.
Part VIII concludes with chapters that discuss trends in contemporary policy formula-
tion research and practice such as the significance of movements towards the increased
‘politicization’ of policy advice, the struggle between proponents of experience-­versus
evidence-­based formulation processes, the changing role of media in influencing and
informing policy actors, as well as the phenomenon of the intentional disproportionality
of government responses to problems in many circumstances.

WHAT DOES ‘POLICY FORMULATION’ MEAN?

As set out above, formulation is that stage of policy-­making where a range of available
options is considered and then reduced to some set that relevant policy actors, especially in
government, can agree may be usefully employed to address a policy issue. This generally
occurs before the issue progresses onwards to official decision-­makers for some definitive
determination, although those decision-­makers may have in their public pronouncements
or electoral platforms and other statements already signalled which kinds of efforts they
might countenance and which they would not.
Formulation activity hence entails not only calculations of the relative benefits and
risks of the various policy means that can be considered to match stated policy goals but
also their potential feasibility or likelihood of acceptance and thus involves both a techni-
cal as well as a political component.
That is, once a social problem has been elevated to the formal agenda of the govern-
ment, policy-­makers are usually expected to act in devising alternatives or potential
solutions to it. Although they may ultimately do nothing or react in a purely symbolic
way, as Charles Jones (1984) highlighted, the essence of policy formulation is simply
that various ways to deal with societal problems are proposed and deliberated upon by
government officials and others knowledgeable about the problem. Their proposals for
action or inaction may come about during the initial agenda-­setting discussions, during
which a policy problem and a possible solution can become coupled on the government’s
agenda (Kingdon, 1984); they may also arise from past efforts, successful and otherwise,
in dealing with an issue.
This depiction paints formulation as involving several disjointed activities within a
larger policy-­making process that will be carried out differently in each jurisdiction and
situation given the range of different actors, institutions and histories involved in efforts
to define and resolve policy issues. However, others have noted that it is possible to
identify general attributes of the formulation process that are similar across jurisdictions
(Howlett et al., 2009).
Several characteristics of generic policy formulation activities have been described in
the policy studies literature. Jones (1984, p. 78), for example, depicted the following broad
attributes of formulation in practice:
 Policy formulation  7

● Formulation need not be limited to one set of actors. There may well be two or more
formulation groups producing competing (or complementary) proposals.
●● Formulation may proceed without a clear definition of the problem, or without for-
mulators ever having much contact with the affected groups. Along the same vein,
ill-­structured problems, or those which embody a great deal of uncertainty in terms
of the range of their impact, are also dealt with during formulation, as explored in
Chapter 2 by Nair and Howlett.
●● There is no necessary coincidence between formulation and particular institutions,
though formulation is a frequent activity of bureaucratic agencies.
●● Formulation and reformulation may occur over a long period of time without ever
building sufficient support for any one proposal.
●● There are often several appeal points for those who lose in the formulation process
at any one level.
● The process itself never has neutral effects. Somebody wins and somebody loses
even in the workings of science.

In terms of process, Harold Thomas (2001) noted that four aspects of policy formula-
tion are usually visible: appraisal, dialogue, formulation and consolidation.
During appraisal, information and evidence necessary to understand the issue at
hand is sought and considered. This step in formulation is where data about policy
problems and their solutions – in the form of research reports, expertise and input
from stakeholders and the general public – is considered. Following this, a dialogue
phase between actors engaged in policy formulation ensues. This dialogue is centred
on the deliberation and exchange of different viewpoints about the policy goals and
potential means to resolve them. Dialogues can be structured with the involvement
of chosen experts and representatives from the private sector, labour organizations
or other interest groups, or they can take place as more open and unstructured pro-
cesses. The structure of the dialogue can make a significant difference on the impact
of that participation in the formulation process (Hajer, 2005). While established expert
opinion is often sought at the expense of new input during formal proceedings, efforts
to involve participants from less established organizations and viewpoints can invigor-
ate the discussion.
Central to this process is the actual formulation phase, wherein administrators and
public officials scrutinize the costs, benefits, challenges and opportunities of various
policy alternatives in the effort to consolidate a proposal or proposals about which alter-
natives or mix of alternatives will proceed through to authoritative decision-­makers. This
phase embodies the actual policy ‘work’ that defines policy formulation, an aspect that
is discussed further by Nekola and Kohoutek in Chapter 3. The choice of some policy
alternatives over others is likely to draw opposition from actors whose preferred instru-
ments are sidelined. These and other forms of feedback about shortlisted policy options
are considered during a final consolidation phase in which proposed policy solutions are
amended or refined before moving forward.
While some of the issues involved in formulation are technical and have a significant
knowledge component, the issues that lead policy formulators to choose some policy
options over others need not be based on facts (Merton, 1948). If powerful policy actors
believe that a policy option is infeasible or unacceptable, this contention can be enough
8  Handbook of policy formulation

to exclude it from further discussion (Carlsson, 2000). Burroughs addresses some of these
effects of negative and positive feedback on formulation processes in Chapter 4.

THE CONTENT OF POLICY FORMULATION: INSTRUMENTS

AND DESIGN

While this discussion says something about ‘how’ a policy is formulated, it is less clear
about ‘what’ is being formulated. Here it should be noted that the policy options that are
considered during formulation are the embodiment of techniques or artefacts of govern-
ance that, in some way, use resources of the state to define and attain government goals
(Dahl & Lindblom, 1953; Hood, 2007).
These goals result from the translation of multifaceted and interconnected societal
problems into governmental aims and objectives. This translation has implications for
what items are considered to be administratively achievable, technically feasible and politi-
cally acceptable, and is thus an often contentious process (Dror, 1969; Majone, 1975, 1989;
Meltsner, 1976; Webber, 1986). Indeed, even if policy-­makers agree that a problem exists,
they may not share an understanding of its causes or ramifications (Howlett et al., 2009,
p. 113). Focusing specifically on the delimitation of policy goals that are considered during
formulation, Veselý’s chapter (Chapter 5) discusses the processes involved in the construc-
tion of different types of policy problems, their current scenarios and desirable future states.
This raises the question, of course, about the means which governments have at their
disposal to address policy problems. Policy instruments, alternatively known as ‘policy
tools’, ‘governing instruments’ or the ‘tools of government’, are ‘the set of techniques by
which governmental authorities wield their power in attempting to ensure support and
effect social change’ (Vedung, 1998, p. 21).
These two fundamental ambitions, at the most basic level of analysis, place policy tools
into one of two categories. The first involves instruments that aim to affect social change,
that is, the substance of social behaviour or activity as they directly ‘effect or detect’ change
in the production, distribution and consumption of social goods and services (Hood,
1986). The second category is procedural and focuses internally towards the ­governments’
own policy-­making activities. These instruments affect the political or policy behaviour
involved in the process of formulation in order to ensure that government initiatives are
supported (Howlett, 2000). Chapter 6 by Howlett goes into greater length to distinguish
between these two major categories of tools and review various typologies of substantive
and procedural tools and their contributions to the policy process.
In the creation of both substantive and procedural policy responses to issues and prob-
lems, policy formulation again can be seen to involve the identification of both technical
and political limitations in the path of effective state action. That is to say, formulation
faces a number of constraints that limit the ability of policy-­makers to embark on just
any possible proposed course of action (Majone, 1989).
Substantive constraints may arise within the problem itself. The problem of global
warming, for example, cannot be entirely eliminated because there is no known effective
solution that can be employed without causing tremendous economic and social dislo-
cations. This often leaves policy-­makers to tinker with options that barely scratch the
surface of the problem (Howlett et al., 2009, p. 112). Such constraints can be considered
 Policy formulation  9

‘objective’ in a sense, since reinterpreting them or recasting them in different terms does
not eliminate them.
Procedural constraints, on the other hand, are those that directly impinge upon the
process of adopting policy options and are more subjective in nature and subject to
reformulation and reinterpretation. These constraints are embedded in the social and
institutional contexts within which formulation unfolds, and can include constitutional
specifications, the organization of society and the policy-­making administration, and
established patterns of ideas and beliefs that can lead decision-­makers and formulators
to favour some options over others (DeLeon, 1992; Falkner, 2000; Montpetit, 2002; Yee,
1996). The specific relationship between social groups and the state, as well as the groups’
internal organization and political styles, for example, can create ‘policy horizons’ or
limits to the arrangement of acceptable policy options for certain policy actors that condi-
tion their actions, but which are also subject to government manipulation and reorienta-
tion (Bradford, 1999; Warwick, 2000).
While many policy formulation studies have been engaged in the exploration of various
kinds of policy tools and how they are implemented, there has also been a dedicated focus
in the literature on how policy tools and outcomes can be better matched or ‘designed’.
This latter concentration on the specific devising of policy responses captures the essence
of what has come to be known as the policy design ‘orientation’ in the policy sciences
(Howlett et al., 2015). Policy design scholars recognize that policy decisions may often
be made in a contingent and irrational fashion, but highlight the nuances of translating
ideal ‘technical’ models of instrument use to context-­sensitive solutions. As Linder and
Peters argued, the

design orientation to analysis can illuminate the variety of means implicit in policy alternatives,
questioning the choice of instruments and their aptness in particular contexts. The central role
it assigns means in policy performance may also be a normative vantage point for appraising
design implications of other analytical approaches. More important, such an orientation can be
a counterweight to the design biases implicit in other approaches and potentially redefine the
fashioning of policy proposals. (1990, p. 104)

In contrast with earlier policy design studies that were concerned with the relatively
simple mechanics of mapping single-­tool uses, the new policy design orientation adopts
a more complex multi-­level analytical orientation (Howlett, 2011a). It emphasizes the
design of policy instrument bundles and the interactions that take place within such
bundles when multiple tools are used in policy portfolios designed to address multiple
goals (Givoni et al., 2013; Howlett et al., 2015; Oikonomou et al., 2011).
The design orientation thus envisions policies as being composed of multiple com-
ponents ranging from abstract policy goals to the more operational objectives of policy
programmes, to policy instrument settings and calibrations at the micro level of policy
formulation (Cashore and Howlett, 2007; Hall, 1993; Howlett & Cashore, 2009). Effective
formulation seeks to integrate these various goals and means within policy mixes, so that
their component elements reinforce rather than contradict each other (Briassoulis, 2005;
Meijers, 2004).
Drawing on these various themes of multiple policy tool mixes and multiple elements
or layers of policies, as well as the temporal processes through which policies develop, the
topic of policy design has propagated a burgeoning body of literature.1 Drawing from
10  Handbook of policy formulation

this literature, Chapters 7 (Howlett and Rayner) and 8 (Howlett, Mukherjee and Rayner)
delve deeper into the considerations of dealing with multi-­tool policy mixes as well as
enumerate and investigate the principles of effective design.

HOW DOES FORMULATION OCCUR AND WHY?

Another major question is exactly ‘how’ policy formulation occurs. As highlighted

above, the analyses and comparisons between potential alternatives that are considered
during formulation depend greatly on the nature of the policy actors who propose them,
their beliefs and ideas about society, and the problems that they feel formulation should
address, in conjunction with the nature of the institutional and organizational frame-
works within which they work. As noted by Charles Anderson, policy design and formula-
tion are tantamount to ‘statecraft’ – the exercise of government as ‘the art of the possible’
(1971, p. 120). Formulation ‘is always a matter of making choices from the possibilities
offered by a given historical situation and cultural context’, and those who are engaged in
formulation and policy design use the tools of statecraft to ‘find appropriate possibilities
in the equipment of society’ in order to meet their goals (Anderson, 1971, p. 121).
Understanding the variety of inputs these actors bring to the policy formulation
activity and the contexts within which they function thus can shed considerable light
on why some policy options gain significant attention while others fall by the wayside.
As mentioned above, formulation can take place even without a definite depiction of
the policy problem at hand (Weber & Khademian, 2008), and it often proceeds over
time in successive ‘rounds’ of formulation and reformulation of policy goals and means
(Teisman, 2000; Thomas, 2001). Within this process, while some policy-­makers may look
for ‘win-­win’ solutions that maximize the complementarities between the views of differ-
ent actors, the costs and benefits of different policy choices are borne disproportionately
by different participants, leading to contested processes of evaluation and deliberation
(Wilson, 1974).
Formulation and design processes are therefore fraught with both political and techni-
cal considerations. This reality, however, does not suggest that the systematic effort to pair
policy means with goals is impractical and not worthwhile. Instead, it simply means that
the implementation of some designs may be impossible in certain contexts and that the
choice of any policy alternative involves different policy actors trying to raise and evalu-
ate different preferred policy designs (Dryzek, 1983; Thomas, 2001). This evaluation of
the benefits and costs of different policy options by various policy actors can still occur
through more or less formal modes of policy analysis, and thus remains a central activity
of modern policy formulation (Dunn, 2008; Gormley, 2007; Sidney, 2007).
Discussions of the range of formal and informal analytical techniques that can be
undertaken to evaluate options ex ante during policy formulation are featured in the con-
tributions to Part III of this Handbook. Chapter 9 by Adelle and Weiland explores formal
policy appraisal techniques in policy formulation, while Lehtonen in Chapter 10 discusses
the use of measures and indicators to operationalize information during policy formula-
tion. In Chapter 11, Van der Steen focuses on the use of forecasting tools in policy formu-
lation such as impact assessments, future scenarios and planning. In Chapter 12, Johnson
looks into the role of public participation and consultation during policy formulation.
 Policy formulation  11

SUPPLY, DEMAND AND BROKERAGE OF POLICY ADVICE

These chapters on the ‘who’, ‘what’, ‘where’, ‘why’ and ‘how’ of formulation lead the
way to the fourth part of the Handbook, which explores in more detail the nature of
policy advice and of the advisors who inform decision-­makers during the policy formula-
tion process. While powerful political and administrative leaders with decision-­making
authority are the ones who eventually decide on and thus ‘make’ public policy, in modern
states they do so by following the counsel of bureaucrats, civil servants and other advisors
whom they trust to evaluate and consolidate policy options into coherent designs, and
who provide policy leaders with expert advice about the merits and risks of the proposals
being considered (Heinrichs, 2005; MacRae & Whittington, 1997).
It is useful to think about this wide array of policy advisors as being arranged in an
overall ‘policy advisory system’ within which proximate decision-­makers occupy central
positions. Studies of advisory systems in a variety of jurisdictions such as New Zealand,
Israel, Canada and Australia have furthered this idea of government decision-­makers
operating at the centre of a network of policy advisors which includes both ‘traditional’
policy advisors, such as civil servants and non-­state actors (non-­governmental organi-
zations and think tanks), as well as informal forms of advice supplied by colleagues,
members of the public and political party affiliates, among others (Dobuzinskis et al.,
2007; Maley, 2000; Peled, 2002).
It is generally considered beneficial to have a large range of actors in the policy advisory
system, as this indicates ‘a healthy policy-­research community outside the government
[which] can play a vital role in enriching public understanding and debate of policy issues’,
serving ‘as a natural complement to policy capacity within’ (Anderson, 1996, p. 486). The
existence of different types of policy advisory systems is linked with the nature of the
demand and supply of knowledge in particular policy contexts and sectors (Halffman &
Hoppe, 2005).
Policy advisory systems thus are central to the study of policy formulation and to
the understanding of the selection and reception given to different policy alternatives
and arrangements (Brint, 1990). Conceived of as knowledge utilization venues or ‘mar-
ketplaces’ of ideas and information, advisory systems are comprised of three separate
­components: the supply of policy advice; its demand by government decision-­makers;
and a set of brokers who work as intermediaries to match knowledge supply with demand
(Brint, 1990; Lindquist, 1998). That is, policy advisory systems undertake one of several
general types of analytical activities linked to the types of positions that participants hold
in the creation and exchange of knowledge in the policy formulation process.
More specifically, members of policy advisory systems can be identified as being in one
of four ‘communities’ of advisors depending on the advisory role they perform as well
their proximity to policy actors and their location either inside or outside government
(Table 1.1).
The core actors are those members of the public sector who are closest to the official
policy-­making units of government and include central government agencies, executive
staff and professional government policy analysts. Governmental actors or insiders who
work further away, at the periphery of policy advisory systems, belong to federal commis-
sions, special committees and task forces, or serve on research councils and as scientists
at international organizations. From the non-­governmental sector, actors who are close
12  Handbook of policy formulation

Table 1.1 The four communities of policy advisors

Proximate actors Peripheral actors

Public/governmental CORE ACTORS PUBLIC SECTOR INSIDERS
sector
● Central agencies ● Commissions, committees and
● Executive staff task forces
● Professional governmental ● Research councils/scientists
policy analysts ● International organizations

Non-­governmental PRIVATE SECTOR INSIDERS OUTSIDERS

sector
● Consultants ● Public interest groups
● Political party staff ● Business associations
● Pollsters ● Trade unions
● Donors ● Academics
● Think tanks
● Media
● International non-­governmental
organizations

Source: Howlett (2011b).

to decision-­makers during formulation and are considered private sector insiders may
include private sector consultants, political party staff, pollsters and donor representa-
tives. Actors who are considered to be farthest from the central core of policy formulators,
or outsiders, include those belonging to public interest groups, business associations, trade
unions, think tanks or non-­governmental organizations, or are independent academics or
members of the media.
In terms of the specific contribution that policy advisors can make to different policy
components, these sets of actors can be thought to exist on a spectrum of influence
(Table 1.2). Members of the general public, non-­governmental outsiders and insiders
often impact the policy discourse at the broad level of abstract policy goals and general
policy preferences, while insiders and core actors become more influential as ­formulation

Table 1.2 Advisory system actors by policy level

High-­level abstraction Programme-­level Specific on-­the-­

operationalization ground measures
Policy General abstract policy aims Operationalizable policy Specific policy targets
goals objectives
(Normative) Public, outsiders and insiders Insiders and core actors Core actors
Policy means General policy implementation Operationalizable policy Specific policy tool
preferences tools calibrations
(Cognitive) Public, outsiders and insiders Insiders and core actors Core actors

Source: Howlett (2014).

 Policy formulation  13

and design moves to programme-­level operations and then on to specifying on-­the-­

ground measures and instruments (Page, 2010).
The contributions in Part IV of the Handbook explore the role that different policy
advisors can play during formulation. Craft and Howlett’s chapter (Chapter 13) provides
an overview of the policy advisory system concept and outlines the significance of various
advisory roles during policy formulation. Chapter 14, by Matheson, puts forward a typol-
ogy to understand the organizational structure of policy formulation in government,
while Chapter 15 by Bandola-­Gill and Lyall focuses specifically on the role of brokers
or the ‘third community’ in policy advisory systems. In Chapter 16, Gunn examines how
individuals can operate in such contexts, focusing on the contribution of policy entrepre-
neurship to alternative generation and acceptance during policy formulation.

WHY IS FORMULATION DONE THE WAY IT IS?: THE ROLE OF

IDEAS

A key component of the content of formulation, also mentioned above, is the type of
beliefs and ideas that policy advisors and policy-­makers have about the feasibility and
optimality of the deployment of various arrangements of policy tools to address social
concerns and policy problems. Understandably, the beliefs held by decision-­ makers
about these and other issues plays a key part in influencing their efforts to construct
policy alternatives and assess policy options (Chadwick, 2000; George, 1969; Gormley,
2007; Ingraham, 1987; Jacobsen, 1995; Mayntz, 1983). It follows that some ideas about
policy instrument choices and options are likely to be more influential than others when
it comes to policy formulation, assessment and design (Lindvall, 2009) and that differ-
ent types of ideas will impact different elements of formulation. For example, abstract
policy-­level goals such as economic development or ecological conservation emerge out
of general ethical logics about alleviating poverty or protecting the environment, while
more specific causal constructs about issues such as how increasing household incomes
can lead to greater economic gains or how limiting agriculture near ecologically sensitive
areas results in environmental gains are also significant at the operational level. The same
is true for policy means that address these various goals, as they stem from ideas about
what has worked and what has not.
Differentiating between these different types of ideas in terms of their degree of
abstraction and their normative appeal is thus an important step in understanding the
reasoning that policy formulators apply when creating policy content (Campbell, 1998).
In their work on the influence of ideas in foreign policy-­making situations, Goldstein and
Keohane (1993) and their colleagues noted at least three types of ideas that combined
normative and cognitive elements but at different levels of generality: world views; prin-
cipled beliefs; and causal ideas (see also Braun, 1999; Campbell, 1998). World views or
ideologies have long been recognized as helping people make sense of complex realities
by identifying general policy problems and the motivations of actors involved in politics
and policy. These sets of ideas, however, tend to be very diffuse and do not easily translate
into specific views on particular policy problems.
Principled beliefs and causal stories, on the other hand, can exercise a much more
direct influence on the recognition of policy problems and on policy content. These ideas
14  Handbook of policy formulation

Table 1.3 Ideational components of policy contents

Level of policy debate affected

Background Foreground
Level of ideas affected Normative (Value) Public sentiments Symbolic frames
Cognitive (Causal) Policy paradigms Programme ideas

Source: Howlett (2014), adapted from Campbell (1998).

can influence policy-­making by serving as ‘road maps’ for action, defining problems,
­affecting the strategic interactions between policy actors, and constraining the range of
policy options that are proposed (Carstensen, 2010; Stone, 1989, 1998). At the micro
level, ‘causal stories’ and beliefs about the behaviour patterns of target groups heavily
influence choices of policy settings or calibrations (Schneider & Ingram, 1993, 1994;
Stone, 1989).
As laid out in Table 1.3, ideas stemming from public sentiments or symbolic frames
appeal to the perception of appropriateness or ‘legitimacy’ of a certain policy choice
and are largely expected to affect policy goals (Durr, 1993; Stimson, 1991; Stimson et al.,
1995; Suzuki, 1992). On the other hand, policy paradigms indicate ‘a set of cognitive
background assumptions that constrain action by limiting the range of alternatives that
policy-­making elites are likely to perceive as useful and worth considering’ (Campbell,
2002, p. 35; see also Surel, 2000). Programme ideas similarly represent a selection of par-
ticular solutions from the set of options that are designated as being appropriate within a
prevailing policy paradigm. Paradigms and programme ideas thus influence the selection
of policy means (Hall, 1993; Stone, 1989).
The contributions of Part V of this Handbook examine these issues of policy-­level
ideas and the framing of issues as problems. Chapter 17 by May, Koski and Stramp
explores the impact of policy expertise during policy formulation and identifies a small
group of ‘go-­to’ experts whose ideas carry an outsize weight in many areas of policy
deliberations in the United States. Zito in Chapter 18 highlights the role of expert net-
works and epistemic communities in articulating knowledge while Schneider and Ingram,
in Chapter 19, explore the impact that ideas about the nature and motivations of policy
targets (those members of society whose behaviour the policy is meant to address) have on
policy formulation. Gunter (Chapter 20) looks into the role of consultants and the ideas
they bring to the formulation process.

THE ARTICULATION OF SOLUTIONS: KEY ACTORS IN

POLICY FORMULATION

Part VI of the Handbook then looks specifically at what factors influence the develop-
ment and articulation of some policy solutions rather than others. Simons and Voß in
Chapter 21 discuss the role of instrument constituencies involved in framing and forward-
ing particular policy solutions during formulation. In Chapter 22, McGann looks at the
role of think tanks, academics and research institutes during the formulation of policy
 Policy formulation  15

options, while institutional isomorphism, or the penchant for governments to mimic each
other, sometimes with reason and sometimes without, is critically reviewed by Jarvis in
Chapter 23.
In many formulation situations, general abstract policy aims and implementation pref-
erences are taken as given, establishing the context in which design decisions relating to
programme-­level and on-­the-­ground specifications are made by policy insiders and core
actors. And in many cases, even the goal components of programmes may be already
established, leaving the formulator only the task of establishing specific policy tool cali-
brations which must cohere with these already existing or well-­established policy elements.
Galizzi in Chapter 24 explores the behavioural aspects of this kind of policy formulation
through a review of the use of experiments and behavioural insights such as ‘nudges’ that
affect the reception of policy initiatives on the ground by target groups and individuals.

FORMULATION IN SPACE AND TIME: THE IMPORTANCE OF

CONTEXT AND INFLUENCE

In general, it is understood that policy formulation takes place within present day govern-
ance structures that have their own existing policy logic. In order to address these issues,
it is recognized that policy-­makers need to be cognizant about the internal mechanisms
of their polity and constituent policy sectors (Braathen, 2007; Braathen & Croci, 2005;
Grant, 2010; Skodvin et al., 2010). The amount of ‘elbow room’ or ‘degrees of freedom’
that formulators have in a given formulation context heavily impacts how formulation
activities proceed. Where earlier work on formulation and design often assumed that
policy-­makers were operating with a constrained yet blank slate, modern thinking about
policy design is more rooted in empirical experience that policy designers work in sce-
narios with already established policy mixes and significant policy histories. This work
draws on historical and sociological neo-­institutionalists such as Kathleen Thelen (2004;
Thelen et al., 2003), who noted how macro-­institutional arrangements have normally not
been the product of calculated planning but rather the result of processes of incremental
modifications or reformulations such as ‘layering’ or ‘drift’.
That is, legacies from earlier rounds of decision-­making affect the introduction of new
elements that may conflict with pre-­existing policy components. As Martin Carstensen
has argued, policies may change through gradual processes and are often created much
less through systematic reflection on (practice-­derived) first principles than through a
process of bricolage (Carstensen, 2015).
Contributions to Part VII of the Handbook look more closely at the situational anoma-
lies and their politics that can influence the contexts within which formulation takes place.
Although new or different policy instrument groupings could theoretically be more com-
plementary and thus create a more successful combination of policy elements (Barnett &
Shore, 2009; Blonz et al., 2008; Buckman & Diesendorf, 2010; del Río, 2010; Grabosky,
1994; Gunningham et al., 1998; Hou & Brewer, 2010; Howlett, 2004; Howlett & Rayner,
2007; Roch et al., 2010), it may be very difficult to accomplish or even propose such
changes, and designs instead often focus on reform rather than replacement of an existing
arrangement. Bricolage can ensue through the work of policy formulators as irregularities
emerge through the accumulation of new policy elements that are ­conflicting with ­original
16  Handbook of policy formulation

policy goals (Wilder & Howlett, 2014). Wilder in Chapter 25 explores these dynamics
between the accrual of conflicts and a continual process of bricolage that can dominate
formulation.
The contextual ‘lock in’ that leads to layering thus can impact the formulation process
by restricting a government’s ability to evaluate alternatives and plan or design in a purely
optimal instrumental manner (Howlett, 2009; Oliphant & Howlett, 2010; Williams,
2012). Layering thus typically results in processes of (re)design which alter only some
aspects of a pre-­existing arrangement and can thus be distinguished from processes of
new policy packaging or complete replacement. While complete replacement or a brand
new ‘package’ of policies is rare, there are exceptionally rare instances whereby entirely
dedicated, or ‘bespoke’, policy packages are created to address an unprecedented policy
problem. Customization of policy might be somewhat more common as a means for a
new, multi-­dimensional policy problem by adapting lessons from other similar policy
contexts.
Another type of policy adaptation is ‘patching’, which can be done well if govern-
ments possess sufficient capacity but poorly if they do not. An example of poor patch-
ing is policy ‘stretching’ (Feindt & Flynn, 2009). This is where, operating over periods
of decades or more, elements of a mix are simply extended to cover areas they were
not intended to at the outset. Stretching is especially problematic as small changes in
the mixture of policy elements over such a time period can create a situation where the
elements can fail to be mutually supportive, incorporating contradictory goals or instru-
ments whose combination creates perverse incentives that frustrate initial policy goals.
When these problems are identified, further rounds of tinkering and layering may make
them worse (Feindt & Flynn, 2009). Jorgensen, in his discussion of the politics of policy
formulation in Chapter 26, adds to the discussion about problematic layering by going
beyond the topic of a ­government’s ability to design, to a critical evaluation of govern-
ment willingness to design.
Layering as a formulation process can thus have two sides. On the one hand, nega-
tive stretching or destructive layering exacerbates tensions between regime elements and
more politicized or less instrumental forms of policy-­making and outcomes. However,
layering can also have a positive side and help ameliorate or reduce tensions through
‘smart’-­patching. Stretching and poor patching are thus formulation practices that exist
at the break point between design and non-­design activities of government. Both these
processes fall between the ‘pure’ design and ‘pure’ non-­design ends of the spectrum of
design processes suggested in Figure 1.1.
Non-­design types of policy formulation also vary in the same way as partisan and
ideological, religious or other criteria cloud, crowd out or replace instrumental design
intentionality. Non-­design mechanisms, as highlighted above, include activities such as
bargaining or log-­rolling, through corruption or co-­optation efforts, or faith-­based or
pure electoral calculations that are not instrumental in the same sense as are design efforts.
In such contexts, the ability to meet policy goals or the means to achieve them are second-
ary to other concerns such as ideological purity, the need to retain or augment legislative
or electoral support, or other similar kinds of coalition behaviour.
Highlighting these different processes of formulation, the other two contributors to
Part VII of the Handbook address several related issues. In Chapter 27, Eichbaum and
Shaw shed light on the increasing impact of political parties and political staff on policy
 Policy formulation  17

Extent of alteration of status quo by layering

Bespoke Policy Smart- Tense Non-

Stretching
Packaging Customization Patching Layering design

Frequency of replacement

Corruption or Log- Electoral

Bargaining
Clientelism rolling Opportunism

Extent of irrationality of non-design type

Source: Adapted and modified from Howlett and Mukherjee (2014).

Figure 1.1 A spectrum of policy design and non-­design types

formulation in many countries, adding to the complexity of formulation processes. In

Chapter 28, Scott addresses questions about the actual mechanisms of influence that
interest groups and lobbyists employ during formulation.

CONCLUDING COMMENTS

In bringing to life policy solutions to address complex policy problems, formulation is

the result of various processes of policy evolution that are shaped by time as well as a
government’s intention towards creating policy. These policy-­making efforts can be sys-
tematic in attempting to match policy ends and means in a logical, knowledge-­informed
fashion, or they can result from much less systematic and more irrational processes. How
‘unintended’ policy mixes evolve, are created and limited by historical legacies, and are
hampered due to internal inconsistencies are equally subjects of investigation in formula-
tion studies as are the factors which lead to formulation and policy success (del Río, 2010;
Grabosky, 1994; Gunningham et al., 1998; Howlett & Rayner, 2007).
Transforming policy ambitions into practice is a complex process. While formulation
efforts can be done well or poorly, they reflect some wholesale or partial effort to match
policy goals and means in a sophisticated way to improve outcomes. Non-­design types of
formulation also vary in the same way but more by process of decision-­making than by
their sphere of activity. The efforts of policy-­makers often have failed due to poor designs
that have inadequately incorporated the inherent complexity in policy formulation (Cohn,
2004; Howlett, 2011a). These experiences have led to a greater awareness of the various
obstacles to policy design and have gradually fuelled a desire for a better understanding of
the unique characteristics of policy formulation processes and the spaces in which design
efforts are embedded.
While early formulation and design thinking tended to suggest that the creation of
instruments could only occur in spaces where new policy packages could be designed
from scratch, it is now widely recognized that in most circumstances policy f­ormulation
18  Handbook of policy formulation

involves building on the foundations created in another era and working with the con-
straints they pose. Students of policy formulation are thus interested in how policy for-
mulators, like software designers, can issue ‘patches’ in order to correct flaws in existing
mixes or allow them to adapt to changing circumstances (Howlett, 2011a; Howlett &
Rayner, 2014; Rayner, 2013). In this context, subjects such as policy experiments that can
help to examine the possibilities of redesign (Hoffmann, 2011) and building temporal
properties into tool mixes through adaptive policy-­making (Swanson et al., 2010), which
can make policy mixes more flexible or resistant to shifting conditions (Haasnoot et al.,
2013; Walker et al., 2010) are all subjects of interest in contemporary formulation studies.
Some of the major trends and patterns in policy formulation studies related to these
issues are picked up in the last part of this Handbook. Craft and Howlett’s chapter
(Chapter 29) looks at developments in the realm of policy advisory systems and the incli-
nation towards greater externalization and politicization of policy advice. Strassheim,
in Chapter 30, reviews the movement towards experience-­based versus evidence-­based
policy-­making. In Chapter 31, Linders and Ma discuss the changing role of the public
in policy formulation through an exploration of changes brought on by new mass and
social media. Maor’s chapter (Chapter 32) concludes the collection with an analysis of
the proportionality of government responses, looking specifically at the reasons behind
under-­and over-­reaction in the formulation of policy options in the short and long term.
Portrayed in this Handbook is the very essence of policy-­making activity: the articula-
tion of policy goals, the presentation of policy alternatives and the choice of policy tools.
The contributions in this collection provide a multi-­dimensional understanding of policy
formulation, and a deeper exploration of what formulation activity entails, how the for-
mulation process unfolds, who influences and who formulates policy, and what are their
incentives.

NOTE

1. A full bibliography of policy design publications is available through the Policy Design Lab and Policy Tools
Wiki. http://policy-­design.org/wiki/towards-­a-­new-­generation-­of-­policy-­design-­studies/further-­readings/foot-­​
notes/full-­bibliography/ (accessed 2 November 2016).

REFERENCES

Anderson, C.W. (1971). Comparative policy analysis: the design of measures. Comparative Politics 4(1), 117–31.
Anderson, G. (1996). The new focus on the policy capacity of the federal government. Canadian Public
Administration 39(4), 469–88.
Barnett, C.K., & Shore, B. (2009). Reinventing program design: challenges in leading sustainable institutional
change. Leadership & Organization 30(1), 16–35.
Béland, D., & Howlett, M. (2016). How solutions chase problems: instrument constituencies in the policy
process. Governance 29(3), 393–409.
Bennett, C.J. (1991). What is policy convergence and what causes it? British Journal of Political Science 21(2),
215–33.
Bennett, C.J., & Howlett, M. (1992). The lessons of learning: reconciling theories of policy learning and policy
change. Policy Sciences 25(3), 275–94.
Blonz, J.A., Vajjhala, S.P., & Safirova, E. (2008). Growing Complexities: A Cross-­sector Review of U.S. Biofuels
Policies and their Interactions. Washington, DC: Resources for the Future.
 Policy formulation  19

Braathen, N.A. (2007). Instrument Mixes Addressing Non-­point Sources of Water Pollution. Paris: OECD.
Braathen, N.A., & Croci, E. (2005). Environmental agreements used in combination with other policy instru-
ments. In E. Croci (ed.), The Handbook of Environmental Voluntary Agreements, Vol. 43, pp. 335–64.
Dordrecht: Springer.
Bradford, N. (1999). The policy influence of economic ideas: interests, institutions and innovation in
Canada. Studies in Political Economy, 59, 17–60.
Braun, D. (1999). Interests or ideas? An overview of ideational concepts in public policy research. In A. Busch
and D. Braun (eds), Public Policy and Political Ideas, pp. 11–29. Cheltenham, UK and Northampton, MA,
USA: Edward Elgar.
Briassoulis, H. (2005). Analysis of policy integration: conceptual and methodological considerations. In
H. Briassoulis (ed.), Policy Integration for Complex Environmental Problems: The Example of Mediterranean
Desertification, pp. 81–116. Aldershot: Ashgate.
Brint, S. (1990). Rethinking the policy influence of experts: from general characterizations to analysis of varia-
tion. Sociological Forum 5(3), 361–85.
Buckman, G., & Diesendorf, M. (2010). Design limitations in Australian renewable electricity policies. Energy
Policy 38(7), 3365–76.
Campbell, J.L. (1998). Institutional analysis and the role of ideas in political economy. Theory and Society 27(5),
377–409.
Campbell, J.L. (2002). Ideas, politics and public policy. Annual Review of Sociology 28(1), 21–38.
Carlsson, L. (2000). Policy networks as collective action. Policy Studies Journal 28(3), 502–20.
Carstensen, M.B. (2010). The nature of ideas, and why political scientists should care: analysing the Danish
jobcentre reform from an ideational perspective. Political Studies 58(5), 847–65.
Carstensen, M.B. (2015). Bringing ideational power into the paradigm approach: critical perspectives on policy
paradigms in theory and practice. In J. Hogan and M. Howlett (eds), Policy Paradigms in Theory and Practice,
pp. 295–318. London: Palgrave Macmillan.
Cashore, B., & Howlett, M. (2007). Punctuating which equilibrium? Understanding thermostatic policy dynam-
ics in Pacific Northwest forestry. American Journal of Political Science 51(3), 532–51.
Chadwick, A. (2000). Studying political ideas: a public political discourse approach. Political Studies
48, 283–301.
Cochran, C.L., & Malone, E.F. (1999). Public Policy: Perspectives and Choices. Boulder, CO: Lynne Rienner.
Cohn, D. (2004). The best of intentions, potentially harmful policies: a comparative study of scholarly complex-
ity and failure. Journal of Comparative Policy Analysis 6(1), 39–56.
Considine, M. (2012). Thinking outside the box? Applying design theory to public policy. Politics & Policy 40(4),
704–24.
Dahl, R.A., & Lindblom, C.E. (1953). Politics, Economics and Welfare: Planning and Politico-­economic Systems,
Resolved into Basic Processes. New York: Harper & Brothers.
del Río, P. (2010). Analysing the interactions between renewable energy promotion and energy efficiency support
schemes: the impact of different instruments and design elements. Energy Policy 38(9), 4978–89.
DeLeon, P. (1992). Policy formulation: where ignorant armies clash by night. Review of Policy Research 11(3–4),
389–405.
Dobuzinskis, L., Howlett, M., & Laycock, D. (eds) (2007). Policy Analysis in Canada: The State of the Art.
Toronto: University of Toronto Press.
Dror, Y. (1969). The prediction of political feasibility. Futures 1(4), 282–8.
Dryzek, J. (1983). Don’t toss coins in garbage cans: a prologue to policy design. Journal of Public Policy 3(4),
345–67.
Dunn, W.N. (2008). Public Policy Analysis: An Introduction. Upper Saddle River, NJ: Pearson-­Prentice Hall.
Durr, R.H. (1993). What moves policy sentiment? American Political Science Review 87, 158–72.
Falkner, G. (2000). Policy networks in a multi-­level system: convergence towards moderate diversity? West
European Politics 23(4), 94–120.
Feindt. P., & Flynn, A. (2009). Policy stretching and institutional layering: British food policy between security,
safety, quality, health and climate change. British Politics 4(3), 386–414.
George, A.L. (1969). The ‘operational code’: a neglected approach to the study of political leaders and decision-­
making. International Studies Quarterly 13, 190–222.
Givoni, M., Macmillen, J., Banister, D., & Feitelson, E. (2013). From policy measures to policy pack-
ages. Transport Reviews 33(1), 1–20.
Goldstein, J., & Keohane, R.O. (eds) (1993). Ideas and Foreign Policy: Beliefs, Institutions and Political Change.
Ithaca, NY: Cornell University Press.
Gormley, W.T. (2007). Public policy analysis: ideas and impact. Annual Review of Political Science
10, 297–313.
Grabosky, P.N. (1994). Green markets: environmental regulation by the private sector. Law and Policy 16(4),
419–48.
20  Handbook of policy formulation

Grant, W. (2010). Policy instruments in the Common Agricultural Policy. West European Politics 33(1), 22–38.
Gunningham, N., Grabosky, P., & Sinclair, D. (1998). Smart Regulation: Designing Environmental Policy.
Oxford: Clarendon Press.
Haasnoot, M., Kwakkel, J.H., Walker, W.E., & ter Maat, J. (2013). Dynamic adaptive policy pathways: a method
for crafting robust decisions for a deeply uncertain world. Global Environmental Change 23(2), 485–98.
Hajer, M.A. (2005). Setting the stage: a dramaturgy of policy deliberation. Administration & Society 36(6),
624–47.
Halffman, W., & Hoppe, R. (2005). Science/policy boundaries: a changing division of labour in Dutch expert
policy advice. In S. Maasen and P. Weingert (eds), Democratization of Expertise? Exploring Novel Forms of
Scientific Advice in Political Decision-­making, pp. 135–51. Dordrecht: Springer.
Hall, P.A. (1993). Policy paradigms, social learning and the state: the case of economic policy making in Britain.
Comparative Politics 25(3), 275–96.
Heinrichs, H. (2005). Advisory systems in pluralistic knowledge societies: a criteria-­based typology to assess
and optimize environmental policy advice. In S. Maasen and P. Weingart (eds), Democratization of Expertise?
Exploring Novel Forms of Scientific Advice in Political Decision-­making, pp. 41–61. Dordrecht: Springer.
Hoffmann, M.J. (2011). Climate Governance at the Crossroads: Experimenting with a Global Response after
Kyoto. New York: Oxford University Press.
Hood, C. (1986). The Tools of Government. Chatham: Chatham House Publishers.
Hood, C. (2007). Intellectual obsolescence and intellectual makeovers: reflections on the tools of government
after two decades. Governance 20(1), 127–44.
Hou, Y., & Brewer, G. (2010). Substitution and supplementation between co-­functional policy instruments:
evidence from state budget stabilization practices. Public Administration Review 70(6), 914–24.
Howlett, M. (2000). Managing the ‘hollow state’: procedural policy instruments and modern govern-
ance. Canadian Public Administration 43(4), 412–31.
Howlett, M. (2004). Beyond good and evil in policy implementation: instrument mixes, implementation styles
and second generation theories of policy instrument choice. Policy & Society 23(2), 1–17.
Howlett, M. (2009). Governance modes, policy regimes and operational plans: a multi-­level nested model of
policy instrument choice and policy design. Policy Sciences 42, 73–89.
Howlett, M. (2011a). Designing Public Policies: Principles and Instruments. New York: Routledge.
Howlett, M. (2011b). Revisiting policy design: the rise and fall (and rebirth?) of policy design studies. Paper
prepared for the General Conference of the European Consortium for Political Research (ECPR) Section 63.
Available at http://ecpr.eu/filestore/paperproposal/777bba9e-­7de8-­423d-­9c1d-­1c4362c0e92a.pdf (accessed
2 November 2016).
Howlett, M. (2014). Policy design: who, what, how and why? In C. Halpern, P. Lascoumes, & and P. Le Galès
(eds), L’instrumentation de l’action publique: controverses, résistances, effets, pp. 281–316. Paris: Presses de
Sciences Po.
Howlett, M., & Cashore, B. (2009). The dependent variable problem in the study of policy change: understand-
ing policy change as a methodological problem. Journal of Comparative Policy Analysis 11(1), 33–46.
Howlett, M., & Lejano, R.P. (2013). Tales from the crypt: the rise and fall (and rebirth?) of policy design.
Administration & Society 45(3), 357–81.
Howlett, M., & Mukherjee, I. (2014). Policy design and non-­design: towards a spectrum of policy formulation
types. Politics and Governance 2(2), 57–71.
Howlett, M., & Ramesh, M. (2016). Achilles’ heels of governance: critical capacity deficits and their role in
governance failures. Regulation & Governance.
Howlett, M., & Rayner, J. (2007). Design principles for policy mixes: cohesion and coherence in ‘new governance
arrangements’. Policy & Society 26(4), 1–18.
Howlett, M., & Rayner, J. (2014). Patching vs packaging in policy formulation: assessing policy portfolio design.
Politics and Governance 1(2), 170–82.
Howlett, M., Ramesh, M., & Perl, A. (2009). Studying Public Policy: Policy Cycles and Policy Subsystems.
Toronto: Oxford University Press.
Howlett, M., Mukherjee, I., & Woo, J.J. (2015). From tools to toolkits in policy design studies: the new design
orientation towards policy formulation research. Policy & Politics 43(2), 291–311.
Ingraham, P. (1987). Toward more systematic considerations of policy design. Policy Studies Journal 15(4),
611–28.
Jacobsen, J.K. (1995). Much ado about ideas: the cognitive factor in economic policy. World Politics, 47, 283–310.
Jones, C.O. (1984). An Introduction to the Study of Public Policy, 3rd edn. Monterey, CA: Brookes/Cole.
Kingdon, J.W. (1984). Agendas, Alternatives, and Public Policies. Boston: Little Brown and Company.
Lasswell, H.D. (1936). Politics: Who Gets What, When, How. New York: Whittlesey House.
Linder, S.H., & Peters, B.G. (1990). The design of instruments for public policy. In S.S. Nagel (ed.), Policy Theory
and Policy Evaluation: Concepts, Knowledge, Causes, and Norms, pp. 103–19. New York: Greenwood Press.
Lindquist, E. (1998). A quarter century of Canadian think tanks: evolving institutions, conditions and
 Policy formulation  21

s­trategies. In D. Stone, A. Denham, & M. Garnett (eds), Think Tanks Across Nations: A Comparative
Approach, pp. 127–44. Manchester: Manchester University Press.
Lindvall, J. (2009). The real but limited influence of expert ideas. World Politics 61(4), 703–30.
MacRae, D., & Whittington, D. (1997). Expert Advice for Policy Choice: Analysis and Discourse. Washington,
DC: Georgetown University Press.
Majone, G. (1975). On the notion of political feasibility. European Journal of Political Research 3, 259–74.
Majone, G. (1989). Evidence, Argument, and Persuasion in the Policy Process. New Haven, CT: Yale University
Press.
Maley, M. (2000). Conceptualising advisers’ policy work: the distinctive policy roles of Ministerial Advisers in
the Keating government, 1991–96. Australian Journal of Political Science 35(3), 449–470.
Mayntz, R. (1983). The conditions of effective public policy: a new challenge for policy analysis. Policy & Politics
11, 123–43.
Meijers, E. (2004). Policy integration: a literature review. In D. Stead, H. Geerlings, & E. Meijers (eds), Policy
Integration in Practice: The Integration of Land Use Planning, Transport and Environmental Policy-­making in
Denmark, England and Germany, pp. 9–24. Delft: Delft University Press.
Meltsner, A.J. (1976). Policy Analysts in the Bureaucracy. Berkeley, CA: University of California Press.
Merton, R.K. (1948). The self-­fulfilling prophecy. The Antioch Review 8(2), 193–210.
Montpetit, É. (2002). Policy networks, federal arrangements, and the development of environmental regulations:
a comparison of the Canadian and American agricultural sectors. Governance 15(1), 1–20.
Oikonomou, V., Flamos, A., Gargiulo, M., Giannakidis, G., Kanudia, A., Spijker, E., & Grafakos, S. (2011).
Linking least-­cost energy system costs models with MCA: an assessment of the EU renewable energy targets
and supporting policies. Energy Policy 39(5), 2786–99.
Oliphant, S., & Howlett, M. (2010). Assessing policy analytical capacity: comparative insights from a study
of the Canadian environmental policy advice system. Journal of Comparative Policy Analysis: Research and
Practice 12(4), 439–45.
Page, E.C. (2010). Bureaucrats and expertise: elucidating a problematic relationship in three tableaux and six
jurisdictions. Sociologie du Travail, 52(2), April–June, 255–73.
Peled, A. (2002). Why style matters: a comparison of two administrative reform initiatives in the Israeli public
sector, 1989–1998. Journal of Public Administration Research and Theory 12(2), 217–40.
Rayner, J. (2013). On smart layering as policy design: tackling the biofuels policy mess in Canada and the United
Kingdom. Policy Sciences, Special issue on policy design.
Roch, C., Pitts, D., & Navarro, I. (2010). Representative bureaucracy and policy tools: ethnicity, student disci-
pline, and representation in public schools. Administration & Society 42(1), 38–65.
Salamon, L.M. (1989). Beyond Privatization: The Tools of Government Action. Washington, DC: Urban
Institute.
Salamon, L.M. (2002). The Tools of Government: A Guide to the New Governance. New York: Oxford University
Press.
Schmidt, V.A. (2008). Discursive institutionalism: the explanatory power of ideas and discourse. Political
Science 11(1), 303–26.
Schneider, A., & Ingram, H. (1993). Social construction of target populations: implications for politics and
policy. American Political Science Review 87(2), 334–47.
Schneider, A., & Ingram, H. (1994). Social constructions and policy design: implications for public administra-
tion. Research in Public Administration 3, 137–73.
Schneider, A.L., & Ingram, H.M. (1997). Policy Design for Democracy. Lawrence, KS: University Press of
Kansas.
Sidney, M.S. (2007). Policy formulation: design and tools. In F. Fischer, G.J. Miller, & M.S. Sidney (eds),
Handbook of Public Policy Analysis: Theory, Politics and Methods, pp. 79–87. New Brunswick, NJ: CRC
Taylor & Francis.
Skodvin, T., Gullberg, A.T., & Aakre, S. (2010). Target-­group influence and political feasibility: the case of
climate policy design in Europe. Journal of European Public Policy 17(6), 854–73.
Stimson, J.A. (1991). Public Opinion in America: Moods Cycles and Swings. Boulder, CO: Westview Press.
Stimson, J.A., Mackuen, M.B., & Erikson, R.S. (1995). Dynamic representation. American Political Science
Review 89, 543–65.
Stone, D.A. (1989). Causal stories and the formation of policy agendas. Political Science Quarterly 104(2),
281–300.
Stone, D.A. (1998). Policy Paradox and Political Reason. Glenview, IL: Scott, Foresman.
Surel, Y. (2000). The role of cognitive and normative frames in policy-­making. Journal of European Public Policy
7(4), 495–512.
Suzuki, M. (1992). Political business cycles in the public mind. American Political Science Review 86, 989–96.
Swanson, D., Barg, S., Tyler, S. et al. (2011). Seven tools for creating adaptive policies. Technological Forecasting
and Social Change 7(6), July, 924–39.
22  Handbook of policy formulation

Teisman, G.R. (2000). Models for research into decision-­making processes: on phases, streams and decision-­
making rounds. Public Administration 78(4), 937–56.
Thelen, K. (2004). How Institutions Evolve: The Political Economy of Skills in Germany, Britain, the United
States and Japan. Cambridge: Cambridge University Press.
Thelen, K., Mahoney, J., & Rueschemeyer, D. (2003). How institutions evolve: insights from comparative histori-
cal analysis. In J. Mahoney and D. Rueschemeyer (eds), Comparative Historical Analysis in the Social Sciences,
pp. 208–40. Cambridge: Cambridge University Press.
Thomas, H.G. (2001). Towards a new higher education law in Lithuania: reflections on the process of policy
formulation. Higher Education Policy 14(3), 213–23.
Vedung, E. (1998). Policy instruments: typologies and theories. In M.L. Bemelmans-­Videc, R.C. Rist, &
E. Vedung (eds), Carrots, Sticks & Sermons: Policy Instruments and their Evaluation, pp. 21–58. New
Brunswick, NJ: Transaction Publishers.
Walker, W.E., Marchau, V.A.W.J., & Swanson, D. (2010). Addressing deep uncertainty using adaptive policies:
introduction to Section 2. Technological Forecasting and Social Change 77(6), 917–23.
Warwick, P.V. (2000). Policy horizons in West European parliamentary systems. European Journal of Political
Research 38(1), 37–61.
Webber, D.J. (1986). Analyzing political feasibility: political scientists’ unique contribution to policy analysis.
Policy Studies Journal 14(4), 545–54.
Weber, E.P., & Khademian, A.M. (2008). Wicked problems, knowledge challenges and collaborative capacity
builders in network settings. Public Administration Review 68(2), 334–49.
Wilder, M., & Howlett, M. (2014). The politics of policy anomalies: bricolage and the hermeneutics of
­paradigms. Critical Policy Studies 8(2), 183–202.
Williams, R.A. (2012). The limits of policy analytical capacity – Canadian financial regulatory reform.
International Journal of Public Sector Management 25(6/7), 455–63.
Wilson, J.Q. (1974). The politics of regulation. In J.W. McKie (ed.), Social Responsibility and the Business
Predicament, pp. 135–68. Washington, DC: Brookings Institute.
Yee, A.S. (1996). The causal effects of ideas on policies. International Organization 50(1), 69–108.
2. The central conundrums of policy formulation:
ill-­structured problems and uncertainty
Sreeja Nair and Michael Howlett

INTRODUCTION: ILL-­STRUCTURED POLICY PROBLEMS AND

THE CHARACTERIZATION OF UNCERTAINTY

The concept of uncertainty has been widely interpreted and studied in diverse disciplines
that influence public policy. These include the physical sciences, social sciences, mathemati-
cal sciences, engineering, economics, philosophy and psychology. The theoretical basis, his-
torical context, relevance, and tools and methods for addressing uncertainty are thus often
grounded within specific discourses originating in different disciplines (Walker et al., 2012).
Historically, policy scholars studying problem solving in policymaking – such as
Churchman (1967), Rittel and Webber (1973) and Simon (1973) – typically thought
about uncertainty in a purely ‘objective’ sense – that is, whether the problem’s causes
and solutions were known or unknown. Bivariate concepts of ‘wicked’ and ‘tame’ or
‘well-­structured’ and ‘ill-­structured’ problem contexts introduced by these authors have
dominated thinking in the area. These, however, form only part of a larger group of
policy problems that become intractable owing to several kinds of uncertainties. An ill-­
structured problem in this sense is one ‘whose structure lacks definition in some respect’
(Simon, 1973, p. 181). ‘Wicked’ problems can be considered to comprise a subset of ill-­
structured problems, generally characterized by lack of agreement regarding the nature
of the problem as well as potential solutions (Rittel & Webber, 1973). A major challenge
in formulating strategies to deal effectively with uncertainty has been the inadequacy of
various schemes and models used to classify different levels and types of uncertainty
and to assess their impacts. A seminal paper by Knight (1921) addressed ill-­structured
problems, usefully distinguishing between uncertain futures in which there is a reasonably
quantifiable probability distribution (Knightian risk) and those in which there is not and
such distributions are unknown (Knightian uncertainty).
Morgan and Henrion (1990) in an oft-­cited text underscored the importance of prop-
erly classifying the types and sources of uncertainty in policymaking so that they can be
effectively addressed. They argued against Knight that a classification of uncertainty as
centered on known/unknown probability makes it difficult to proceed from analysis to
‘real-­world decision-­making’. Instead, they argued for a subjectivist or Bayesian approach
which classified uncertainty in terms of the different kinds of sources from which it can
arise. This includes statistical variation owing to random measurement errors, ‘linguistic
imprecision’ (that is, cases where the quantities being studied or measured are not well
specified or characterized), variations over time and space, randomness, subjective judge-
ment, disagreement between experts, and differences between the real and approximated
value of the quantity (Morgan & Henrion, 1990). This analysis was useful in noting that
while uncertainty often arises due to imperfect information, including wrong ­information

23
24  Handbook of policy formulation

or missing information, all existing information is also prone to ambiguity – includ-

ing multiple interpretations and diverse perspectives (Jones & Baumgartner, 2005).
Uncertainties surrounding the choice of policy options and their consequences, and
levels of confidence regarding available information and values of multiple stakeholders
(including decision-­makers) are also significant (Hansson, 1996).
While Morgan and Henrion’s classification focused on uncertainty in quantitative
policy analysis, Koppenjan and Klijn (2004) extended this analysis further in presenting a
classification focused on the interactions among actors and knowledge (or information)-­
related uncertainty needed to resolve complex policy problems. Some of these uncertainties
overlap with the analysis of Morgan and Henrion, for example, those related to decision
variables and value parameters and related uncertainties. Koppenjan and Klijn’s classifica-
tion, however, also includes: (1) substantive uncertainty that arises due to a lack of relevant
information related to the nature of the complex problem, or the different interpretations
of information coming from different ‘frames of reference’ of the social actors; (2) strategic
uncertainty that arises due to the unpredictability of strategies deployed by different actors
based on their perception of the problem and strategies likely to be deployed by other
actors; and (3) institutional uncertainty that arises owing to the complexity of interaction
of different actors guided by institutional frameworks (that is, rules and procedures of
the organizations they represent). Similarly, Brugnach et al. (2008) present uncertainty as
involving an object(s) of perception or knowledge, various actors including the decision-­
maker, and the relationships that bind the object(s) and the actors. They consider ambi-
guity as uncertainty of a third kind, separate from the uncertainty inherent in a system
­(ontological uncertainty) and that arising due to lack of knowledge (epistemic uncertainty).
Both kind of uncertainty affect policymaking at both the level of objective knowledge
of problems as well as at the level of the relative nature of decision-­makers’ knowledge of
that ‘knowledge base’. The ‘wicked’ problem space, again, can be seen in this view to itself
be a superset of several other interconnected problems spanning multiple policy domains
and levels of government, and requiring a high degree of deliberation and cooperation
for effective problem solving (Jentoft & Chuenpagdee, 2009).
In addition, such problems are also ‘relentless’, that is, they are not typically able to be
solved permanently, thus making it important to ensure a ‘continuous transfer, receipt
and integration of knowledge’ over time (Weber & Khademian, 2008, p. 336). Such prob-
lems rank high in terms of their complexity, uncertainty regarding problem characteristics
and solutions, and divergence of perspectives (Head, 2008; Table 2.1).
Usefully synthesizing these approaches, Walker et al. (2010) identify five policy-­relevant

Table 2.1 Characteristics of wicked problems

Degrees of difficulty
Complexity of elements, sub-­systems and interdependencies Low Moderate High
Uncertainty in relation to risks, consequences of action and changing Low Moderate High
patterns
Divergence and fragmentation in viewpoints, values, strategic intentions Low Moderate High

Source: Modified from Head (2008).

 The central conundrums of policy formulation  25

levels of uncertainty. These include the ‘Level I’ ‘shallow’ or parameter uncertainties

where alternative states of a system within specific probabilities exist but in some sce-
narios may be well known and established. Policy problems characterized by parameter
uncertainty are not very difficult problems to handle – at least in theory – and are likely
to be resolvable by standard treatments.
These can be distinguished from ‘Level II’ medium or fuzzy uncertainty, where multiple
alternatives exist within a specific scenario but can be ranked based on the ‘perceived like-
lihood’ of their occurrence. ‘Level III’ situations are then those where different possible
overall scenarios exist but these can still be ranked in terms of their likelihood. ‘Level IV’
uncertainty represents a more complex form of Level III uncertainty in which multiple
plausible alternative scenarios exist but cannot be ranked in terms of their perceived
likelihood. Finally, in the most complex ‘Level V’ situations, it is not feasible to present
or agree upon a full range of alternative scenarios and there is a very real ‘possibility of
being surprised’ by events occurring outside of normal boundary conditions and solu-
tions (Walker et al., 2013a; Figure 2.1).
This multi-­level model is useful because it allows for the identification of policy

Level I Level II Level III Level IV Level V

Context
A clear enough Alternate Alternate A multiplicity Unknown
future (with futures (with futures of plausible future
sensitivity) probabilities) (with futures
ranking) (unranked)

A
B
C

System model A single system A single Several Several system Unknown

model system model system models, with system
Complete certainty

with a models, one different model; know

probabilistic of which is structures we don’t Total ignorance
parameteriz- most likely know
ation

System outcomes Point estimates Several sets Several sets A known range Unknown
with sensitivity of point of point of outcomes outcomes;
estimates with estimates, know we
confidence ranked don’t know
intervals, with according
a probability to their
attached to perceived
each set likelihood

Weights on A single estimate Several sets Several sets A known range Unknown
outcomes of the weights of weights, of weights, of weights weights;
with a ranked know we
probability according don’t know
attached to to their
each set perceived
likelihood

Source: Walker et al. (2013a).

Figure 2.1 Different levels and orders of uncertainty

26  Handbook of policy formulation

responses that can effectively ‘adapt’ to match with the rate and level of current as well
as projected uncertainty in policy environments and impacts. Level I and II problems, for
example, as noted above, can be anticipated and factored into predictions of future events
and trajectories. Level III problems are more complex, as different alternative scenarios
are possible, a number of tools could be used in each scenario and the complexity of
accurately forecasting changes over time (Taeihagh et al., 2013), but can still be dealt with
reasonably efficaciously.
The final two situations of multiple, contested scenarios, however, fall into a category
that Walker et al. (2001) refer to as ‘deep’ uncertainty. These are the worst-­case problems
with multiple perspectives regarding the nature of the issue or problem at hand as well as
multiple potential solutions whose prospects for success are unknown (Rittel & Webber,
1973). As Maxim and van der Sluijs (2011) note, for example, most policy typologies are
focused on the ‘producer’ of information and ignore uncertainty related to process and
communication between the producer and the end-­user (that is, the decision-­maker).
These uncertainties can relate to ‘qualification of the knowledge base’ or the degree of
agreement upon or the absolute size of the evidentiary support for models. Uncertainties
can also arise owing to the ‘value-­ladenness’ of policy choices, which includes differ-
ent actor perspectives on the value of the knowledge and information being utilized
for decision-­making and arguments concerning preferred policy alternatives pathways
(Mathijssen et al., 2008).1
Both of these types of uncertainty are well beyond calculations of risk and involve a
much higher ratio of ignorance and ambiguity, requiring a very different type of policy
response and design (Stirling, 2010), ones which incorporate the real possibility of sur-
prise into actions which embody possibilities for flexibility, adaptiveness and change.
Becker and Brownson (1964) argue that even at a relatively simple level, when knowl-
edge is available on a subject, policymakers may not be aware of it and thus may undertake
decision-­making on the basis of uninformed ignorance rather than informed awareness.
This tendency is worsened when collective or absolute knowledge of a subject or phenom-
enon is not as readily available. Decision-­makers may be aware of this gap and function
with an attitude of prudent awareness or, when they are unaware of their ignorance, with
a hubristic attitude or over-­confidence (Table 2.2).

UNCERTAINTY AND CHALLENGES TO POLICY

FORMULATION

All of these kinds of ill-­structured problems have been studied in the context of various
policy issues. Recent studies, for example, have dealt with policies pertaining to environ-
mental health (Kreuter et al., 2004), development of genetically modified foods (Durant
& Legge, 2006), fisheries and coastal governance (Jentoft & Chuenpagdee, 2009), organi-
zational learning (Crul, 2014), educational research (Jordan et al., 2014) and global envi-
ronmental issues such as climate change.
Levin et al. (2012) argue that problems such as climate change have certain problematic
features which illustrate a special class of ‘super-­wicked problems’. These include, firstly,
that as action towards addressing a problem (climate change in this case) is delayed,
it gets more difficult to solve; secondly, that the problem is exacerbated since those
 The central conundrums of policy formulation  27

Table 2.2 Policymakers’ knowledge and comprehension matrix

Nature of existing collective knowledge of a phenomenon

Aspects of a problem and Aspects are unknown

possible solutions are known
Nature of Aware Known-­Known: Known-­Unknown:
decision-­ Key policy actors are aware of Key policy actors are aware
makers’  the known aspects of a  that certain aspects of the
awareness phenomena phenomenon are unknown
of existing (INFORMED AWARENESS) (PRUDENT AWARENESS)
knowledge of a Ignorant Unknown-­Known: Unknown-­Unknown:
phenomenon Key policy actors are unaware Key policy actors are unaware
 of known aspects of a  that certain aspects of the
phenomenon phenomenon are unknown
(UNINFORMED (IMPRUDENT
IGNORANCE) IGNORANCE)

Source: Based on Becker and Brownson (1964).

responsible for causing the problem and who possess the means to solve it lack any clear
incentive to act; and thirdly, that there is no legal institutional framework to sufficiently
address the impacts of a problem such as climate change spread over time and geographic
scales.
This is a good case to illustrate the many policy-­relevant problems associated with
this level of uncertainty. For example, uncertainty in climate assessments can emerge
for a number of reasons: lack of data or lack of agreement on results; choice of sta-
tistical methods; error of measurement; use of approximations; subjectivity in judge-
ment; uncertainty in human behaviour; errors in model structure; errors in values
of parameters; changes in parameters from historical values; differences in concepts
and terminology; choice of spatial/temporal units; and assumptions taken. In climate
projections and impact assessments, uncertainty gathers and often magnifies through
a ‘cascade of uncertainty’ or an ‘uncertainty explosion’ (Schneider & Kuntz-­Duriseti,
2002). This refers to the process whereby uncertainty accumulates throughout the
process of climate change projections and impact assessment. The cascade also
implies that in a causal chain such as climate impact assessments, the characteristics
of the ­aggregate distribution might be very different from the individual components
themselves.
Additionally, climate change is a global phenomenon with local impacts, and there is a
time delay when these impacts are manifested (Schneider & Kuntz-­Duriseti, 2002). And
apart from empirical and methodological challenges, there may be uncertainty owing to
institutional barriers for garnering consensus, combining expert judgement and integrat-
ing multiple perspectives (Webster, 2003).
Dealing with such high levels of uncertainties requires a different kind of policy than
might be adopted when only lower level issues exist. The focus of policy design under low
levels of uncertainty, on the other hand, is to either reduce uncertainty where possible or,
28  Handbook of policy formulation

in other cases, to assess the range of uncertainty and then identify policy measures that
are expected to be ‘robust’ within this range (Bredenhoff-­Bijlsma, 2010).
Day and Klein (1989) suggest that while most government policies are crafted in
response to events that are ‘reasonably predictable’, policy events at higher levels of uncer-
tainty can also be: (1) unpredictable, ‘unforeseen’ and ‘unprojectable’; (2) c­ atastrophic;
and (3) where the interpretation of uncertainty signals is convoluted because of associ-
ated moral and social issues.2 As an example of the third category, Day and Klein (1989)
discuss the spread of AIDS in Britain in the 1980s. In the specific case of strategies
designed to reduce vulnerability to climate risks, policies that do not consider the existence
of the diversity of risks, impacts and responses in a system can end up as ‘policy misfits’
(Bunce et al., 2010) or may become counter-­intuitive or ‘maladaptive’ as they increase
risks in the long run (Barnett & O’Neill, 2011). That is, if environmental degradation and
change lead to certain ‘thresholds’ being crossed, current policy responses may not be as
effective (Kwadijk et al., 2010).
Under conditions of deep uncertainty, policies should be prepared to deal with worst-­
case scenarios, allow for quick recovery and be ready for potential reforms in policy
design (Walker et al., 2010). On climate change, for example, Smith et al. (2010) argue
that current decision-­making on adapting to the impacts of climate change focuses on
‘adjustments’ to current activities, leaving the possibility of a potential transformation
in social and political regimes largely unaddressed (Pelling, 2011; Smith et al., 2010). If
policymakers assume that certain policy choices are ‘no-­harm’ or ‘no-­regret’ in the short
term, they may overlook their possible adverse (sometimes irreversible) effects in the
long run and thus delay timely preventive action. Policymakers must learn to recognize
early warnings or changes, especially as new knowledge emerges (European Environment
Agency, 2001).
However, adequate reflection of on-­the-­ground realities remains a key concern; for
example, a lack of awareness on climate change issues among decision-­makers can lead
them to rely on a largely expert-­driven approach for climate change adaptation planning
that may not reflect reality (Bisaro et al., 2010). The World Resources Report (2011)
highlights the need for decision-­making under uncertainty to be flexible to accommodate
conditions of change, robust to withstand multiple future scenarios, and/or enable deci-
sions to withstand long-­term change.
Hallegatte et al. (2012) argue that it is difficult to define a ‘best solution’ for climate
change and other deep uncertainties, and instead suggest that ‘a menu of ­methodologies’
(p. 36) and tools is needed, together with some indications on which strategies are most
appropriate in which contexts. The idea of ‘policy packaging’ is gaining attention in this
area as implementing a combination of measures (rather than individual ones) or ‘policy
bundling’ can enhance synergies and reduce inconsistencies among the measures (Howlett
& Rayner, 2013; Taeihagh et al., 2013).
However, how to arrive at such policy mixes is an issue for formulators and requires
complex implementation and formulation technologies. Conventional forecasting methods
like Monte Carlo simulations and other kinds of statistical analyses, for example, can
cover low levels of uncertainty or parameter uncertainty by providing likelihood estimates
and probabilities (Brugnach et al., 2008; Walker et al., 2010). However, unexpected events
or ‘wild-­cards’ (Wardekker et al., 2010) can still impact policymaking and have significant
social and political implications.
 The central conundrums of policy formulation  29

DEALING WITH UNCERTAINTY IN POLICY FORMULATION

Failure to address ‘deep uncertainty’ hampers the effectiveness of policies designed for the
long term (Lempert et al., 2003). Both top-­down and bottom-­up methods have different
strengths and weaknesses in dealing with such situations (Dessai & van der Sluijs, 2007).
Decision theory also provides useful tools for decision-­making when the information base is
sufficient, but these tools may not be as robust when there are uncertainties, including infor-
mation gaps. Such tools may be combined with other methods such as scenario planning
(Polasky et al., 2011) or threshold approaches considering critical limits beyond which policy
effectiveness can cease (Kwadijk et al., 2010) – especially useful in cases where crossing such
thresholds can have long-­term, irreversible consequences (World Resources Report, 2011).
Eckles and Schaffner (2011) also argue that public opinion plays a role in affecting
policy outcomes and knowledge of uncertainty and risks, in turn, are important in
forming public opinion. Based on a model for uncertainty management, Herian et al.
(2012) found that using public perceptions on policy planning to inform government
initiatives, such as budgetary planning, can enhance public support for the government
and its decisions under uncertainty.
For long-­term policies, including environmental issues such as climate change, policy-
makers also grapple with uncertainties in the policy formulation stage owing to an incom-
plete understanding of the biophysical and social systems affecting and being affected by
the environmental processes. This incomplete knowledge may lead to an overestimation
or underestimation of the policy problem. This is a major problem in relation to invest-
ments, for example. In order to boost investment in innovative environmental technology,
policymakers often have to ensure and/or create a ‘stable’ facilitating environment for such
innovation to occur and dispel investor concerns pertaining to the risks of failure that are
associated with innovations (Janicke & Jorgens, 2000).
Uncertainty also relates to the problem definition, nature and extent of the problem,
and the extent to which policymakers are dependent on scientific information to formu-
late the policy (Brown, 2000). Problems that are likely to manifest themselves fully only
in the future call for alternative mechanisms for agenda setting and policy formulation. In
such cases an appropriate measure to deal with uncertainty is to facilitate learning over
time, as and when new knowledge regarding the policy problem becomes available. This is
a central characteristic of ‘open, flexible and adaptive institutional environments’, which
further depend on the nature of the governmental regime (Arentsen et al., 2000).
The next subsections present three approaches that have been found to resonate with
policy scholars and practitioners alike in dealing with different levels of uncertainty in policy
formulation: adaptive policymaking, strategic foresight and policy experimentation.

Adaptive Policymaking

The concept of adaptive policies dates back to John Dewey’s (1927) proposition that
‘policies be treated as experiments, with the aim of promoting continual learning and
adaptation in response to experience over time’. One of the most cited pieces on policy
uncertainty in the last decade has been that of Walker et al. (2001), which presents a
spectrum of uncertainty moving from determinism to total ignorance. Adaptive policy-­
making is a model specifically suited to higher-­level problems.
30  Handbook of policy formulation

Dynamic
Adaptation
Dynamic adaptive
pathways
policy pathways
Dynamic nature

Adaptive
policymaking
Static
robust Assumption-based
planning & robust
decision-making

Adaptation
tipping points
Static

Well-characterized uncertainty Deep uncertainty

Uncertainty

Source: Walker et al. (2013b).

Figure 2.2 Approaches for developing adaptive policies

That is, conventional forecasting methods such as Monte Carlo simulations and quan-
titative methods using statistical analyses are not adequately equipped to deal with such
situations (Brugnach et al., 2008; Walker et al., 2010). Walker et al. (2013b) thus map the
possible approaches towards adaptive planning (Figure 2.2) based on their dynamics –
that is, the degree to which they vary over time – and on the level of uncertainty. The
level of uncertainty can range from low to deep uncertainty, as identified in Walker et al.
(2001). The dynamic nature of adaptive policies is represented on the y-­axis in Figure 2.2,
moving from static (indicating that changes over time are not considered in policy func-
tioning), to static robust (indicating that changes or adaptation in the policy are antici-
patory in nature) to dynamic (indicating that adaptation of policy can be anticipatory,
simultaneous and reactive (ex post) over time).
The various approaches highlighted in Figure 2.2 include:

● Assumption-­ based planning which aims at planning to protect an existing

plan from failure in the event that any of the key assumptions of the plan were to
change.
●● Robust decision-­making which can be used to develop a new static plan that is
robust, that is, functions well across a range of plausible futures.
●● Adaptive policymaking which focuses on monitoring and adapting to changes over
time to prevent the static plan from failure (thus ‘static robust’).
●● Adaptation tipping points is a static approach that helps identify the conditions
and time frame beyond which current policies/plans do not continue to function
effectively.
 The central conundrums of policy formulation  31

●● Adaptation pathways is a dynamic extension of the adaptation tipping point

approach that generates an alternate route for continuation of the policy/plan in a
new form to achieve the initial intended objectives.
● Dynamic adaptive policy pathways combine the adaptation pathways and adaptive
policymaking approaches to identify alternative options over time across a range of
plausible futures.

The central objective of these planning approaches is to avoid failure. By deploying

these approaches, policymakers to some extent accept the ‘irreducible character’ of future
uncertainties and aim to reduce uncertainty about policy performance despite these
uncertainties. Many of these approaches are supported by computer models which may be
unavailable or impractical in developing country contexts – for example, robust decision-­
making, which relies heavily on computer runs. There are some quicker and simpler policy
models that could also be deployed if the policymakers believe it’s better to be ‘roughly
right than precisely wrong’ (Walker et al., 2013b, p. 972).
Drawing a parallel between evolutionary biology and policies for sustainable development
(both operating under conditions of change), Rammel and van den Bergh (2003) argue that
‘every successful adaptation is only a temporary “solution”’ to changing conditions and
that diversity of adaptation options and flexibility to deploy these options can contribute
to long-­term stability. The discussion of adaptive policies is also pertinent to issues that face
natural variations, for example, management of fisheries, which is prone to natural cyclical
patterns as well as uncertainty related to harvesting. The adaptive policymaking process in
such cases can be passive (that is, operating on available ‘best’ scientific information until
new knowledge emerges) or active (that is, consciously experimenting with policy alterna-
tives to identify better strategies as new conditions emerge) (Walter, 1992).
In recent years, adaptive policies have been discussed widely in the context of deci-
sions for long-­term infrastructure planning and climate change (Buurman et al., 2009;
Gersonius et al., 2013; Giordano, 2012; Ranger et al., 2013). These research papers
explore the impacts of climate change on long-­lived infrastructure and the influence
of uncertainties on infrastructure policies and plans. Giordano (2012) highlights the
importance of introducing flexibility and adaptiveness from the initial stage of planning.
Similar to the adaptive policymaking concept, policies that are designed to be ‘robust
across a range of plausible futures’ are preferred in this context rather than those aimed
at being ‘optimal’, as they can respond to changes over time and accommodate learning
in this process. The adaptive policy approach can also be applied in the case of trans-
boundary air pollution, which is a complex policy issue with uncertainties related to the
long-­range forecasting of emissions, economic costs of abatement and political concerns
(Kelly & Volleburgh, 2012).
Swanson and Bhadwal (2009) present seven tools that can be used to design adaptive
policies that can deal with a range of anticipated and unanticipated future conditions. For
example, adaptive policies can anticipate future conditions using:

1. Integrated and forward-­looking analysis, including scenario planning.

2. 
Multi-­
stakeholder deliberation to identify potential drawbacks and unintended
impacts.
3. Monitoring key performance indicators to activate automatic policy adjustments.
32  Handbook of policy formulation

In addition, adaptive policies can function effectively when faced with unanticipated
conditions through:

4. Regular and systematic policy review and improvement.

5. Enabling self-­organizing and social networking in communities.
6. Decentralizing decision-­making to the lowest accountable unit of governance.
7. Promoting variation in policy responses.

However, there is little evidence or detailed guidance on the operationalization of these

tools.
There are a number of institutional challenges in actually implementing these policies
primarily owing to the increased costs and time needed for adaptive policies as com-
pared to ‘traditional static approaches’. This makes it difficult for policy practitioners to
justify such adaptive policies, even if the benefits might offset the costs in the long run.
In addition, the complex nature of the adaptive policy product also makes it difficult for
a policymaker to present or defend. As a result, its uptake and usability is rather limited
as compared to conventional straightforward policy planning approaches. Additionally,
robust and adaptive policies might require significant changes to the original policy
design, which may not be politically or socially desirable.
Van der Pas et al. (2012) also draw attention to the need to evaluate adaptive policymak-
ing, which could differ based on the criteria for evaluation and whether the evaluation is
of the plan itself, the process of drafting the plan or the product, that is, the outcomes
of the plan.

Strategic Foresight

Strategic foresight is one of the ways ‘to broaden the boundaries of perception and to
expand the awareness of emerging issues and situations’ in medium levels of uncertainty
(Major et al., 2001, p. 93). Strategic foresight attempts to integrate multiple perspectives
and methods to identify current and emerging issues and trends and help assess policy
options for attaining a desired future. Based on experiences from the United Kingdom,
Singapore and the Netherlands, for example, Habegger (2010) identifies a number of ele-
ments of successful foresight exercises. These include having a scientific edge in specific
foresight methods and processes, allowing for innovation, fostering iterative interactions
between stakeholders, and obtaining the trust and support of top bureaucrats to support
the idea of exploring futures that may be quite different from present conditions.
Foresight can be instrumental for environmental planning, for example, by providing
insights about a range of futures of social-­ecological systems and critical thresholds, and
thus aid in anticipatory planning to avoid adverse impacts (Bengston et al., 2012). It can
also inform policy by enhancing the knowledge base for policy design. This can be done
by ‘increasing the bandwidth’ to allow a greater volume of information to be shared
with policymakers; ‘optimizing the signal’, that is, improving foresight content by ensur-
ing better quality, relevance, usability and timing of foresight studies; and ­‘improving
reception’, that is, enhancing the receptivity of policymakers for foresight (Da Costa
et al., 2008). Given the short-­term focus of many policy cycles, foresight can also help in
identifying current policy gaps to deal with longer-­term issues such as climate change.
 The central conundrums of policy formulation  33

Foresight may also be used as a signalling device by policymakers to indicate to the public
that they are using objective scientific processes in making policies. Under conditions
where policymakers are hesitant to openly share their policy strategies, foresight may help
to engage citizens in a shared vision process via instruments such as transition manage-
ment (Da Costa et al., 2008; Loorbach & Rotmans, 2010).

Policy Experimentation, Including Pilots

Policy experimentation is a predictive tool deployed by various agencies, including the

government, to pre-­test different programmes and policies for their likely impacts, process
of implementation and stakeholder acceptability in advance of launching them at a larger
scale. Experimental projects are generally small-­scale, highly exploratory ‘risky ventures’
whose benefits are often realized through the ‘acquisition of knowledge’ (Rondinelli,
1993) and are well suited to dealing with lower levels of uncertainty. Through knowledge
acquisition and with experience, policy experiments can help reduce uncertainty and aid
decision-­making for the future (Cooney & Lang, 2007). The goal of policy experiments is
to get an indication of the outputs, outcomes and challenges that can be expected when
these programmes and policies are implemented fully.
Policy pilots are a common and important form of policy experimentation and are
instrumental for evaluating new programmes at a ‘controlled small scale’ before introduc-
ing them as full-­scale programmes (Weiss, 1975). Policy pilots can be used to test the likely
effects of new policies and their early outcomes (impact pilots), or to explore the implica-
tions of specific ways of implementing a policy, or assess the best methods of delivery
(process pilots) (Cabinet Office, 2003). Pilots can thus aid in policy appraisal (Turnpenny
et al., 2009) and provide useful insights for dealing with complex policy issues and high
uncertainty (Vreugdenhil et al., 2010).3
Experiments have helped policymakers diversify their policy responses and thus spread
risks. Experiments have also been useful as a source of evidence for policymaking. Under
high levels of uncertainty, societal change or transitions may be required, and these can
be facilitated by experiments. Under conditions of uncertainty, experiments can help test
the design, suitability and acceptability of plausible policy solutions. Policy designers,
however, need to recognize the level of uncertainty in the policy environment and consider
the role of continual monitoring and social learning over time. Towards this end, in recent
years, the literature on experimentation has shifted its focus to the process of experi-
mental policy design, including the role of various stakeholders, compared to the earlier
focus on the content of the experiments itself (Van der Heijden, 2014). Also referred to
as ­‘experimentalist governance’, this new wave involves ‘provisional goal setting’ that is
redefined in an iterative manner based on the learning from trying out alternate modes
of goal achievement in different contexts (Sabel & Zeitlin, 2012).
While the benefits of policy experiments to address situations of uncertainty and ambi-
guity are many, there are also challenges in their application. The first set of challenges
relates to the political aspects of experimentation in terms of the design, implementation
and evaluation of pilots. At times, pilot projects may be used as tools to avoid conflict
rather than enhance evidence-­based policymaking. In addition, if pilots address issues
that are politically contested, they may be delayed until more favourable political condi-
tions ensue (Jann & Wegrich, 2007). A contrary situation may also exist: there can be
34  Handbook of policy formulation

pressure to expedite evaluations to obtain timely ‘positive’ evidence to support certain

decisions (Sanderson, 2002) or to rapidly scale up pilots once initial positive results are
observed. This is problematic, however, for two reasons: firstly, it is important to ensure
sufficient capacity to properly conduct and sustain a pilot (PHR, 2004) and secondly,
initial results may need to be monitored to ensure that the positive results are sustained
over time before moving to the scaling-­up phase. Scholars have used results from labora-
tory and field experiments to provide policymakers and practitioners with evidence of the
impacts of selected experimental interventions as well as their feasibility and acceptability
by key stakeholders, including the intended beneficiaries. Behavioural variables at the level
of the individual are thus key factors in influencing the overall outcomes of such policy
experiments. While behaviour can be regulated to some extent with incentives, there are
limits on how much local observations can indicate the overall success or failure of a
scaled-­up experiment.
The second set of challenges relates to the evaluation of policy experiments. Policy
pilots are usually conducted as ‘one-­off evaluations’ to measure success (Stoker & John,
2009). In sectors that are rapidly changing or for those projects with a longer gestation
period between the start of a new experiment and the realization of its benefits, it may be
necessary to conduct repeated evaluations over time (Cabinet Office, 2003).
The third set of challenges relates to the scaling up of experiments. Positive results
from smaller-­scale experiments may not be observed when the experiments are scaled up
or applied in a different context (Simmons et al., 2007). The context dependency of pilots
and related dynamics means that pilots may not accurately predict the impact of diffusing
the project in a different context or scaling it up. In such cases, policy experiments should
be considered as an early evaluation of how specific policies or programmes might work
under certain conditions in certain settings.

CONCLUSIONS AND SCOPE FOR FUTURE RESEARCH

The historical context and theoretical background of treatment of uncertainty is a subject

of some interest to policy scholars. Uncertainties broadly emanate from both the quanti-
ties considered in policy models and the structure of those models themselves. Policies
can embody varying levels of uncertainty along this spectrum, from limited knowledge to
deep uncertainty or ‘unknown unknowns’.
As set out above, policy design under uncertainty is rather complicated in the case
of wicked problems, such as climate change, that have no clear agreement on causes or
solutions. As uncertainty rises, the level of knowledge about the system decreases, and
this alters the suitability of policy solutions to address specific policy problems. Efforts
to seamlessly integrate knowledge between the academic and policymaking communities
are also impeded by the presence of different perspectives, timescales and vocabularies
for concepts and processes, which make the transmission of knowledge difficult. A faulty
policy design owing to uncertainty can further hamper the effective functioning of poli-
cies and the realization of intended policy goals and objectives.
For long-­term policies that address complex and dynamic policy problems, there is
a need to constantly monitor and evaluate whether the policies continue to meet their
intended goals and objectives (Ramjerdi & Fearnley, 2013).
 The central conundrums of policy formulation  35

Adaptive policy-­making, scenario forecasting and policy experiments hold immense

potential to aid policy designers under uncertainty and ambiguity. Policy experiments can
be used to test many policies that are being deployed to deal with complex policy issues
and rapidly changing policy environments. Under normal conditions of uncertainty and
ambiguity, policy experimentation offers three major contributions to policy formulation:
promoting variation and diversification of risks; supporting evidence-­based policymak-
ing to prepare societies for transitions; and fostering social learning. Higher levels of
problems, however, require alternative formulation aids, such as scenario analysis and
adaptive policies.

NOTES

1. In tracing how uncertainty has been considered by policy scholars from the modern to post-­modern era in
the context of policy analysis and application, Bredenhoff-­Bijlsma (2010) highlights that while modernism
focused on the ‘positivist’ notion of using objective knowledge for policy analysis, post-­modernism focused
on the ‘socially constructed nature of scientific knowledge’ that emphasizes the role of actor interactions
(an idea central to network theory).
2. The concept of ‘surprise’ or unexpected changes has largely been used in the ecological context
(Lindenmayer et al., 2010), but offer little or no scope for the decision-­maker to respond from history or
experience (Lempert et al., 2003; Walker et al., 2010). Under such high levels of uncertainty, there is little
agreement on the choice of variables that should be included in models and it is difficult to assign probability
distributions to possible future scenarios with any confidence (Lempert et al., 2003; McInerney et al., 2011;
Walker et al., 2010).
3. In the development sector, policy experiments are frequently used to assess alternative courses of action.
These include (1) projects that focus on problem definition itself; (2) projects that focus on problems which
are partly or wholly undefined; (3) projects that explore the most effective way of achieving pre-­set policy
goals; (4) projects that aim to identify gaps and barriers in situations where problems and goals are already
well known; and (5) natural experiments that occur over a period of time without conscious intervention
(Rondinelli, 1993).

REFERENCES

Arentsen, M.J., Bressers, H.T.A. and O’Toole, L.J. (2000), ‘Institutional and policy responses to uncertainty in
environmental policy: a comparison of Dutch and U.S. styles’, Policy Studies Journal, 28 (3), 597–611.
Barnett, J. and O’Neill, S. (2010), ‘Maladaptation’, Global Environmental Change, 20 (2), 211–13.
Becker, S.W. and Brownson, F.O. (1964), ‘What price ambiguity? Or the role of ambiguity in decision-­making’,
Journal of Political Economy, 72 (1), 62–73.
Bengston, D.N., Kubik, G.H. and Bishop, P.C. (2012), ‘Strengthening environmental foresight: potential contri-
butions of futures research’, Ecology and Society, 17 (2), 10.
Bisaro, S., Hinkel, J. and Kranz, N. (2010), ‘Multilevel water, biodiversity and climate adaptation governance:
evaluating adaptive management in Lesotho’, Environmental Science and Policy, 13 (7), 637–47.
Bredenhoff-­Bijlsma, R. (2010), ‘Policy development under uncertainty. A framework inspired by cases of water
management’, PhD Thesis, Gildeprint, Enshede, the Netherlands.
Brown, M.L. (2000), ‘Scientific uncertainty and learning in European Union environmental policymaking’,
Policy Studies Journal, 28 (3), 576–96.
Brugnach, M., Dewulf, A., Pahl-­Wostl, C. and Taillieu, T. (2008), ‘Toward a relational concept of uncertainty:
about knowing too little, knowing too differently, and accepting not to know’, Ecology and Society, 13 (2),
30.
Bunce, M., Brown, K. and Rosendo, S. (2010), ‘Policy misfits, climate change and cross-­scale vulnerability in
coastal Africa: how development projects undermine resilience’, Environmental Science and Policy, 13 (6),
485–97.
Buurman, J., Zhang, S. and Babovic, V. (2009), ‘Reducing risk through real options in systems design: the case
of architecting a maritime domain protection system’, Risk Analysis, 29, 366–79.
36  Handbook of policy formulation

Cabinet Office. (2003), Trying it Out: The Role of ‘Pilots’ in Policy-­making, Report of a Review of Government
Pilots, Strategy Unit, Government of the United Kingdom.
Churchman, C.W. (1967), ‘Wicked problems’, Management Science, 14 (4), B141–B2.
Cooney, R. and Lang, A. (2007), ‘Taking uncertainty seriously: adaptive governance and international trade’,
European Journal of International Law, 18 (3), 523–51.
Crul, L. (2014), ‘Solving wicked problems through action learning’, Action Learning: Research and Practice,
11 (2), 215–24.
Da Costa, O., Warnke, P., Cagnin, C. and Scapolo, F. (2008), ‘The impact of foresight on policy-­making:
insights from the FORLEARN mutual learning process’, Technology Analysis & Strategic Management,
20 (3), 369–87.
Day, P. and Klein, R. (1989), ‘Interpreting the unexpected. The case of AIDS policymaking in Britain’, Journal
of Public Policy, 9, 337–53.
Dessai, S. and van der Sluijs, J.P. (2007), Uncertainty and Climate Change Adaptation – a Scoping Study,
Report NWS-­E-­2007-­198, Department of Science Technology and Society, Copernicus Institute, Utrecht
University.
Dewey, J. (1927), The Public and its Problems, New York: Holt and Company.
Durant, R.F. and Legge, J.S. (2006), ‘Wicked problems, public policy, and administrative theory: lessons from
the GM food regulatory arena’, Administration and Society, 38 (3), 309–34.
Eckles, D.L. and Schaffner, B.F. (2011), ‘Priming risk – the accessibility of uncertainty in public policy decision
making’, Journal of Insurance Issues, 34 (2), 151–71.
European Environment Agency. (2001), Late Lessons from Early Warnings: The Precautionary Principle
1896–2000, Environment Issue Report No. 22. Luxembourg: Office for Official Publications of the European
Communities.
Gersonius, B., Ashley, R., Pathirana, A. and Zevenbergen, C. (2013), ‘Climate change uncertainty: building
flexibility into water and flood risk infrastructure’, Climatic Change, 116 (2), 411–23.
Giordano, T. (2012), ‘Adaptive planning for climate resilient long-­lived infrastructures’, Utilities Policy, 23, 80–9.
Habegger, B. (2010), ‘Strategic foresight in public policy: reviewing the experiences of the UK, Singapore, and
the Netherlands’, Futures, 42, 49–58.
Hallegatte, S., Shah, A., Lempert, R., Brown, C. and Gill, S. (2012), ‘Investment decision making under deep
uncertainty – application to climate change’, Policy Research Working Paper Series 6193, World Bank.
Hansson, S.O. (1996), ‘Decision making under great uncertainty’, Philosophy of the Social Sciences, 26 (3),
369–86.
Head, B. (2008), ‘Wicked problems in public policy’, Public Policy, 3 (2), 101–18.
Herian, M.N., Hamm, J.A., Tomkins, A.J. and Pytlik Zillig, L.M. (2012), ‘Public participation, procedural
fairness and evaluations of local government: the moderating role of uncertainty’, Journal of Public
Administration Research and Theory, 22, 815–40.
Howlett, M. and Rayner, J. (2013), ‘Patching vs packaging in policy formulation: assessing policy portfolio
design’, Politics and Governance, 1 (2), 170–82.
Janicke, M. and Jorgens, H. (2000), ‘Strategic environmental planning and uncertainty: a cross-­national com-
parison of green plans in industrialized countries’, Policy Studies Journal, 28 (3), 612–32.
Jann, W. and Wegrich, K. (2007), ‘Theories of the policy cycle’, in F. Fischer, G.J. Miller and M.S. Sidney (eds),
Handbook of Public Policy Analysis: Theory, Politics, and Methods, Boca Raton, FL: CRC Press, Taylor &
Francis Group, pp. 43–62.
Jentoft, S. and Chuenpagdee, R. (2009), ‘Fisheries and coastal governance as a wicked problem’, Marine Policy,
33 (4), 553–60.
Jones, B.D. and Baumgartner, F.R. (2005), The Politics of Attention, Chicago, IL: University of Chicago Press.
Jordan, M.E., Kleinsasser, R.C. and Roe, M.F. (2014), ‘Wicked problems: inescapable wickedity’, Journal of
Education for Teaching, 40 (4), 415–30.
Kelly, J.A. and Volleburgh, H.R.J. (2012), ‘Adaptive policy mechanisms for transboundary air pollution
­regulation: reasons and recommendations’, Working Papers No. 2012.32, Fondazione Eni Enrico Mattei.
Knight, F.H. (1921), Risk, Uncertainty, and Profit, Boston, MA: Hart, Schaffner and Marx, Houghton Mifflin
Company.
Koppenjan, J. and Klijn, E.H. (2004), Managing Uncertainties in Networks. A Network Approach to Problem
Solving and Decision Making, London and New York: Routledge.
Kreuter, M.W., De Rosa, C., Howze, E.H. and Baldwin, G.T. (2004), ‘Understanding wicked problems: a key to
advancing environmental health promotion’, Health Education and Behaviour, 31 (4), 441–54.
Kwadijk, J.C.J., Haasnoot, M., Mulder, J.P.M. et al. (2010), ‘Using adaptation tipping points to prepare for
climate change and sea level rise: a case study in the Netherlands’, Wiley Interdisciplinary Reviews Climate
Change, 1 (5), 725–40.
Lempert, R.J., Popper, S.W. and Bankes, S.C. (2003), Shaping the Next One Hundred Years: New Methods for
Quantitative, Long-­term Policy Analysis, Santa Monica, CA: RAND.
 The central conundrums of policy formulation  37

Levin, K., Cashore, B., Bernstein, S. and Auld, G. (2012), ‘Overcoming the tragedy of super wicked problems:
constraining our future selves to ameliorate global climate change’, Policy Sciences, 45, 123–52.
Lindenmayer, D.B., Likens, G.E., Krebs, C.J. and Hobbs, R.J. (2010), ‘Improved probability of detection of
ecological “surprises”’, Proceedings of the National Academy of Science of the United States of America,
107 (51), 21957–62.
Loorbach, D. and Rotmans, J. (2010), ‘The practice of transition management: examples and lessons from four
distinct cases’, Futures, 42 (3), 237–46.
Major, E., Asch, D. and Cordey-­Hayes, M. (2001), ‘Foresight as a core competence’, Futures, 33 (2), 91–107.
Mathijssen, J., Petersen, A., Besseling, P., Rahman, A. and Don, H. (2008), Dealing with Uncertainty in
Policymaking, The Hague/Bilthoven/Leiden: CPB/PBL/Rand Europe.
Maxim, L. and van der Sluijs, J.P. (2011), ‘Quality in environmental science for policy: assessing uncertainty as
a component of policy analysis’, Environmental Science and Policy, 14, 482–92.
McInerney, D., Lempert, R. and Keller, K. (2011), ‘What are robust strategies in the face of uncertain climate
threshold responses?’, Climatic Change, 112, 547–68.
Morgan, M.G. and Henrion, M. (1990), Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk
and Policy Analysis, Cambridge: Cambridge University Press.
Pelling, M. (2011), Adaptation to Climate Change: From Resilience to Transformation, London: Routledge.
PHR (Partners for Health Reformplus). (2004), The Role of Pilot Programs: Approaches to Health Systems
Strengthening, Bethesda, MD: PHRplus, Abt Associates Inc.
Polasky, S., Carpenter, S.R., Folke, C. and Keeler, B. (2011), ‘Decision-­making under great uncertainty: environ-
mental management in an era of global change’, Trends in Ecology and Evolution, 26 (8), 398–404.
Ramjerdi, F. and Fearnley, N. (2013), ‘Risk and irreversibility of transport interventions’, Transportation
Research Part A, 60 (issue C), 31–9.
Rammel, C. and van den Bergh, J.C.G.M. (2003), ‘Evolutionary policies for sustainable development: adaptive
flexibility and risk minimising’, Ecological Economics, 47, 121–33.
Ranger, N., Reeder, T. and Lowe, J. (2013), ‘Addressing deep uncertainty over long-­term climate in major
infrastructure projects: four innovations of the Thames Estuary 2100 Project’, European Journal of Decision
Process, 1, 233–62.
Rittel, H.W.J. and Webber, M.M. (1973), ‘Dilemmas in a general theory of planning’, Policy Sciences, 4 (2),
155–69.
Rondinelli, D. (1993), Development Projects as Policy Experiments: An Adaptive Approach to Development
Administration, 2nd edn, New York: Routledge.
Sabel, C. and Zeitlin, J. (2012), ‘Experimentalist governance’, in D. Levi-­Faur (ed.), The Oxford Handbook of
Governance, Oxford: Oxford University Press, pp. 169–83.
Sanderson, I. (2002), ‘Evaluation, policy learning and evidence-­based policymaking’, Public Administration,
80 (1), 1–22.
Schneider, S.H. and Kuntz-­Duriseti, K. (2002), ‘Uncertainty and climate change policy’, in S.H. Schneider,
A. Rosencranz and J.-­O. Niles (eds), Climate Change Policy: A Survey, Washington, DC: Island Press,
pp. 53–88.
Simmons, R., Fajans, P. and Ghiron, L. (eds) (2007), Scaling up Health Service Delivery: From Pilot Innovations
to Policies and Programmes, Geneva: World Health Organization.
Simon, H.A. (1973), ‘The structured of ill-­structured problem’, Artificial Intelligence, 4, 181–201.
Smith, M.S., Horrocks, L., Harvey, A. and Hamilton, C. (2010), ‘Rethinking adaptation for a 4° C world’,
Philosophical Transactions of the Royal Society, 369, 196–216.
Stirling, A. (2010), ‘Keep it complex’, Nature, 468 (7327), 1029–31.
Stoker, G. and John, P. (2009), ‘Design experiments: engaging policy makers in the search for evidence about
what works’, Political Studies, 57 (2), 356–73.
Swanson, D. and Bhadwal, S. (eds) (2009), Creating Adaptive Policies: A Guide for Policymaking in an Uncertain
World, New Delhi and Ottawa: Sage and International Development Research Centre.
Taeihagh, A., Bañares-­Alcántara, R. and Givoni, M. (2013), ‘A virtual environment for the formulation of
policy packages’, Transportation Research A, 60, 53–68.
Turnpenny, J., Radaelli, C.M., Jordan, A. and Jacob, K. (2009), ‘The policy & politics of policy appraisal: emerg-
ing trends & new directions’, Journal of European Public Policy, 16 (4), 640–53.
Van der Heijden, J. (2014), ‘Experimentation in policy design: insights from the building sector’, Policy Sciences,
47 (3), 249–66.
Van der Pas, J.W.G.M., Kwakkel, J.H. and Wee, B.V. (2012), ‘Evaluating adaptive policymaking using expert
opinions’, Technological Forecasting & Social Change, 79, 311–25.
Vreugdenhil, H., Slinger, J., Thissen, W. and Ker Rault, P. (2010), ‘Pilot projects in water management’, Ecology
and Society, 15 (3), 1–13.
Walker, W.E., Rahman, S.A. and Cave, J. (2001), ‘Adaptive policies, policy analysis, and policy making’,
European Journal of Operational Research, 128, 282–289.
38  Handbook of policy formulation

Walker, W., Marchau, V. and Swanson, D. (2010), ‘Addressing deep uncertainties using adaptive policies’,
Technological Forecasting and Social Change, 77 (6), Special Section 2, 917–23.
Walker, W.E., Lempert, R.J. and Kwakkel, J.H. (2012), Deep Uncertainty, Delft and Santa Monica, CA: Delft
University of Technology and RAND.
Walker, W.E., Lempert, R. and Kwakkel, J.H. (2013a), ‘Deep uncertainty’, in S.I. Gass and M.C. Fu (eds),
Encyclopedia of Operations Research Management Science, New York: Springer Science, pp. 395–402.
Walker, W.E., Haasnoot, M. and Kwakkel, J.H. (2013b), ‘Review. Adapt or perish: a review of planning
approaches for adaptation under deep uncertainty’, Sustainability, 5, 955–79.
Walter, C.J. (1992), ‘Perspectives on adaptive policy design in fisheries management’, in K. Jain and
L.W. Botsford (eds), Applied Population Biology, Dordrecht: Kluwer Academic Publishers, pp. 249–62.
Wardekker, J.A., de Jong, A., Knoop, J.M. and van der Sluijs, J.P. (2010), ‘Operationalising a resilience approach
to adapting an urban delta to uncertain climate changes’, Technological Forecasting and Social Change, 77,
987–98.
Weber, E.P. and Khademian, A.M. (2008), ‘Wicked problems, knowledge challenges and collaborative capacity
builders in network settings’, Public Administration Review, 68 (2), 334–49.
Webster, M. (2003), ‘Communicating climate change uncertainty to policy-­makers and the public’, Climatic
Change, 61 (1–2), 1–8.
Weiss, C. (1975), ‘Evaluation research in the political context’, in E.L. Struening and M. Guttentag (eds),
Handbook of Evaluation Research, Los Angeles, CA: Sage, pp. 13–26.
World Resources Report. (2011), Decision Making in a Changing Climate, Washington, DC: World Resources
Institute.