Ballot Marking Errors in Ranked-Choice Voting

Stephen Pettigrew∗ Dylan Radley†

December 20, 2023

Abstract

Ballots are meant to be an expression of a voter’s electoral preferences. As more states and local

jurisdictions adopt ranked choice voting (RCV), they face a trade-off between allowing their voters to

express more complex preferences and the more complicated ballot potentially yielding higher rates of

ballot-marking errors and ballot rejections. What is missing from debates about the use of RCV is

empirical estimates of rates of improper marking and rejection rates, and how they compare to those

rates on traditional, single-mark ballots. We contribute to this debate by developing a general framework

for categorizing ways voters can improperly mark a single-mark or ranked ballot. We then apply that

framework to what we believe is the most comprehensive database of ranked choice ballots ever assembled–

including over 3 million cast vote records in four different electoral jurisdictions that use RCV. The data

show that in a typical ranked choice race, nearly 1-in-20 (4.8%) voters improperly marks their ballot in at

least one way. The rate of one type of improper marking (overvoting) is 14 times more likely to occur on

a ranked choice race than a non-ranked choice race that appears on the same ballot. Furthermore, we

find that votes in ranked choice races are nearly 10 times more likely to be rejected due to an improper

mark than votes in non-ranked choice races. These findings raise key normative questions about voter

participation and representation in ranked choice systems and have important policy implications for

jurisdictions that already have or are considering adopting ranked choice voting.

∗ University of Pennsylvania
† University of Pennsylvania

Electronic copy available at: https://ssrn.com/abstract=4670677

1 Introduction

In a democracy, it is essential that voters can express their electoral preferences. Voters express those

preferences through ballots (Katz 1997), and in the United States a new kind of ballot is gaining momentum

through the adoption of Ranked Choice Voting (RCV). It has been used in a handful of municipalities for

decades, but in the past five years its use has grown, being adopted by New York City, the largest city in the

country, for municipal elections in 2021. It has also been used in two states, Maine and Alaska, for Federal

races since 2018 and 2022 respectively, and are on track to be joined by a third, Nevada, if voters re-approve

a ballot measure in 2024. RCV’s growing momentum makes it an important subject of study as millions of

voters start to use it as a new way to express their preferences.

This is especially true given that ballot design itself can influence how final vote counts reflect the attitudes

of voters. Researchers have explored this for typical single-mark ballots for decades (Cottrell et al. 2022;

Kimball and Kropf 2005)—most notably in the 2000 election (Wand et al. 2001; Herron and Sekhon 2003).

There have been some studies examining this for RCV (Graham-Squire and McCune 2024), alongside a good

deal of debate emanating from news and advocacy organizations (Shabad 2022; “Sarah Palin’s Defeat in

Alaska Proves Ranked-Choice Voting Works” 2022). Many studies have evaluated the tabulation methods

used in RCV, but this is only one part of how voter preferences are translated into vote counts. The second

part is the focus of this paper: how voters engage with their ballots, specifically, the ranked ballots use for

RCV.

This paper evaluates whether the more complex ballots used for RCV lead to more errors and ballot rejections.

We start by building on previous work on voter confusion (Atkeson et al. 2023) and voter irrationality (M.

Alvarez, Hall, and Levin 2018) in ranked choice systems to create an exhaustive framework for characterizing

all possible ways in which a voter can improperly “bubble-in” a ranked choice ballot. Importantly, we

distinguish between filling out a ballot in a politically strategic way–perhaps by not utilizing all available

rankings–and improper marks that reflect a mis-understanding of the ranked choice system, potentially

(although not necessarily) resulting in the vote not being counted. Our focus in this article is quantifying

prevalence of the latter set of cases.

We then apply this framework to what we believe to be the largest database of cast vote records of ranked

choice votes that have been studied. Cast vote record data from nearly 3 million ballots in Alaska, Maine,

New York, and San Francisco allow us to identify ballot improper marks at the individual vote/ballot level.

Furthermore, because most of the cast vote records in our database include both ranked choice and non-ranked

choice offices, we can compare error and rejection rates between the two groups of offices that appeared on

Electronic copy available at: https://ssrn.com/abstract=4670677

the same ballot. We find that for offices using ranked choice voting, about 5 percent of voters improperly

mark their ballot. We also demonstrate that voters are 14 times more likely to overvote— a type of error

found in both ranked and traditional, single-mark ballots— in ranked choice races than single-mark races

on the same ballot. Importantly, not every improper marking results in a ballot rejection: we find that on

average 0.35% of ballots are rejected in the first round of tabulation, with rejection rates increasing to 0.53%

in the final round of tabulation1 . Crucially, however, we also find that rejection rates tend to be more than

an order of magnitude higher in RCV than in non-RCV races. All of these findings hold true after accounting

for high rates of ballot exhaustion in RCV races,2 though it is worth noting that ballot exhaustion—the

literature for which we cover in Section 2— can also affect how voter preferences are translated into vote

counts under RCV.

Our paper proceeds as follows. In Section 2 we describe previous work on RCV, specifically highlighting

research about voter confusion and ballot rejection. In Section 3 we then detail our framework for identifying

improper marks and explain how it can be used to better understand how voters engage with their ballot.

We then describe our cast vote record data (Section 4), underscoring the breadth and depth of our database.

We use that data in Section 5 to measure how often voters improperly mark their ranked choice ballots

and provide an initial exploration of the correlates of these error rates, specifically the number of available

rankings, voter roll-off, and office salience. In Section 6 we show that while most ranked choice ballots with

an improper mark are ultimately counted, the ballot rejection rates in ranked choice offices are still more than

an order of magnitude higher than those in non-ranked offices. We conclude in Section 7 with a discussion of

the academic and policy implications of our findings, as well as the future avenues for research on the topic.

2 Ranked choice versus single-mark ballots

Our focus in this research is on ranked ballots, which have become increasingly utilized by American voters in

recent years. We distinguish multiple-mark, ranked ballots from traditional, single-mark ballots (Rae 1967).3

The vast majority of elected offices in the United States are elected via a single-mark ballot, on which voters

cast their vote for a single candidate. There are two major differences between elections that used ranked

ballots and those that use single-mark ballots.

The first of these differences is how ballots are tabulated and winners are determined. Most (although not
1 Section3 illuminates how rejection rates can increase from the beginning to the end of tabulation.
2 In
ranked choice voting, a ballot is exhausted when all candidates for whom the voter provided a ranking have been eliminated
during vote counting.
3We recognize that there are both other ballot types (such as approval voting, cumulative voting, or grading systems) and

many rules for counting votes on a ranked ballot (such as the Borda and Condorcet methods or multi-winner districts). Our
focus in this paper, however, is exclusively on ranked ballots and traditional, single-mark ballots in single-winner elections using
either Ranked Choice Voting or plurality winner tabulation procedures.

Electronic copy available at: https://ssrn.com/abstract=4670677

all) elections on a single-mark ballot utilize a plurality rule for determining the winner. Ranked ballots

may be used in conjunction with one of several tabulation methods (Santucci 2021; Maloy 2019; Blais 1988;

McCune and McCune 2022). The most common of these methods in the United States is an instant runoff

system,4 where candidates with the fewest votes are sequentially eliminated and voters’ preferences among

the remaining candidates are counted until one candidate has a majority of support on the remaining ballots

(Santucci 2021; Miller 2017; Graham-Squire and Zayatz 2021).

In this paper, our main focus is on the second feature that distinguishes ranked choice from single-mark

ballots: the design of the ballot itself and how voters engage with it. On a race utilizing a single-mark ballot,

voters are presented with one bubble per candidate (or button, if they are voting on an electronic machine).

For their vote to be counted, the voter must bubble-in or press the button for exactly one candidate. On

a ranked choice ballot, there are one bubble/button per candidate times the number of available rankings,

which either equals the number of candidates or some upper limit defined by law, and voters are asked to

rank-order their preferred candidates. Whether their vote is included in the first, intermediate, and final

vote-counting rounds depends both on whether they had ballot marking errors, where those errors were on

their ballot, and whether the candidates they ranked on their ballot had enough support from other voters to

not be eliminated from contention.

Our work on improperly marking ballots builds on a growing literature about how RCV impacts political

behavior. McDaniel (2016) evaluates its impact on turnout of minority voters, and McDaniel (2018) assesses

its implications on racially-polarized voting. Clark (2020) also considers the impact that RCV has on the

cost of voting, finding that the “costs of ranked-choice voting. . . vastly outweight the benefits of the system of

voting.” Other scholars have studied whether candidates of color perform better under RCV (Crowder-Meyer,

Gadarian, and Trounstine 2023), if descriptive representation by women and minorities is improved (John,

Smith, and Zack 2018; Terrell, Lamendola, and Reilly 2021), or if campaign messaging is more civil (Donovan

and Tolbert 2023; Kropf 2021), potentially increasing turnout in the process (Juelich and Coll 2021).

Other scholars have also focused on the prevalence of ballot truncation in ranked choice voting–voters who

do not use all the rankings available to them. These studies generally find that ballot truncation is quite

common and can have effects on the outcome (Kilgour, Gregoire, and Foley 2020), from affecting who the

winner is (Tomlinson, Ugander, and Kleinberg 2022) to resulting in winners elected with less than a majority

of the original ballots cast (Burnett and Kogan 2015). In our study, we do not consider ballot truncation to

be a form of improper marking. Voters may have a sincere preference against ranking some candidates on

their ballot (McCune and Wilson 2023; Bartholdi and Orlin 1991). Others may choose to only rank a single
4 In Australia, this system is called the ‘Alternative Vote’ (Bean 1997).

Electronic copy available at: https://ssrn.com/abstract=4670677

candidate as a form of protest against ranked choice voting itself. In our analysis of improper marks, we limit

ourselves to situations where it is difficult the believe that voters are improperly marking their ballots for

strategic reasons, rather than because of confusion.

Our study most closely builds on previous work about the impact that ranked ballot complexity has on voter

confusion. Most of these studies utilize survey data to show that voters tend to find ranked ballots more

complicated than single-mark ballots (Curtice and Marsh 2014). In some cases, studies found that older

voters found ranking more difficult or had less understanding of RCV rules (Coll 2021; Donovan, Tolbert,

and Gracey 2019), thought not in others (Donovan, Tolbert, and Harper 2022). The literature finds few

systemic differences in understanding of RCV rules across racial/ethnic lines, though there does appear to be

such a relationship with education (Coll 2021; Donovan, Tolbert, and Gracey 2019; Donovan, Tolbert, and

Harper 2022). More recently, Atkeson et al. (2023) use exit polls to demonstrate that a considerable number

of voters, especially Hispanic ones, reported confusion on their ranked ballots.

By analyzing actual rates of improper marks, rather than self-reported rates of confusion, we are able to

better understand how voters are interacting with the ranked choice system. Our empirical approach is more

comprehensive than prior work, but we are not the first to study ballot errors in ranked choice systems (M.

Alvarez, Hall, and Levin 2018 ). Previous studies of invalid ballots date as far back as the Progressive Era,

when ranked-ballot systems were adopted in jurisdictions across the US (Mott 1926). More recent work,

mostly on San Francisco, has shown that overvote rates are higher in RCV elections, and that voters of color

and older voters overvote at higher rates using RCV Neely and Cook (2008).

3 Marking a ranked choice ballot

Because ranked ballots are more complex than single-mark one, there are more ways for a voter to make

a mistake in filling it out. In this section, we provide a general framework for thinking about the types of

mistakes a voter can make when filling out a ranked or single-mark ballot,5 which represent the vast majority

of ballots used for elections in the US. This framework provides for three types of improper marks that a

voter can make: overvotes, overranks, and skips.6 Later in this this section we describe these error types in

more detail.

We conceptualize two ways in which a voter can “correctly” mark a ranked choice ballot. First, a complete
5 In some ways, a single-mark ballot could be considered to be a special case of a ranked ballot. Voters only get to rank one

candidate, and the winner is determined by sequentially removing the candidates with the smallest number of votes until one
candidate has more than half of the remaining votes.
6 This framework formally categorizes the ballot marking approaches that (M. Alvarez, Hall, and Levin 2018) describe as

being “inconsistent with individual rationality”.

Electronic copy available at: https://ssrn.com/abstract=4670677

 Figure 1: Examples of properly marked ranked ballots

ranked ballot is one in which the voter utilizes all available rankings, and does not choose any candidate

more than once. The left panel of Figure 1 shows a complete ballot cast in the November 2022 Alaska Senate

election. This voter’s preferences were (in order) Murkowski, Chesbro, Kelley, write-in, and Tshibaka.7 In

jurisdictions that allow its voters to rank a write-in candidate, we consider a ballot to be “complete” even if

they did not specify a write-in choice for one of their rankings.8 This choice has no bearing on the empirical

results we present later in this paper.

The second group of properly marked ranked ballots are truncated ballots. A ballot is truncated if the voter

ranked one candidate for each ranking 1 through n, where n is less than the total number of available rankings,

and no candidate or ranking is used multiple times. The right side of Figure 1 provides an example of a

truncated ballot, where the voter’s rankings were Chesbro, Murkowski, and Kelley, leaving the fourth and

fifth ranking options blank. Although some voters may truncate their ballots due to confusion about ranked

choice rules, we do not consider ballot truncation to be a type of improper marking since it may convey a

sincere expression of the voter’s preferences.9

In our analysis, we treat both complete and truncated ballots as correctly marked ballots. We now turn to

describing the three ways in which a ballot can be improper marked, which may sometimes result in the
7 The ballot image reproductions throughout his section come from Alaska’s cast vote records data, which we describe in

more detail later. The data do not include information about which write-in votes were actually cast for.
8 RCV may make it necessary to revisit rules for counting write-in votes, specifically in instances where a voter writes in the

name of a candidate who appears on the ballot. For example, some states count write-in voters for a named candidate toward
that candidate’s overall total during a recount. If that state adopted RCV, it would need to provide guidance on how such
write-ins should be handled during not just the final round of ranked choice vote counting, but also in intermediate rounds
where the order of candidate elimination can have cascading effects on the final winner of a race (Tucker et al. 2022; Williams,
Baltz, and Stewart 2023).
9 It is also worth noting that a truncated ballot that ranks all but one named candidate is identical to a complete ballot

from the perspective of tabulating the votes. At the latest, the winner of a race is guaranteed to be chosen when there are two
candidates left. If those candidates happen to be a voter’s last and second-to-last preferences, their vote will be counted for the
second-to-last preference, regardless of whether they filled in the final ranking column.

Electronic copy available at: https://ssrn.com/abstract=4670677

voter’s ballot not being counted.

3.1 Overvoting

The first category of improper markings are overvotes. An overvote occurs when a voter selects more than

one candidate for a given ranking. Of the three types of ballot marking errors we describe in this section,

overvoting is described most extensively in the literature, since it is a type of error that can occur on a

single-mark ballot (Kimball and Kropf 2005). Most of this literature couples overvotes with undervotes

(i.e. leaving an office completely blank) to calculate the residual vote rate, which has been used to study

problems with voting technology (R. M. Alvarez, Beckett, and Stewart III 2013; Sinclair and Alvarez 2004),

ballot design (Kimball and Kropf 2005; Cottrell et al. 2022; Herron et al. 2023), and expressive non-voting

(Stewart et al. 2020).

Figure 2: Examples of overvotes

Overvoting on a ranked ballot is similar to overvoting on a single-mark ballot, but with more opportunities

for it to happen. We conceptualize an overvote as a feature of a numeric ranking (i.e. the columns of the

example ballots). Figure 2, shows two examples of overvotes from the 2022 Alaska Senate election. The

ballot on the left side includes first-choice votes for both Chesbro and Tshibaka and is thus an overvote in

column 1. The ballot on the right has an overvote in the second column.

These two examples highlight an important aspect of improperly marked ranked ballots: improperly marking

a ranked ballot does not necessarily preclude a vote from being counted. The vote on the left side of Figure 2

would never be counted because the voter’s intent for their first-choice preference is ambiguous. But because

Tshibaka was not eliminated during vote tabulation, the vote on the right side of that figure was counted in

the final tabulation. If their first-choice vote had instead been for Kelley (who was eliminated after the first

Electronic copy available at: https://ssrn.com/abstract=4670677

round), then the vote would not have been included in the final tally. As we show in Section 6, ranked ballots

with an overvote usually, but do not always result in a voided vote. This is an important distinction from

overvoting on a single-mark ballot, which always yields an invalidated/spoiled vote (Kimball and Kropf 2005).

3.2 Skipping

The second category of improper markings are skips, which are definied as leaving a ranking blank, but filling

in a subsequent ranking. This definition distinguishes skips from truncated ballots, wherein a voter selects

candidates for rankings 1 through n, but leaves blank all rankings after n. Skipping can come in one of two

forms.

Figure 3: Examples of skips

The first form, demonstrated on the left side of Figure 3, is an interior skip. They occur when a blank ranking

column is preceded and succeeded by non-blank ranking columns. The voter in this example ballot skipped

rankings 2 and 4, but marked rankings 1, 3, and 5. The second form of skipping is a front skip, which occurs

when the voter leaves blank the first n rankings, but marks a vote in ranking n + 1. The right side of Figure

3 shows a ballot with a front skip. This ballot also illustrates that a voter can both front and interior skip on

the same race.

More than overvotes, whether or not a skip results in a voided vote depends on the vote counting rules of

the jurisdiction. Some jurisdictions, like New York City and San Francisco, ignore skips in the tabulation

process and would treat each of the ballots in Figure 3 as if the voter had utilized rankings 1, 2, and 3 and

left blank 4 and 5 (New York City Charter 2023; San Francisco 2023). Alaska and Maine’s vote counting

rules terminate the counting of a ballot if the tabulation algorithm encounters more than one consecutive

skipped ranking. In the case of the ballots in Figure 3, both would be counted under this rule, but if the

Electronic copy available at: https://ssrn.com/abstract=4670677

ballot on the right had ranked Chesbro third instead of second, the whole ballot would have been thrown out

because of a double front skip.

In our analysis we consider skips to be a form of improper marking a ballot. The only scenario we can imagine

where a voter would skip rankings for the purpose of expression would be if they ranked a disliked candidate

last, with front or interior skips immediately before. But doing this is also probably an indication that the

voter does not understand the vote tabulation rules, since (depending on their jurisdiction’s rules about

ignoring skipped rankings) it could result in their vote being counted for this extremely disliked candidate.

We believe the population of voters who would cast their ballot in this way and also have sufficient knowledge

about the nuances of ranked choice vote counting is small enough to justify treating all skips as errors that

result from voter confusion about ranked choice rules.

3.3 Overranking

Figure 4: Examples of overrankings

The final category of ballot marking errors is overrankings, which occurs when a voter selects the same

candidate for more than one ranking. Figure 4 shows two overrankings in the Alaska Senate race. The voter

on the left overranked Murkowski by voting for her as their second and fourth choice. The voter on the right

overranked both Tshibaka and Murkowski by selecting each multiple times.

In contrast to overvoting, which has a high chance of resulting in an uncounted ballot, overranking a candidate

has a much smaller chance of yielding a voided ballot. The ballot on the left of Figure 4 is guaranteed to be

counted, since all named candidates are ranked and the over-ranking of Murkowski occurs after valid rankings

for all other candidates. Whether the ballot on the right of that figure would be counted depends on whether

or not Tshibaka is eliminated, and if the jurisdiction’s election law treats overrankings in choices 2 and 3 as if

Electronic copy available at: https://ssrn.com/abstract=4670677

they are skips and applies its rules about skipping to determine if the vote for Murkowski would be counted10

It is also worth noting that that (unlike over-voting) over-ranking could be a form of political expression,

rather than a manifestation of voter confusion. Take, for example, a voter who selects the same candidate

for every available ranking and omits all other candidates from their ballot. This pattern could result from

voter confusion, but may also be an expression of a particularly strong preference for this candidate and

dislike of the others. Our data suggest that this type of overranking is the modal pattern of overranking, but

it still makes up a small minority of all overranked ballots. For this reason, we treat all ballots with any

overranking as improperly marked, but all our analyses provide enough detail to allow the reader to interpret

our conclusions based on making the opposite decision.

4 Cast vote record data of ranked choice races

We now turn to assessing the prevalence of each types of marking errors on ranked ballots. To do this, we

utilize cast vote record (CVR) data, which records all markings on a voter’s ballot, rather than just the fina,

aggregate vote totals. Researchers have used CVRs to gain greater insight into voting behaviors such as

ticket splitting (Kuriwaki 2023; Agadjanian and Robinson 2019), party cues in low-information elections (M.

Alvarez, Hall, and Levin 2018), and under/overvoting (Herron and Sekhon 2003). CVRs have also been used

to study aspects of ranked choice voting, such as the prevalence of overvotes (Neely and Cook 2008; Neely

and McDaniel 2015), ballot exhaustion and truncation (Burnett and Kogan 2015; Kilgour, Gregoire, and

Foley 2020), and to compare RCV to other preferential voting methods (McCune and McCune 2022).

For this study, we assembled what we believe to be the largest CVR database of ranked ballots, with about

3 million ballots included. These data span ten election events in four jurisdictions, which when combined

represent approximately 80% of the voters who use RCV(“Where Is Ranked Choice Voting Used?” 2022). In

three of the jurisdictions (Alaska, New York City, and San Francisco) the CVRs provide us ballot marking

information for all offices (ranked choice or single-mark). In Maine, the CVR data only include ranked choice

races which went to multiple rounds of tabulation to determine a winner.11

These jurisdictions are some of the most noteworthy places to have used ranked choice voting in the United

States. In 2002, San Francisco became the first major city to adopt RCV for its local elections (San Francisco

2023). Maine was the first state to adopt RCV for federal primary and general elections and state-level

primary elections in 2018 (Santucci 2018). New York City is the largest US jurisdiction to adopt RCV, using

it for its Mayoral and City Council primary elections since 2021 (Montanaro 2021). And in 2022, Alaska
10 In Alaska, where overranks are treated as skips, the ballot on the right’s vote for Murkowski would not be counted.
11 For this reason, we omit the Maine data from some of our analyses where its inclusion could produce selection bias.

10

Electronic copy available at: https://ssrn.com/abstract=4670677

became the first state to utilize RCV for all federal, statewide, and state legislative general elections (Thiessen

2022). Our database includes 100% of counted ballots in Maine, New York City, and San Francisco. The

data in Alaska are over 99.9% complete, with 253 missing ballots in the August 2022 Special Election and

250 in the November 2022 General Election.

Although the cast vote records do not provide an indication of which ranking was ultimately counted in

the final round of tabulation, we have applied each jurisdiction’s vote counting algorithm to identify the

first-choice and final-counted preference of every voter in every race in our database. In Maine, New York

City, and San Francisco, we perfectly replicate the official vote counts of every round of tabulation for every

office. In Alaska, we almost perfectly replicate the results, and are no more than about 250 votes off (due

to missing data). In our analyses, we omit any races where a single candidate ran unopposed. We also

omit completely undervoted races (i.e. no marks or selections made by the voter for the race), so all of our

estimates of error rates are conditional on a voter having made some sort of mark or selection.

5 Improper marking rates on ranked ballots

We start by examining improper marking rates on the 3.09 million ballots that voted in the 165 ranked choice

races in our dataset. Figure 5 shows the percentage of voters in each of these races who skipped, overranked,

or overvoted in each race, as well as the percentage who committed at least one of these errors. Each point

in this graph represents one of the 165 elections, colored and shaped based on their electoral jurisdiction

(i.e. AK, ME, NYC, or SF). The x-axis shows the type of improper marking, and the y-axis the percent of

voters who made that error. The small, horizontal, colored bars show the average error rate within each

jurisdiction, and the larger black bars show the error rate (and 95% confidence interval) averaged across

geographies.

The average error rate across jurisdictions was 4.8%, meaning that for a typical ranked-choice race nearly 1

in 20 voters improperly marked their ballot.12 Each jurisdiction had different error rates: rates tended to be

lowest in Alaska, averaging around 2% (1 in 50 voters), and highest in San Francisco and Maine, averaging

around 6.7% (1 in 15 voters). This is particularly noteworthy because it cuts against the idea that improper

marking rates will decrease as voters become more accustomed to voting on a ranked choice ballot. The data

in Alaska are from the first time voters used RCV, while the data from San Francisco come from elections

that occurred more than a decade after RCV was implemented.

Figure 5 also reports the rates for each type of error. For an average ranked-choice race, 2.0% of voters front
12 This average error rate is consistent with M. Alvarez, Hall, and Levin (2018), who find the rate of “irrational” candidate

rankings to be 6.43%, 3.84% and 5.33% in the three RCV elections in Pierce County, Washington.

11

Electronic copy available at: https://ssrn.com/abstract=4670677

 Figure 5: Office-level error rates in 165 ranked choice races

or interior skipped somewhere in their rankings for that race. In Alaska, rates of skipping were the lowest at

1.2%, while they were the highest in New York City, at 2.9%.

On average in ranked choice races, 2.4% of voters overranked at least one candidate, with high variance

across jurisdictions. Only about 0.8% (< 1 in 100) of voters overranked in Alaska races, while about 4.4%

(1 in 20) overranked in Maine races. Maine also features one anomalous race (the 2022 ME-02 US House

race) where over 20% of voters overranked their ballot. Most of these overrankings followed the pattern of a

voter choosing the same candidate for every ranking. While this datapoint is certainly an extreme outlier,

its omission only decreases the overall overranking rate from 2.4% to 2.3%. It is also worth noting that the

overranking rates in New York City and Maine are almost certainly underestimates of the true rate because

of how the data are reported. The cast vote records in these jurisdictions do not indicate which candidates

were included in an overvote (the data simply read “overvote” for that ranking), so if a voter overvoted and

overranked we would not be able to observe the overranking.

The final error type is overvoting, which was the least common improper mark. On average, 0.6% of voters

overvoted at least one of the rankings on a ranked choice race. About 0.2% of voters overvoted in Alaska,

compared to 0.9% in New York City. In addition, we find that around 0.6% of voters overvote in San

12

Electronic copy available at: https://ssrn.com/abstract=4670677

Francisco, which is consistent with other studies of overvoting in San Francisco (Neely and Cook 2008; Neely

and McDaniel 2015). While these numbers are certainly smaller than the rates of overranking and skipping,

overvoting is also more likely to yield a voided, uncounted ballot, which we explore further in Section 6.

Figure 6: Overvoting rates in ranked choice versus single-mark races

While these rates may seem relatively small, the CVR data allows us to compare the overvote rate in these

ranked choice races to the overvote rate on the non-ranked choice races from the exact same ballots. Figure 6

shows these rates for the six election dates in which we have ranked choice and non-ranked choice data. The

red lines showing the average overvote rate (and 95% confidence interval) for non-ranked choice races (one

circle for each race) and the blue lines representing the averages (and confidence interval) for ranked choice

races (triangles).

This graph makes clear that although the magnitude of overvoting may seem small, it is extremely high when

compared to the overvote rates typical of single-mark races. Averaging across these six these election dates,

the average overvoting rate for ranked choice races is 0.60%–about 14 times higher than rate in non-ranked

choice races (0.04%).13 Each of the six election events in this graph had a statistically significant (p < .01)

jump in the overvote rate between non-ranked choice and ranked choice races. In NYC’s 2023 election, the

ranked-choice overvote rate was 3.3 times higher than the non-ranked choice rate. In the August 2022 primary
13 In Appendix 8.1, we replicate a finding in Neely and McDaniel (2015), which shows that the number of candidates is a

significant predictor of overvote rates in ranked and non-ranked races. We find, however, that the difference in overvote rate
between ranked and non-ranked races is substantively unchanged whether or not you control for the number of candidates.

13

Electronic copy available at: https://ssrn.com/abstract=4670677

in Alaska, it was 32 times higher. This consistent pattern across geographies is also noteworthy given that

ranked choice races appear in different locations on the ballot in each geography. In Alaska and New York

City, ranked choice races appear at the top of the ticket, while in San Francisco, they appear below a large

number of higher salience races.

While it is not possible to make similar comparisons to the skip and overranking rates (which are not possible

on single-mark ballots), the findings here suggest that voters are more likely to improperly mark their ballot

when voting in a ranked choice race. We now turn to identifying whether there are other features of the

ballot that can predict these error rates.

5.1 Improper marking rates by number of rankings

Figure 7: Improper marking rates by maximum number of possible rankings

One of the factors which seems to impact improper marking rates is the number of available rankings for

a given office. In Figure 7, each panel is a different type of improper mark, with the x-axis displaying the

number of candidates a voter could rank.14 The y-axis displays the percent of voters who made the given

error. Each dot is an office, with OLS best-fit lines showing the pattern.
14 Because we only focus on races with at least two candidates on the ballot, and because voters have the opportunity to rank

a write-in candidate, the fewest number of possible rankings is three.

14

Electronic copy available at: https://ssrn.com/abstract=4670677

Perhaps unsurprisingly, the more ranking opportunities a voter is given, the more likely they are to improperly

mark their ballot, although this pattern varies by the type of improper mark. For overvotes and skips, a

higher number of potential rankings correlates with significantly higher rates of voter errors. This pattern

holds when all races are pooled together, as well as when including fixed effects for geography.15 .

On the other hand, we do not find a significant bivariate relationship between the number of rankings and the

overranking rate and find a negative significant relationship when controlling for geography. If this finding

is not a spurious one, it would cut against the idea that providing more opportunities to rank candidates

will necessarily translate into higher error rates by voters. Future research should explore whether there is,

perhaps, something fundamentally different about ranked choice races that have just two candidates (plus a

write-in) versus ranked choice races with more.

Because of the conflicting directionality of these correlations, when all three types of errors are combined

together, we find no statistically significant relationship between the number of available rankings and the

probability that a voter has any type of improper mark. While this is encouraging, the fact that overvoting

rates increase with the number of candidates is worth considering further. Allowing voters to express more

detailed electoral preferences via the ranking process is touted as one of the normative benefits of ranked

choice voting. But the findings in Figure 7 highlights the cost of voters being more likely to overvote, which

as we show in Section 6 makes it highly likely that their vote will be uncounted.

5.2 Office salience and roll-off

The second major factor that seems to contribute to improper marking rates is the salience of the office being

elected. Previous research indicates that voters who lack information about candidates are more likely to

roll-off on down-ballot offices (Wattenberg, McAllister, and Salvanto 2000; Lamb and Perry 2020). It also

stands to reason that voters who roll-off for this reason are also the same type of voters who are more likely

to improperly mark their ranked choice ballot. If this hypothesis were true, we would observe that improper

marking rates would be lower for lower salience (down-ballot) races because the voters that attempt to vote

in them tend to be more informed about the election and perhaps politics more generally.

We test this hypothesis using data from the November 2022 Alaska general election. This election provides

a useful test because ranked choice voting was used in Alaska’s high salience US Senate, US House, and

Governor races, as well as for lower salience State House and State Senate races. We expect to find the error

rates in the state legislative races to be lower than those in the three statewide races.
15 For regression tables showing these results, see Appendix Section 8.2

15

Electronic copy available at: https://ssrn.com/abstract=4670677

 Figure 8: Top-of-ticket versus down-ballot error rates in Alaska

Figure 8 reports the ballot improper marking rates among the statewide, State Senate, and State House races.

Each point is an individual race and the horizontal bars (and vertical 95% confidence intervals) mark the

average error rate for each category. The clear pattern in the graph is that for all types of improper marking,

the average error rate is significantly higher for the three statewide races than for the state legislative races.

The overall improper marking rates are, on average, 3.7% in the statewide races and just under 2% for both

State Senate and House races. For skipping, overranking, and overvoting, the error rates in the statewide

races tend to be roughly twice the rates for the state legislative races. This is suggestive evidence in support

of our hypothesis.

To further investigate our hypothesis, we group voters based on whether they voted in their State House and

State Senate race, voted in just voted in one race or the other, and those who voted in neither. Figure 9

shows how the improper marking rates in statewide races differed between these three categories of voters.16

The idea here is that voters who skipped both state legislative races tend to be less politically informed and

thus more likely to make an error when voting for the higher profile statewide races.

The blue squares and 95% confidence intervals represent the improper marking rates of voters who skipped
16We omit the 17% of voters who lived in a State Senate or State House district where there was not a contested election.

16

Electronic copy available at: https://ssrn.com/abstract=4670677

 Figure 9: Top-of-ticket error rates in Alaska, conditional on down-ballot roll-off

both state legislative races on their ballot. For every statewide office and every type of improper mark, this

group of voters had higher error rates than the groups who voted in one (green triangles) or both (red circles)

state legislative races. With the exception of skipping in the governor’s race and overranking in the US

Senate race, there is a statistically significant (p < 0.01) difference in the error rates between those who voted

in both state legislative races and those who voted in neither.17 And with the exception of skips, there is a

statistically significant (p < 0.05) difference in the error rates between those who voted in at least one and

those who voted in neither.

These results suggest that the chances that a voter improperly marks their ballot is positively correlated with

their probability of rolling-off in down ballot races. The implication is that improper marks do not happen

randomly–certain types of voters seem to be more likely to do it. Understanding more about who these voters

are (such as their political attitudes or demographic traits) is an important avenue for future research.
17 For a table with these results, see Appendix Section 8.3.

17

Electronic copy available at: https://ssrn.com/abstract=4670677

6 Ranked ballots are more likely to be rejected

In the previous section, we showed that roughly 1-in-20 voters improperly marks their ranked choice ballot

in a way that suggests a misunderstanding about the proper way to fill it out. We also show that for the

one type of improper mark that allows for comparison to single-mark ballots–overvotes–the error rate is

roughly 14 times higher on ranked ballots than single-mark ones. The question we turn to in this section

is whether those high error rates translate into higher rates of ballot rejection. What we find is that while

most improper markings do not result in a rejected ballot, the rejection rate in ranked choice races is still

considerably higher than in non-ranked choice ones.

Figure 10: Rejection rates by error type (among those who made an error)

We begin by assessing the proportion of improperly marked rankings that actually result in the vote not

being counted due to the error. Figure 10 shows the percentage of improperly marked ballots that were not

included in the final-round vote count due to an error.18 In calculating the rejection rates, we omit from the

numerator “exhausted” ballots – those where a voter did not rank at least one of the candidates remaining in

the final round of tabulation–but leave them in the denominator. Excluding them from the denominator

would result in rejection rates that are even higher than those in the figure.

Among the 4.8% of votes that contain any type of improper mark, Figure 10 shows that only 9.8% of them
18 For this graph, the “final-round” vote includes the first round vote in any races that did not take multiple rounds to
determine a winner, as well as final-round votes in races that did require multiple rounds.

18

Electronic copy available at: https://ssrn.com/abstract=4670677

are ultimately rejected from counting19 . In Alaska and San Francisco, the ballot rejection rates were slightly

lower (5.1% and 5.8%, respectively), whereas in Maine and New York City the rates were higher (12.8% and

12.9%, respectively).20

By far, the type of improper mark that is most likely to translate into a rejected ballot is an overvote. Roughly

two-thirds of the time (65.8%) overvoting any ranking for a race results in the ballot being excluded from

the final round tabulation. This is particularly high when you consider that most elections in our data had

a winner determined in the first round of counting. This means that a large proportion of these rejections

occurred because the voter overvoted in their first ranking.

Ballots with overrankings are the least likely to be rejected. Only about 1.5% of them result in an uncounted

vote. The jurisdiction with the highest rejection rate of ballots with overrankings was Alaska, with a rejection

rate of just 2.8%. It is important to remember that although we treat overranking as evidence of voter

confusion over the ranked choice system, the rules around tabulating votes would not typically cause a ballot

to be rejected due to overranking. Instead, this 1.5% rejection rate happens because voters who overrank are

also more likely to also skip or overvote on the same office, thus yielding a voided ballot. Appendix Section

8.6 shows that voters who overranked on an office were more than 2.5 times more likely to also overvote,

compared to those who did not overrank.

The final category of improper marks reported in Figure 10 is skips. Voters who skipped a ranking had their

ballots rejected about 5.8% of the time. Maine’s rejection rates are noticeably higher here because of all cast

vote records in Maine come from elections which required more than one round to counting to be resolved.

Interior and many front skips cannot result in a rejected ballot in round 1, and because we do not observe

the CVR data from races resolved in one round of counting, our data in Maine exclude the races where we

would expect the rejection rate to skipped ballots to be lowest. While Maine is an outlier, omitting its data

does not meaningfully change in the overall average rejection rate–it decreases from 5.8% to 4.6%.

What these results highlight is that although the improper marking rates on ranked ballots seem to be high,

about 90% of the time improperly marked ballots are included in the final vote tabulation. However, the

question still remains whether the ranked choice rejection rates are comparable to the rejection rates in

non-ranked choice offices. What we now show is that ranked choice votes are rejected at much higher rates

than the votes for non-ranked choice offices.

19 There are two reasons that these rejection rates would not automatically be 100%. First, not all types of improper marks

will result in a ballot rejection. For example, New York City and San Francisco ignore skips in the vote tabulation process.
Second, some improper marks will occur in later rankings than are necessary to determine a winner. If a voter overvotes in
ranking number 3, but their choice for ranking number 1 is the ultimate winner, their vote will still be counted.
20 For comparison, Appendix Section 8.5 includes a version of Figure 10 for voters who did not make each type of improper

mark. All the rejection rates in that figure are zero or extremely close to zero. They are not all exactly zero because it is possible
to not skip on a ballot, but to have it rejected due to overvoting.

19

Electronic copy available at: https://ssrn.com/abstract=4670677

 Figure 11: Rejection rates on ranked choice and non-ranked choice races

Figure 11 shows the ballot rejection rates in six election events in our data.21 The circles and red lines

(and vertical 95% confidence intervals) display the percentage of ballots that were rejected for counting in

non-ranked choice races. The triangles and blue lines show the percentage (and CI) of ballots that were

uncounted during the first round of tabulation in all ranked choice races. And the squares and yellow lines

correspond to the final-round rejection rates, but only in the subset of ranked choice offices that required

multiple rounds to determine a winner. The percentages in this figure are considerably smaller than those in

Figure 10 because these include all ballots, whereas those in the previous figure included only votes with

improper marks.

What this figure shows is that in each election event, the average proportion of vote attempts that are rejected

in ranked choice offices is statistically significantly higher than the rejection rate in non-ranked choice offices.

Averaging across the six election events, the average rejection rate in a non-ranked choice race is 0.04%. The

rates in ranked choice races are roughly an order of magnitude higher. In the first round of ranked choice

tabulation the rate is 0.35%, and in the final round of races requiring multiple rounds it is 0.53%. These

differences are noteworthy because these offices appear on the same exact ballots, so the population of voters
21 Maine is excluded because we do not have cast vote records for non-ranked choice races.

20

Electronic copy available at: https://ssrn.com/abstract=4670677

is constant within each election event.22

While this average ballot rejection rate may appear small, it is important to consider that RCV is most

advantageous when there are several candidates with similar levels of support. These are the instances that

provide the strong case for RCV to allow voters to be more detailed in their expression of electoral preference.

But it is these exact circumstances in which high rates of ballot rejection are the most likely to change the

outcome of the election by changing which candidate is eliminated in any round of counting, not just in the

final round. The November 2022 State Senate race in Alaska’s District E provides an example. In the first

round of voting, the three candidates’ vote percentages were 33.84%, 33.12%, and 33.04%. Roselynn Cacy,

the only Democrat in the race, was eliminated in the first round by a margin of 14 votes. In the first round

tally, there were 4 votes that were uncounted due to overvoting and another 9 that were rejected because

of skipping. While this is not enough rejected ballots to change the outcome of the race, it is not hard to

imagine a scenario where they could.

The comparison of rejection rates to non-ranked choice races also provides important context. Take, as an

example, the November 2022 general election in San Francisco. At the top-of-the-ticket, non-ranked choice

race for Governor, there were 59 San Francisco voters (out of about 300k) who attempted to vote (i.e. didn’t

undervote), but had their ballot rejected, and there were 99 voters who had their ballot rejected for single-mark

US Senate race. Compare this to the ranked choice race for San Francisco District Attorney, which had an

unusual amount of media attention due to a successful recall election earlier in the year (Seitz-Wald 2022). In

that race, there were 1,293 voters whose vote attempts were rejected in the first round count, which increased

to 1,569 by the final count as more improper marks were counted in subsequent rounds of tabulation.

Figure 11 also shows interesting heterogenity in rejection rates between Alaska, New York City, and San

Francisco. Despite its voters having used ranked choice voting since 2007, San Francisco’s rejection rate is

significantly higher than Alaska’s, where voters used RCV for the first time in 2022. The ranked choice offices

on San Francisco’s ballot in 2020 and 2022 were also further down-ballot than the ranked choice races on the

Alaska and New York City ballots. Ballot roll-off by less-informed voters may actually be driving down San

Francisco’s RCV rejection rates from where it would be if they used it in top-of-ticket races.

We conclude this section with a deeper dive into San Francisco’s ballot rejection rates. In this analysis, we

depart from using cast vote records and instead analyze vote tabulation reports in all SF elections since 2007

that included at least one ranked choice race.23 Figure 12 shows the difference in rejection rates between
22 Maloy and Ward (2021) use an experiment where subjects are randomly assigned to cast a fake vote using either a ranked

choice or non-ranked choice ballot. They find no difference in the ballot rejection rates between the two groups. The stark
differences in rejection rates in Figure 11 casts doubt on the external validity of the null findings of this experiment.
23 Such an extensive analysis is only possible because the city’s vote certification documents either explicitly include rejected

ballot counts or include enough information to calculate them. Even in non-ranked choice races in other cities and states, it is

21

Electronic copy available at: https://ssrn.com/abstract=4670677

 Figure 12: Ballot rejection rates in San Francisco 2007-2022

22

Electronic copy available at: https://ssrn.com/abstract=4670677

ranked choice and non-ranked choice offices, grouping the non-ranked choice races by office type.

The average first and final round rejection rates for ranked choice races are significantly (p < 0.01) higher

than those for all other types of races. Rejection rates for first-round votes are between 2.5 and 8 times as

high as rejection rates for different types of non-ranked choice offices. In the races that required multiple

rounds of tabulation to resolve, final round rejection rates are about 5 to 16 times as high the non-ranked

choice offices.

On San Francisco ballots, the local elections that use ranked choice voting appear after federal, statewide,

and state legislative races, before any ballot propositions, and generally in the same part of the ballot as

‘Other district offices’ (such as BART Director or municipal judges). The ranked choice offices have rejections

rates that are higher than the offices before and after it on the ballot. This suggests that these differences

are not attributable to a ballot order effect. Instead, we believe the evidence in this graph, as well as our

findings in previous sections, suggests that it is the use of ranked choice voting itself which is resulting in

higher ballot marking mistakes by voters and thus higher ballot rejection rates.

7 Discussion

Voters express their electoral preferences by casting ballots in an election. As American states and cities adopt

ranked choice voting, voters are given the opportunity to express those preferences in a more complex way.

But trade-off to that benefit is that ranked ballots require more complexity than a traditional, single-mark

ballot. We show in this article that this complexity results in roughly one-in-twenty (4.8%) attempted votes

for an office including an improper marking. One type of improper mark—overvotes—is about 14 times more

likely to occur on a ranked ballot than on a single-mark ballot.

We also show that giving voters the opportunity to rank more candidates–perhaps the chief benefit of ranked

choice voting–is correlated with higher’ rates of overvoting and skipping. And the data suggest that these

errors are most common among voters who are less engaged with politics and also less likely to vote in

down-ticket races. This implies that ranked choice voting is particularly ill-suited for electoral environments

where little information is available about the candidates.

The normative and policy implications of these high error rates would be worth considering, even if the errors

do not translate into higher rates of rejected ballots. We do find that about 90% of ballots that contain an

error are ultimately included in the vote count. But that is overshadowed by an average ballot rejection rates

on ranked choice races (0.35% in the first round of tabulation) that is nearly ten times that of non-ranked
rare to have enough information to calculate these values.

23

Electronic copy available at: https://ssrn.com/abstract=4670677

choice races (0.04%). These percentages may seem small, but in a large jurisdiction like New York City, this

translates into thousands of additional rejected ballots. And when RCV is used in small jurisdictions or in

races with low turnout, margins-of-victory are tight and even modest rejection rates could affect outcomes.

These effects could be even more pernicious if other factors such as ballot exhaustion also affect how voter

preferences are translated into outcomes.

The findings in this article open several new avenues for research about ranked choice voting and ballot

marking errors more generally. Our framework for classifying different types of ballot improper marks provides

future researchers with a consistent language for talking about errors on single-mark and ranked ballots.

There are also important questions about the interaction between vote mode (i.e. in-person versus mail

voting), vote technology (hand-marked versus electronic ballots), and improper marking rates in ranked

systems.

Perhaps the biggest avenue for future research is understanding what types of voters tend to make more

errors when presented with a ranked ballot. If, for example, there are differences in improper marking rates

between different racial groups or people with different levels of education, there could be important normative

implications for voter participation and representation in a ranked choice system. If those differences correlate

with partisan differences in ballot rejection rates, then improper marks on ranked choice races could have an

effect of election outcomes.

The policy implications of our findings also grow in importance as more jurisdictions consider adopting ranked

choice voting. This article provides important evidence about improper marking rates and vote rejection rates

that, to this point, had been mostly missing from debates about the adoption of RCV. And for jurisdictions

that have already or plan to implement RCV, this work can help inform their choices about administrative

rules for counting ranked choice ballots.

24

Electronic copy available at: https://ssrn.com/abstract=4670677

8 Appendix

8.1 Appendix: Overvote rates by ranked choice races and number of candidates

Table 1: Overvote rates by ranked choice and candidate count

Dependent variable: Overvote rate

(1) (2) (3) (4)
Ranked choice race? 0.006∗∗∗
0.005∗∗∗
0.003 ∗∗∗
0.003∗∗∗
(0.0004) (0.0004) (0.001) (0.001)
Number of candidates 0.0005∗∗∗ 0.0003∗∗∗
(0.0001) (0.0001)
(Intercept) 0.0004 −0.001∗∗ 0.0001 −0.001
(0.0003) (0.0004) (0.0004) (0.001)
Geography+date fixed effects ✓ ✓
Observations 310 310 310 310
R2 0.340 0.404 0.568 0.582
Note: ∗
p<0.1; ∗∗
p<0.05; ∗∗∗
p<0.01

25

Electronic copy available at: https://ssrn.com/abstract=4670677

8.2 Appendix: Error rates by candidate counts

Table 2: Errors rates by candidate count (A)

Dependent variable:
Any error Any error Overrank Overrank
(1) (2) (3) (4)
Possible rankings 0.005 −0.003 −0.002 −0.007∗∗∗
(0.003) (0.003) (0.002) (0.002)
(Intercept) 0.028∗∗ 0.030∗∗ 0.033∗∗∗ 0.033∗∗∗
(0.014) (0.012) (0.009) (0.008)
Geo fixed effects ✓ ✓
Observations 165 165 165 165
R2 0.013 0.289 0.006 0.285
Note: ∗
p<0.1; ∗∗
p<0.05; ∗∗∗
p<0.01

Table 3: Errors rates by candidate count (B)

Dependent variable:
Overvote Overvote Skip Skip
(1) (2) (3) (4)
Possible rankings 0.003 ∗∗∗
0.002 ∗∗∗
0.005 ∗∗∗
0.004∗∗
(0.0004) (0.0004) (0.001) (0.001)
(Intercept) −0.006∗∗∗ −0.005∗∗∗ −0.001 −0.001
(0.002) (0.002) (0.006) (0.005)
Geo fixed effects ✓ ✓
Observations 165 165 165 165
R2 0.234 0.439 0.086 0.185
Note: ∗
p<0.1; ∗∗
p<0.05; ∗∗∗
p<0.01

26

Electronic copy available at: https://ssrn.com/abstract=4670677

8.3 Appendix: Roll-off error rate comparisons

Table 4: Error rates of voters who voted in neither state legislative race versus those who voted in both

Mis-mark rate (n voters) of those who...

Mis-mark Didn’t vote for Voted for both Z-score of
type Office State House or Senate State House and Senate difference
Any error US Senate 3.36% (9,490) 2.48% (198,276) 5.38∗∗∗
Any error US House 4.43% (9,850) 3.32% (198,251) 5.94∗∗∗
Any error Governor 4.96% (9,753) 4.26% (197,570) 3.36∗∗∗
Overrank US Senate 1.12% (9,490) 0.97% (198,276) 1.42
Overrank US House 1.39% (9,850) 1.07% (198,251) 3.01∗∗
Overrank Governor 1.64% (9,753) 1.31% (197,570) 2.77∗∗
Overvote US Senate 0.53% (9,490) 0.25% (198,276) 5.10∗∗∗
Overvote US House 0.73% (9,850) 0.33% (198,251) 6.58∗∗∗
Overvote Governor 0.73% (9,753) 0.42% (197,570) 4.59∗∗∗
Skip US Senate 2.02% (9,490) 1.43% (198,276) 4.70∗∗∗
Skip US House 2.79% (9,850) 2.15% (198,251) 4.24∗∗∗
Skip Governor 3.08% (9,753) 2.86% (197,570) 1.27
Note: ∗
p<0.05; ∗∗
p<0.01; ∗∗∗
p<0.001

Table 5: Error rates of voters who voted in neither state legislative race versus those who voted in one

Mis-mark rate (n voters) of those who...

Mis-mark Didn’t vote for Voted for either Z-score of
type Office State House or Senate State House or Senate difference
Any error US Senate 3.36% (9,490) 2.62% (12,229) 3.18∗∗
Any error US House 4.43% (9,850) 3.77% (12,361) 2.46∗
Any error Governor 4.96% (9,753) 4.23% (12,240) 2.58∗∗
Overrank US Senate 1.12% (9,490) 0.78% (12,229) 2.60∗∗
Overrank US House 1.39% (9,850) 0.95% (12,361) 3.09∗∗
Overrank Governor 1.64% (9,753) 1.19% (12,240) 2.82∗∗
Overvote US Senate 0.53% (9,490) 0.20% (12,229) 4.02∗∗∗
Overvote US House 0.73% (9,850) 0.35% (12,361) 3.95∗∗∗
Overvote Governor 0.73% (9,753) 0.50% (12,240) 2.19∗
Skip US Senate 2.02% (9,490) 1.77% (12,229) 1.38
Skip US House 2.79% (9,850) 2.67% (12,361) 0.56
Skip Governor 3.08% (9,753) 2.84% (12,240) 1.01
Note: ∗
p<0.05; ∗∗
p<0.01; ∗∗∗
p<0.001

27

Electronic copy available at: https://ssrn.com/abstract=4670677

8.4 Appendix: Ballot-level improper marking rates

The previous analyses treated all ranked choice offices on a ballot as independent data points. But our cast

vote record data allow us to combine together offices that appeared on the same ballot to study error rates

at the voter level. In this section, we assess how common it is for a voter to make an improper marking

anywhere on any ranked choice race on their ballot.

Figure 13: Ballot-level error rates by number of offices marked

Figure 13 displays ballot-level error rates: the proportion of voters who made at least one mistake on at

least one ranked choice race. The x-axis of the graph shows the number of ranked choice offices that a voter

attempted to cast a vote for (whether or not that vote was ultimately counted). This accounts for two things.

First, ballots include different numbers of ranked choice races. Second, even among voters presented with

identical ballots, some voters may have cast a vote in every ranked choice race, while others may skip some of

them. Each line shows how (in a particular state/city) the error rates change with the number of ranked

choice offices that a voter votes for.24

The figure shows that the percentage of voters who made a any type of mistake in any ranked choice race

ranged from 5.3% (roughly 1-in-18 voters) to 12.6% (roughly 1-in-8), depending on the jurisdiction and the
24We omit Maine because their cast vote records do not allow us to tie together a voter’s choices for separate offices.

28

Electronic copy available at: https://ssrn.com/abstract=4670677

number of ranked choice offices the voter voted for. Over-ranking tended to be the most common mistake,

with between 2.0% and 5.7% of ballots featuring it. Over-voting occurred the least often–between 0.4% and

3.0% of ballots.

29

Electronic copy available at: https://ssrn.com/abstract=4670677

8.5 Appendix: Ballot rejections of ballots without errors

Figure 14: Rejection rates of ballots without errors

30

Electronic copy available at: https://ssrn.com/abstract=4670677

8.6 Appendix: Correlations between types of improper markings

Figure 15: Co-occurrance of different types of ballot-marking errors

Figure 15 reports how common it is for each type of improper marking to co-occur with the other types of

improper marks. Each panel of the three panels represents one of the three types of improper markings. The

blue bars in the panel represent the voters who did make the panel’s error type, and the red bars represent

voters who did not make that type of error. So, the first blue bar on the far left shows that 1.6% of voters

who overranked also overvoted. The red bar next to it shows that 0.6% of voters who did not overrank, did

overvote.

The general pattern here is that making one type of improper mark increases the probability that a voter

also improperly marked their ballot in a second way. Voters who overvote seem to be the most likely to

commit other types of errors. Looking at the middle panel, we see that 6.9% of people who overvoted also

overranked, compared to just 2.8% of people who did not overvote. And 24.8% of people who overvoted also

skipped a ranking, compared to just 2.0% of those who did not overvote.

31

Electronic copy available at: https://ssrn.com/abstract=4670677

Works cited
Agadjanian, Alexander, and Jonathan Robinson. 2019. “Ground Truth Validation of Survey Estimates of

Split-Ticket Voting with Cast Vote Records Data.” Working Paper. https://papers.ssrn.com/sol3/papers.

cfm?abstract_id=3443722.

Alvarez, Michael, Thad Hall, and Ines Levin. 2018. “Low-Information Voting: Evidence from Instant-Runoff

Elections.” American Politics Research 46 (6): 1012–38.

Alvarez, R Michael, Dustin Beckett, and Charles Stewart III. 2013. “Voting Technology, Vote-by-Mail, and

Residual Votes in California, 1990–2010.” Political Research Quarterly 66 (3): 658–70.

Atkeson, Lonna Rae, Eli McKown-Dawson, Jack Santucci, and Kyle L. Saunders. 2023. “The Impact of

Voter Confusion in Ranked Choice Voting.” Working Paper.

Bartholdi, John J, and James B Orlin. 1991. “Single Transferable Vote Resists Strategic Voting.” Social

Choice and Welfare 8 (4): 341–54.

Bean, Clive. 1997. “Australia’s Experience with the Alternative Vote.” Representation 34 (2): 103–10.

Blais, Andre. 1988. “The Classification of Electoral Systems.” European Journal of Political Research 16 (1).

Burnett, Craig M., and Vladimir Kogan. 2015. “Ballot (and Voter) ‘Exhaustion’ Under Instant Runoff

Voting: An Examination of Four Ranked-Choice Elections.” Electoral Studies 37: 41–49.

Clark, Jesse. 2020. “Rank Deficiency? Analyzing the Costs and Benefits of Single-Winner Ranked Choice

Voting.” Working Paper. https://ssrn.com/abstract=3703197.

Coll, Joseph. 2021. “Demographic Disparities Using Ranked-Choice Voting? Ranking Difficulty, Under-Voting,

and the 2020 Democratic Primary.” Politics and Governance 9 (2): 293–305.

Cottrell, David, Felix E Herron, Michael C Herron, and Daniel A Smith. 2022. “Auditing the 2020 General

Election in Georgia: Residual Vote Rates and a Confusing Ballot Format.” Election Law Journal: Rules,

Politics, and Policy 21 (1): 1–18.

Crowder-Meyer, Melody, Shana Kushner Gadarian, and Jessica Trounstine. 2023. “Ranking Candidates in

Local Elections: Neither Panacea nor Catastrophe for Candidates of Color.” Journal of Experimental

Political Science, 1–18.

Curtice, John, and Michael Marsh. 2014. “Confused or Competent? How Voters Use the STV Ballot Paper.”

Electoral Studies 34: 146–58.

Donovan, Todd, and Caroline Tolbert. 2023. “Civility in Ranked-Choice Voting Elections: Does Evidence Fit

the Normative Narrative?” Representation 0 (0): 1–18.

Donovan, Todd, Caroline Tolbert, and Kellen Gracey. 2019. “Self-Reported Understanding of Ranked-Choice

Voting.” Social Science Quarterly 100 (5): 1768–76.

32

Electronic copy available at: https://ssrn.com/abstract=4670677

Donovan, Todd, Caroline Tolbert, and Samuel Harper. 2022. “Demographic Differences in Understanding

and Utilization of Ranked Choice Voting.” Social Science Quarterly 103 (7): 1539–50.

Graham-Squire, Adam, and David McCune. 2024. “Ranked Choice Wackiness in Alaska.” Math Horizons 31

(1): 24–27.

Graham-Squire, Adam, and N. Zayatz. 2021. “Lack of Monotonicity Anomalies in Empirical Data of

Instant-Runoff Elections.” Representation 57 (4): 565–73.

Herron, Michael C., Michael D. Martinez, Marc Meredith, Michael Morse, and Daniel A. Smith. 2023. “A

Typology of Electoral Harms and the 2018 United States Senate Race in Florida.” Working Paper.

Herron, Michael C., and Jasjeet S. Sekhon. 2003. “Overvoting and Representation: An Examination of

Overvoted Presidential Ballots in Broward and Miami-Dade Counties.” Electoral Studies 22 (1): 21–47.

John, Sarah, Haley Smith, and Elizabeth Zack. 2018. “The Alternative Vote: Do Changes in Single-Member

Voting Systems Affect Descriptive Representation of Women and Minorities?” Electoral Studies 54.

Juelich, Courtney, and Joseph Coll. 2021. “Ranked Choice Voting and Youth Voter Turnout: The Roles of

Campaign Civility and Candidate Contact.” Politics and Governance 9 (2): 319–31.

Katz, Richard S. 1997. Democracy and Elections. Oxford University Press.

Kilgour, Marc, Jean-Charles Gregoire, and Angele Foley. 2020. “The Prevalence and Consequences of Ballot

Truncation in Ranked-Choice Elections.” Public Choice 184 (1-2): 197–218.

Kimball, David C, and Martha Kropf. 2005. “Ballot Design and Unrecorded Votes on Paper-Based Ballots.”

Public Opinion Quarterly 69 (4): 508–29.

Kropf, Martha. 2021. “Using Campaign Communications to Analyze Civility in Ranked Choice Voting

Elections.” Politics and Governance 9 (2): 280–92.

Kuriwaki, Shiro. 2023. “Ticket Splitting in a Nationalized Era.” SocArXiv. https://doi.org/10.31235/osf.io/

bvgz3.

Lamb, Matt, and Steven Perry. 2020. “Knowing What You Don’t Know: The Role of Information and

Sophistication in Ballot Completion.” Social Science Quarterly 101 (3): 1132–49.

Maloy, Jason Stuart. 2019. Smarter Ballots: Electoral Realism and Reform. Springer.

Maloy, Jason Stuart, and Matthew Ward. 2021. “The Impact of Input Rules and Ballot Options on Voting

Error: An Experimental Analysis.” Politics and Governance 9 (2): 306–18.

McCune, David, and Lori McCune. 2022. “Does the Choice of Preferential Voting Method Matter? An

Empirical Study Using Ranked Choice Elections in the United States.” Representation 0 (0): 1–16.

McCune, David, and Jennifer Wilson. 2023. “Ranked-Choice Voting and the Spoiler Effect.” Public Choice.

McDaniel, Jason. 2016. “Writing the Rules to Rank the Candidates: Examining the Impact of Instant-Runoff

Voting on Racial Group Turnout in San Francisco Mayoral Elections.” Journal of Urban Affairs 38 (3).

33

Electronic copy available at: https://ssrn.com/abstract=4670677

———. 2018. “Does More Choice Lead to Reduced Racially Polarized Voting? Assessing the Impact of

Ranked-Choice Voting in Mayoral Elections.” California Journal of Politics and Policy 10 (2).

Miller, Nicholas R. 2017. “Closeness Matters: Monotonicity Failure in IRV Elections with Three Candidates.”

Public Choice 173 (1-2): 91–108.

Montanaro, Domenico. 2021. “Ranked-Choice Voting Gets a Prime-Time Shot Under New York City’s

Bright Lights.” NPR. https://www.npr.org/2021/06/22/1008807504/ranked-choice-voting-new-york-city-

mayors-race.

Mott, Rodney L. 1926. “Invalid Ballots Under the Hare System of Proportional Representation.” American

Political Science Review 20 (4): 874–82.

Neely, Francis, and Corey Cook. 2008. “Whose Votes Count?: Undervotes, Overvotes, and Ranking in San

Francisco’s Instant-Runoff Elections.” American Politics Research 36 (4): 530–54.

Neely, Francis, and Jason McDaniel. 2015. “Overvoting and the Equality of Voice Under Instant-Runoff

Voting in San Francisco.” California Journal of Politics and Policy 7 (4).

New York City Charter. 2023. “Section 1057-g. Ranked Choice Voting for Certain Primary Elections and

Elections for Which Nominations Were Made by Independent Nominating Petitions.” https://codelibrary.

amlegal.com/codes/newyorkcity/latest/NYCcharter/0-0-0-3079.

Rae, Douglas W. 1967. The Political Consequences of Electoral Laws. Yale University Press.

San Francisco, City of. 2023. “Ranked-Choice Voting.” https://sf.gov/ranked-choice-voting.

Santucci, Jack. 2018. “Maine Ranked-Choice Voting as a Case of Electoral-System Change.” Representation

54 (3): 297–311.

———. 2021. “Variants of Ranked-Choice Voting from a Strategic Perspective.” Politics and Governance 9

(2): 344–53.

“Sarah Palin’s Defeat in Alaska Proves Ranked-Choice Voting Works.” 2022. The Washington Post. https:

//www.washingtonpost.com/opinions/2022/09/05/sarah-palin-alaska-ranked-choice-works/.

Seitz-Wald, Alex. 2022. “California Voters Oust Reformist DA in San Francisco as House Battlegrounds

Take Shape.” NBC News. https://www.nbcnews.com/politics/2022-election/first-polls-close-seven-state-

primary-elections-rcna32276.

Shabad, Rebecca. 2022. “After Sarah Palin’s Election Loss, Sen. Tom Cotton Calls Ranked-Choice Voting ’a

Scam’.” NBC News. https://www.nbcnews.com/politics/2022-election/sarah-palins-election-loss-sen-tom-

cotton-calls-ranked-choice-voting-s-rcna45834.

Sinclair, Betsy, and R Michael Alvarez. 2004. “Who Overvotes, Who Undervotes, Using Punchcards?

Evidence from Los Angeles County.” Political Research Quarterly 57 (1): 15–25.

Stewart, Charles, R. Michael Alvarez, Stephen S. Pettigrew, and Cameron Wimpy. 2020. “Abstention,

34

Electronic copy available at: https://ssrn.com/abstract=4670677

 Protest, and Residual Votes in the 2016 Election.” Social Science Quarterly 101 (2): 925–39.

Terrell, Cynthia Richie, Courtney Lamendola, and Maura Reilly. 2021. “Election Reform and Women’s

Representation: Ranked Choice Voting in the u.s.” Politics and Governance 9 (2): 332–43.

Thiessen, Mark. 2022. “What’s Next for Ranked-Choice Voting After Alaska?” NBC News. https:

//www.nbcnews.com/meet-the-press/meetthepressblog/s-ranked-choice-voting-alaska-rcna45929.

Tomlinson, Kiran, Johan Ugander, and Jon Kleinberg. 2022. “Ballot Length in Instant Runoff Voting.”

Working Paper. https://arxiv.org/abs/2207.08958.

Tucker, Jill, Jordan Parker, J. D. Morris, and Nami Sumida. 2022. “Alameda County Admits Tallying

Error in Ranked-Choice Voting, Flips One Result and Raises Big Questions.” San Francisco Chronicle.

https://www.sfchronicle.com/bayarea/article/Alameda-County-admits-tallying-error-in-17682520.php.

Wand, Jonathan N., Kenneth W. Shotts, Jasjeet S. Sekhon, Walter R. Mebane, Michael C. Herron, and

Henry E. Brady. 2001. “The Butterfly Did It: The Aberrant Vote for Buchanan in Palm Beach County,

Florida.” The American Political Science Review 95 (4): 793–810.

Wattenberg, Martin P., Ian McAllister, and Anthony Salvanto. 2000. “How Voting Is Like Taking an Sat

Test: An Analysis of American Voter Rolloff.” American Politics Quarterly 28 (2): 234–50.

“Where Is Ranked Choice Voting Used?” 2022. FairVote. https://fairvote.org/our-reforms/ranked-choice-

voting-information/#where-is-ranked-choice-voting-used.

Williams, Jack, Samuel Baltz, and Charles III Stewart. 2023. “Administering Instant Runoff Voting Elections:

What Could Go Wrong?” Working Paper.

35

Electronic copy available at: https://ssrn.com/abstract=4670677