0% found this document useful (0 votes)
27 views15 pages

Local Walkability Index Assessing Built Environmen

Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
27 views15 pages

Local Walkability Index Assessing Built Environmen

Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 15

Bulletin of Geography. Socio-economic Series, No.

46 (2019): 7–21
http://doi.org/10.2478/bog-2019-0031

BULLETIN OF GEOGRAPHY. SOCIO–ECONOMIC SERIES

journal homepages:
https://content.sciendo.com/view/journals/bog/bog-overview.xml
ISSN 1732–4254 quarterly http://apcz.umk.pl/czasopisma/index.php/BGSS/index

Local walkability index: assessing built environment influence on


walking

Marzieh Reisi1, CDFMR, Mozhgan Ahmadi Nadoushan2, CDFMR, Lu Aye3, CDFMR

University of Kurdistan, Department of Natural Resources, Iran, 1email: [email protected], http://orcid.org/0000-0003-1700-4975


(corresponding author), 2email: [email protected], http://orcid.org/0000-0002-9269-1317, 3email: [email protected],
http://orcid.org/0000-0002-5495-1683

How to cite:
Reisi, M. Nadoushan, M.A. and Aye L. (2019). Local walkability index: assessing built environment influence on walking. Bulle-
tin of Geography. Socio-economic Series, 46(46): 7-21. DOI: http://doi.org/10.2478/bog-2019-0031

Abstract. Walking is a more sustainable transport mode, and governments around Article details:
the world are trying to deliver highly walkable areas to their people. Due to its Received: 17 February 2019
importance, walkability has been a research topic in recent years. Vast empirical Revised: 15 July 2019
Accepted: 31 July 2019
studies have reported evidence related to the influence of built environment on
walking as a major physical activity. Considering the recent literature, this study
developed a framework to quantify walkability by applying a set of indicators re-
lated to built environment. The indicators were normalised, weighted and integrat-
ed into an overall walkability index. The research was conducted on Chaharbagh
Street, which is a major and ancient street in the Isfahan metropolitan area, Iran.
Key words:
The proposed framework would be helpful in investigations of whether a specif- Environment,
ic area is an appropriate option for a car-free plan based on its built environment Sustainability,
features. The outcome of the study could be applied to understand issues related Town & city planning,
to pedestrian infrastructure and to propose corrective actions. Transport planning

Contents:
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2. Material and methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1. Study area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2. Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4. Highlights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5. Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

© 2019 (Reisi, M. Nadoushan, M.A. and Aye L.) This is an open access article licensed under the Creative Commons Attribution-NonCommer-
cial-NoDerivs License (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC
8 Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21

1. Introduction Pivo, 1994; Bauman et al., 1996; Wright et al.,


1996; Sallis et al., 1998; Owen et al., 2004). In var-
ious studies investigating walking behaviour, acces-
Walking, due to its minimal costs, is a more sus- sibility, connectivity, convenience, safety, aesthetic
tainable transport mode (Gallin, 2001). Despite the features and attractiveness were the main suggest-
importance of walking, the current level of walking ed factors affecting walking activities (Hawthorne,
in communities is not adequate to provide health 1989; Bauman et al., 1996; Wright et al., 1996; Gala-
benefits, and people need more walking activities nis, 2011; Humpel et al., 2002; Pikora et al., 2003;
to achieve public health objectives (Giles-Corti, Do- Giles-Corti et al., 2003; Galanis, Eliou, 2011; Tal,
novan, 2003). Walking is one of the most popular Handy, 2012; Motamed, Bitaraf, 2016). Ball et al.
forms of physical activity among adults (Bahrainy (2001) and Powell et al. (2003) found a strong rela-
et al., 2015). Urban planners are interested in walk- tionship between convenience and level of walking.
ing as a means of reducing vehicle miles travelled Saelens et al. (2003) believed that choice of trans-
and greenhouse gas emissions (Tribby et al., 2016). port for leisure, exercise or recreation depends on
Insufficient physical activity is a major risk, causing land use, distance, connectivity and pedestrian in-
excess weight and obesity, diabetes and heart dis- frastructure. In the Asian context, land use diversi-
eases (US Department of Health and Human Ser- ty, footpath availability and quality, safety, facilities
vices, 1996). for disabled, and pedestrian amenities all influence
In recent years, urban design and public health walking trips (Efroymson, 2012). Galanis et al.
have been closely related to walkable localities (Ew- (2011) measured 30 indicators, including pedestrian
ing, Handy, 2009), as the ease of walking in a neigh- infrastructure, road segment, corner and crosswalk
bourhood affects the liveability of urban areas and indicators in six selected roads in Greece through
the physical activity of citizens (Burden et al., 2002). street audits and claimed that the measured indica-
Walkability is an important concept in the context tors could be applied to evaluate walkability features
of transportation engineering, urban planning and of the pedestrian infrastructure across urban streets.
health disciplines (Bhadra et al., 2016). Improving Although many attempts have been made to
walkability and promoting walking in urban are- identify built environment indicators affecting
as leads to wealth, health and urban sustainabili- walking, only a limited number of studies aggre-
ty (Gallin, 2001; Zayed, 2016). Consequently, policy gate various indicators to create a single walkability
interventions have been undertaken in urban de- index (WI) (USDG, 2007; Walkscore, 2019; CAI-
sign worldwide to promote physical activities and Asia, 2011). Aggregating individual indicators into
walking (Rebecchi et al., 2019; Sallis et al., 1998). an index is a practical approach helping to meas-
There are numerous definitions of walkability in the ure various dimensions of built environment that
current literature. Walkability can be defined as the cannot be captured completely by individual indica-
suitability of the urban road environment for pe- tors alone (Zhou et al., 2007; Saisana, 2011). Frank
destrians. Walkability is also defined as supports et al. (2010) developed a walkability index for Met-
provided by the built environment to encourage ro Vancouver comprising residential, commercial
walking, including pedestrian comfort and safety, density, land use mix and street connectivity indi-
connecting people with various destinations within cators. However, a large number of effective indi-
a reasonable amount of time and effort, and offering cators specifically related to safety are disregarded
visual interest on journeys throughout the network in the study. Krambeck et al. (2006) developed a
(Southworth, 2005; Galanis, Eliou, 2011). global WI in specific parts of the world consider-
According to the public health literature, it is ing 45 indicators related to safety, convenience, at-
argued that physical environment could influence tractiveness and policy support. The indicators were
physical activity behaviours (Leslie et al., 2007). integrated into a single index, but there are some
Thus, considerable efforts have been made to find a limitations related to integration. First, the indica-
correlation between the physical environment of an tors were not measured directly (i.e. to quantify the
area and the physical activities of people, with par- indicators, pedestrians were asked to scale each in-
ticular focus on walking (Hawthorne, 1989; Frank, dicator using a scale from 1 to 5), which involves

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21 9

subjective quantification of the indicators. Secondly, 2. Material and methods


despite the differing importances of the indicators,
they received the same weighting during aggrega-
tion, which skews the overall index. 2.1. Study area
Gallin (2001) also defined 11 indicators affect-
ing level of services (LOS) for pedestrians, including Isfahan city, with an area of 267.6 km2, is located at
physical characteristics, location and user indica- 32º39ʹ8.86ʺ Northern latitude and 51º40ʹ28.63ʺ East-
tors. To quantify the indicators, a score of 0 to 4 ern longitude. With a population of 1,602,110, it is
was given to the selected path based on its char- the third most populous city in Iran. Isfahan cit-
acteristics. The score for each indicator was then izens experience a desert climate with an average
multiplied by the relative weight assigned to each annual temperature of 16.7ºC and 125 mm annu-
indicator based on stakeholders’ opinions, on a scale al rainfall. According to census data, inter-city traf-
of 1 to 5. The individual weighted score for each fic accidents caused 399 deaths and 15,343 injuries
indicator was summed to obtain a total weight- in 2017, putting Isfahan in third place in Iran in
ed score for the path segment. As in the previous terms of traffic accident frequency (National Cen-
study, indicator quantification was subjective in this sus, 2017).
research and cannot provide the true condition of Chaharbagh Abbasi Street (Chaharbagh, for
the study area. Moreover, the sensitivity of priori- short) is one of the main historical, cultural, com-
ties was not analysed. mercial and entertainment centres in the Isfahan
Despite promising results, there are some chal- metropolitan area, Iran. This historical street hosts
lenges associated with previous studies. Many of a large number of offices, business centres and ho-
the studies focused on self-reported perceptions of tels and is regarded as Isfahan’s central district. The
built environment factors rather than direct meas- neighbourhood suburbs of Chaharbagh are home
urements (Humpel et al., 2002; Pikora et al., 2003). to 22,900 persons making 22,130 daily trips in the
Moreover, the relative importance of built environ- surrounding areas (Saghayi & Sadeghi, 2013). More-
ment factors on walking behaviour has not been over, connecting people to the neighbourhood hos-
determined (Pikora et al., 2003). For example, San pital and medical centres, historical buildings and
Francisco Department of Public Health (2009) de- Zayanderoud bayside, a large number of cars are at-
veloped the Pedestrian Environmental Quality In- tracted to the street. Table 1 summarises some ba-
dex (PEQI) to assess pedestrian safety considering sic characteristics of Chaharbagh Street.
a range of built environmental features including Chaharbagh is a part of Isfahan’s identity, and
number of lanes, sidewalk width, sidewalk sur- attracts pedestrians for walking, due to its par-
face, trees, and public seating. Although using an- ticular spatial features. “Car-free Tuesday” is a lo-
alyst-collected data provides a real understanding cal arrangement currently operating in Chaharbagh
of street features, the indicators are not differentiat- Street to motivate walking habits among Isfahan cit-
ed based on importance. Therefore, there is a clear izens. It aims to reduce traffic, fuel consumption,
need to refine measures applied for assessing the noise and air pollutants through specific urban de-
potential of a built environment for walking. sign and planning for promoting active transport.
Considering the limitation of developing WI us- Therefore, better urban design to improve walkabil-
ing actual measurement of built environment fea- ity along the street is urgently needed.
tures, and to help in achieving public health goals By quantifying built environment features and
related to physical activity, this research aims to de- identifying WI, this research investigation tries to
velop a WI for a street in Isfahan, Iran. To achieve evaluate the street’s suitability for the “car-free” plan.
this aim, the built environment features were meas- The results of the investigation would help the local
ured directly in the study area, weighted by a rec- government to improve the walking experience in
ognised weighting method, and aggregated into a Chaharbagh Street. Moreover, it also would be ben-
single index. eficial for successful expansion of the plan to other
parts of the metropolitan area, and would conse-

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


10 Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21

Table 1. Chaharbagh characteristics

Length 1.16 km
Number of car lanes 2
Number of sidewalks 3 (Two frontage sidewalks and one along the middle of street)
Min:1 m for frontages
Sidewalk width
Max: 1.5 m for middle sidewalk
Bicycle lanes 2 (along the middle of the street)

Fig.1. Study area

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21 11

quently reduce transport-related environmental im- 2016) and by considering the selection criteria, 13
pacts. indicators were selected in three categories, name-
To be able to normalise the indicators in the fol- ly: safety, quality and attractiveness (Table 2). Safe-
lowing steps, Chaharbagh Street was divided into ty indicators are considered in response to the need
five segments (Fig. 1): to provide a liveable and safe environment for peo-
Kazeroni, ple. Quality indicators are related to the physical
Sheikh Bahayi, and structural characteristics of the street. Attrac-
Amadegah, tiveness indicators influence people’s desire to ac-
Seyed Alikhan, cess the place on foot.
Enghelab.
The segments have the same length, and each 2.2.2. Indicator quantification
section provides specific services to citizens.
Figure 2 provides a representative view of Cha- The authors visited the street several times to meas-
harbagh’s atmosphere. ure the 13 selected indicators in the selected seg-
ments (i.e. counting the numbers or measuring the
selected indicators) (Table 2).
2.2. Methods
2.2.3. Indicator normalisation

The development of the WI in this investigation is As the selected indicators were measured in dif-
described in several steps, namely: ferent units, they cannot be aggregated. Therefore,
Selecting built environment indicators that affect it is necessary to convert the indicators to dimen-
walking activities, sionless numbers before aggregation. This process
Quantifying selected indicators, is called normalisation (Nardo et al., 2005). The se-
Normalising the indicators, lected indicators in this study contain both positive
Weighting the indicators, and negative values. Increasing values of positive
Indicator aggregation and index composition and negative indicators have positive and negative
effects on walkability, respectively. In other words,
2.2.1. Indicator selection by increasing positive indicators, WI increases as
well. Meanwhile, by increasing negative indicators,
The selection of indicators is the first step in de- WI would decrease (e.g. number of lights is a posi-
veloping indices such as WI. Indicators are quan- tive indicator improving walkability, while obstruc-
titative variables that are useful for demonstrating tions are negative indicators reducing walkability).
a complex phenomenon (EEA, 2005). It is chal- The normalisation equation differs for positive and
lenging to select a set of indicators that provides negative indicators. Equation 1 shows the normal-
a comprehensive overview of the considered topic isation process for positive and negative indicators
(Castillo, Pitfield, 2010). In this investigation, indi- (Krajnc, Glavic, 2005).
cators were selected based on various selection cri-
teria including relevancy, measurability, simplicity
and data availability. Each indicator must be close-
ly related to the definition of walkability, quantifia-
ble and understandable by users, and the required
data must be available easily and at a reasonable
cost (Li et al., 2009; Dur et al., 2010; Zito, Salvo,
2011; Haghshenas, Vaziri, 2012). Reviewing the lit-
erature listing walking-related indicators (Haw- where: IN, normalised indicator I; “+”, for positive
thorne, 1989; Jacobs, 1993; Atash, 1994; Bauman et indicator; “−”, for negative indicator; min, mini-
al., 1996; Wright et al., 1996; Gallin, 2001; Burden mum value of indicator considering 4 segments;
et al., 2002; Pikora et al., 2003; Motamed, Bitaraf, max, maximum value of indicator considering 4
segments.

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


12 Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21

Fig. 2. Chaharbagh’s atmosphere (http://www.qudsonline.ir, https://www.google.com/maps)

2.2.4. Indicator weighting indicator compared to the other. The importance is


quantified based on experts’ judgments on a scale of
Weight of indicator shows the relative importance 1 to 9, where 1 indicates equal importance of paired
and influence that each considered indicator has on indicators, while 9 indicates that one indicator is
walkability. As the weights given to various indi- nine times more important than the other (Saaty,
cators influence the outcome of the created indi- 1990; Albayrak, Erensal, 2004; Sharna et al., 2008;
ces, a transparent method is needed for weighting Gorener, 2012). Considering pairwise comparisons,
(Freudenberg, 2003; Juwana, 2012). Analytical Hi- the relative weight of each indicator is calculated.
erarchy Process (AHP) was applied for weighting Priorities are not assigned arbitrarily in AHP but
indicators in this study. AHP is a multi-criterion de- are derived based on experts’ judgments and pref-
cision-making method, helping decision-makers by erences. Therefore, the priorities have mathemati-
decomposing a complex problem into a hierarchical cal validity (Mu, Pereyra-Rojas, 2016). One of the
structure of aim and related criteria and sub-criteria. main aspects of pairwise comparisons which should
To find indicator weights, AHP performs pairwise be considered during AHP is consistency of judg-
comparisons among indicators in each level of the ments. Consistency ratio (CR), which demonstrates
hierarchy. Pairwise comparisons are made between the coherence of judgments, should be 0.1 at max-
pairs of indicators, showing the importance of one

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21 13

imum. In other words, CR of 0.10 or less would be al weighted scores for each indicator were summed
acceptable (Borajee, Yakchali, 2011; Aragonés-Bel- to obtain a sub-index for each category and then
trán et al., 2014). The weighting procedure using the weighted sub-indices for each category were
AHP is described in detail in Appendix 1. summed to get the final value of WI for the select-
ed street segments (Table 3). The developed indices
2.2.5. Indicator aggregation and index composition are applicable for evaluating the current situation
of built environment features in Chaharbagh and
The indicators were aggregated using the weighted their capability to promote walking. Segment 3 was
linear combination (WLC) method (Mahini, Gho- ranked top in safety and attractiveness. Segment 1
lamalifard, 2006) (Eq. 2). WLC is a form of index was ranked top in safety. Considering overall WI,
overlay technique, considering the normalised val- Segments 1 and 4 were ranked as the most and least
ue and relative weights of indicators in aggregation walkable segments, respectively.
(Al-shabeeb, 2015). The priorities assigned to each segment are
mainly influenced by the weights given to the eval-
uated indicators. Therefore, it would be beneficial
where: to perform a “what-if ” analysis to see how the fi-
WI = Walkability index nal priorities would change if the indicator weights
Wi = Weight of indicator i in category j were different. This process is called sensitivity anal-
Xi = Normalised value of indicator i in category j ysis (Mu, Pereyra-Rojas, 2016). A sensitivity analysis
was conducted to test the accuracy and robustness
of multi-criterion decisions through variation of in-
The normalised value of each indicator in each dicators. In other words, it can be applied for mod-
category was multiplied by the relative weights of el validation. Indicators’ values and weights could
the indicators extracted using AHP in the previous be evaluated in sensitivity analysis. However, due
stage. The weighted values of indicators in each cat- to their subjectivity, indicator weights are more im-
egory were then integrated into a sub-index. In the portant than values in sensitivity analysis. If the
final stage, the weight of each category was multi- ranking of the alternatives remains unchanged af-
plied by the sub-index value and aggregated into the ter sensitivity analysis, it could be concluded that
single WI (Gallin, 2011). the results of the multi-criterion decision analysis
are sufficiently accurate and robust. Otherwise, the
indicator weights should be redefined (Goh, Nobo-
3. Results rio, 2015; Rikalovic et al., 2015).
To test the sensitivity of the AHP model devel-
oped in this study, the indicator weights were var-
Pairwise comparisons and weight assignment based ied by ±20%. The results showed that the priorities
on the AHP method were conducted using Expert of segments were not changed as a result of a 20%
Choice software. The problem (walkability index change in the weight of indicators (Fig. 8).
development) was decomposed into a hierarchical
structure comprising goal, categories and indicators
(Fig. 3). In each level, the indicators were compared
pairwise according to their level of influence, using
4. Highlights
the judgments of local experts in urban planning
and design, transport planning and public health. The contribution of the current study to the field
The calculated weights of categories and indicators could be summarised as follows:
are illustrated in Figs 4–7. As the CR was 0.02, the • A framework to quantify walkability using a
judgments were considered consistent and could be set of indicators was developed.
applied for weight calculation. • Thirteen physical environment indicators
The normalised value for each indicator was were integrated into a single walkability in-
then multiplied by its relative weight. The individu- dex.

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


14 Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21

Table 1. Chaharbagh characteristics

Length 1.16 km
Number of car lanes 2
Number of sidewalks 3 (Two frontage sidewalks and one along the middle of street)
Min:1 m for frontages
Sidewalk width
Max: 1.5 m for middle sidewalk
Bicycle lanes 2 (along the middle of the street)

Table 2. Selected built-environment indicators affecting walkability

Overall
along
Category Indicator Ways of measurement
entire
street
Lighting Number of lights 86
Number of facilities provided to assist in the safe crossing of
Safety Chaharbagh Street by pedestrians, including median refuges,
Crossing availability 7
pelican crossings, guarded crossings, crosswalks, underpasses
and overpasses
Number of potential vehicle conflict points along Chaharbagh
Potential for vehicle conflicts 8
including intersections
Average sidewalk width along the street including middle and
Sidewalk width 1.16 m
frontages [m]
Number of obstacles e.g. poles, signs, chairs along sidewalks,
both middle and frontages. Stairs are considered an obstruc-
Obstructions 53
tion if no alternative is available for people with mobility dis-
Quality abilities
Number of facilities that assist pedestrians during their
journey along the entire street, including tactile paving, co-
Support facilities 19
lour-contrast kerbing, kerb ramps, lane markings signage,
landings on long ramps
Number of ramps for disabled people along the road and side-
Facilities for disabled people 4
walks
Natural features (trees) or
Area (m2) along Chaharbagh 21,400 m2
parks
Cinemas, cultural centres
(historical places, architec- Number 5
ture)
Amenities
Retail trade/ gastronomy /
and Attrac- Area (m2) 45,600 m2
services
tions
Fixed furniture: presence of
benches and other places to Number 23
rest
Public toilets Number 2
Public transportation Number of stations 3

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21 15

• The framework could be applied for correc- their importance, and integrating them into a sin-
tive actions in urban design. gle WI. It provides a quantitative benchmark to ex-
plore walkability potential provided by the street, as
well as determining factors contributing to the re-
5. Discussion and Conclusions sultant WI. The effect of built environment inter-
vention on walkability of an area is widely accepted
(Frank et al., 2006; Christian et al., 2011; Boulange
This investigation quantified 13 built environment et al., 2017). While other studies applied public in-
indicators under three categories (safety, quali- puts for qualifying built environment features, in
ty, attractiveness) affecting walkability in Chahar- this study built environment features and their ef-
bagh Street. A walkability index was defined and fects on walkability were investigated and quanti-
developed by assigning weights to the indicators by fied based on field measurements. Therefore, the

Fig. 3. Hierarchical structure of the aim

Fig. 4. Category weights using AHP Fig. 5. Safety indicator weights using AHP

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


16 Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21

Fig. 6. Quality indicator weights using AHP Fig. 7. Attractiveness indicator weights using AHP

Table 3. Sub-indices and WI for selected segments

Segment Safety Quality Attractiveness WI


1 0.328 0.216 0.122 0.248
2 0.185 0.200 0.258 0.201
3 0.189 0.228 0.267 0.217
4 0.106 0.175 0.217 0.153
5 0.192 0.181 0.136 0.180

results of the study could be a starting point for de- mixed use, which attracts a large number of people
veloping evidence-based urban planning strategies to Chaharbagh Street. With sub-indices and over-
to promote walking. The transparent and under- all WI below 0.5 in all segments, it could be con-
standable approach to indicator selection, quanti- cluded that Chaharbagh Street design could not be
fication, weighting and index development could supportive for a car-free plan in its current form
also help decision-makers in finding characteristics and more facilities are needed to encourage walk-
of the built environment that need improvement ing. In other words, safety, design and attractive-
to reach high walkability levels. Moreover, with re- ness are not at a level to encourage walking as a
duced funds for motorised transport, finding areas means to reduce the environmental impacts of oth-
with the potential to offset automobile use, and to er modes of transport.
reduce traffic congestion and air pollutants, is re- Despite the applicability of the obtained results
warding. for urban planning, some challenges are associated
Despite introducing car-free Tuesday to pro- with the investigation, including:
mote walking, no built environment interventions 1. Selection of walkability indicators: Selecting
are conducted to encourage walking in Chaharbagh. a set of indicators which provides a com-
Therefore, to promote walking, the right mix of in- prehensive overview of the considered sys-
terventions is needed for the study area. The devel- tem is challenging (Castillo, Pitfield, 2010).
oped indicators and the overall WI in this study are While selecting a small number of indicators
well suited to evaluating the Chaharbagh pedestri- is convenient, it may overlook important im-
an system and determining improvements needed pacts. On the other hand, a large number
for the street. According to the results, the exist- of indicators is comprehensive, but collec-
ing support facilities, crossing availability, facilities tion and analysis costs may be prohibitive.
for the disabled, public toilets and public transport Relevancy, measurability, simplicity and data
stations are insufficient to encourage walking, and availability were considered important selec-
improvements are needed in these aspects. On the tion criteria in this investigation.
other hand, the whole street provides outstanding

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21 17

Fig. 8. Sensitivity analysis for +20% changes in ‘Safety’ weight

2. Weighting: Assigned weights using pairwise References


comparisons could be highly subjective, as
they are subject to experts’ judgments. How-
ever, the acceptable consistency ratio and ro- Al-Shabeeb, A.R.R. (2015). A modified analytical hier-
bustness of assigned weights confirmed that archy process method to select sites for groundwater
a high level of agreement among respond- recharge in Jordan, Department of Geography.
ents was achieved in this study and the re- Albayrak, E. and Erensal, Y.C. (2004). Using analytic
sults are valid. hierarchy process (AHP) to improve human perfor-
3. Single index development: Despite mance: An application of multiple criteria decision
wide-ranging benefits of composite indices, making problem. Journal of Intelligent Manufactur-
there are ideas both for and against indices. ing, 15(4): 491-503. DOI: https://doi.org/10.1023/B:-
There are some ideas against composite in- JIMS.0000034112.00652.4c
dices due to the subjectivity in their crea- Alonso, J.A. and Lamata, M.T. (2006). Consistency in
tion (Cherchye et al., 2004). Moreover, it is the analytic hierarchy process: a new approach. Inter-
believed that no single index can answer all national journal of uncertainty, fuzziness and knowl-
questions and there is a need for multiple in- edge-based systems, 14(4): 445-459. DOI: https://doi.
dicators (Jollands, 2003). On the other hand, org/10.1142/S0218488518500095
some researchers considered composite in- Aragonés-Beltrán, P. Chaparro-González, F. Pas-
dices to be valuable communication tools tor-Ferrando, J.P. and Pla-Rubio, A. (2014) An AHP
because they limit the amount of present- (Analytic Hierarchy Process)/ANP (Analytic Network
ed information and allow for quick and easy Process)-based multi-criteria decision approach for
comparisons (Freudenberg, 2003). These two the selection of solar-thermal power plant invest-
ideas are two sides of the coin and it can ment projects. Energy, 66: 222-238. DOI: https://doi.
be concluded that indicator aggregation is org/10.1016/j.energy.2013.12.016
successful if clear assumptions and method- Atash, F. (1994). Redesigning suburbia for walking and
ology are used and if the index can be disag- transit: Emerging concepts. Journal of urban plan-
gregated to its components (Jollands, 2003). ning and development, 120(1): 48-57. DOI: https://
Overall, considering the usefulness of applying a doi.org/10.1061/(ASCE)0733-9488
walkability index for urban design and planning, it Bahrainy, H. Khosravi, H. Aliakbari, F. and Khosra-
is recommended to identify WI for other parts of vi, F. (2015). The Impact of Built Environment on
Isfahan Metropolitan or even Iran. It would be ben- Walkability, Case Study: North-West of Shiraz. Ar-
eficial for the local governments in designing more manshahr Architecture & Urban Development, 8(14):
liveable cities and promoting public health. 105-117.
Ball, K. Bauman, A. Leslie, E. and Owen, N. (2001).
Perceived environmental aesthetics and conveni-
ence and company are associated with walking for
exercise among Australian adults. Preventive medi-
cine, 33(5): 434-440. DOI: https://doi.org/10.1006/
pmed.2001.0912

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


18 Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21

Bauman, A. Wallner, F. Miners, A. and Westley-Wise, ioral Nutrition and Physical Activity, 8(1): 55. DOI:
V. (1996). No ifs no buts Illawarra physical activi- 10.1186/s12966-015-0269-2
ty project: Baseline research report. Warrong, NSW: Dur, F. Yigitcanlar, T. and Bunker, J. (2010). Towards
Commonwealth Department of Health and Family sustainable urban futures: evaluating urban sus-
Services. tainability performance of the Gold Coast, Austral-
Bhadra, S. Tanbir Sazid, A.K.M. and Esraz-Ul-Zan- ia. Proceedings of the 14th IPHS conference: urban
nat, M. (2016). A GIS Based Walkability Measure- transformation: controversies, contrasts and challeng-
ment within the Built Environment of Khulna City, es.
Bangladesh. Journal of Bangladesh Institute of Plan- EEA. (2005). EEA core set of indicators:Guide. Retrieved
ners, 8: 145-158. from Copenhagen.
Borajee, M. and Yakchali, S.H. (2011). Using the Efroymson, D. (2012), Moving dangerously, moving
AHP-ELECTRE III integrated method in a compet- pleasurably: Improving walkability in Dhaka: Using
itive profile matrix. International Conference on Fi- a BRT walkability strategy to make Dhaka’s transpor-
nancial Management and Economics, Hong Kong. tation infrastructure Pedestrian-Friendly.
Boulange, C. Pettit, C. and Giles-Corti, B. (2017). Ewing, R. and Handy, S. (2009). Measuring the unmeas-
The walkability planning support system: An evi- urable: Urban design qualities related to walkability.
dence-based tool to design healthy communities. Journal of Urban Design, 14(1): 65-84. DOI: doi.org/
International Conference on Computers in Urban 10.1080/13574809.2018.1554997
Planning and Urban Management, Australia. Frank, L. Devline, A. Johnstone, S. and Loon, J. (2010).
Brownson, R.C. Baker, E.A. Housemann, R.A. Bren- Neighbourhood Design, Travel, and Health in Metro
nan, L.K. and Bacak, S.J. (2001). Environmen- Vancouver: Using a Walkability Index. University of
tal and policy determinants of physical activity in British Columbia.
the United States. American journal of public health, Frank, L.D. and Pivo, G. (1994). Impacts of mixed use
91(12): 1995-2003. DOI: https://doi.org/10.2105/ and density on utilization of three modes of travel:
AJPH.2018.304649 single-occupant vehicle, transit, and walking. Trans-
Burden, D. Davis, D. Sprowls, S. Zykofsky, P. and portation research record, 1466: 44-52. DOI: https://
Wallwork, M. (2002). Street Design Guidelines for doi.org/10.3141/2323-07
Healthy Neighborhoods. Calif: Center for Livable Freudenberg, M. (2003). Composite indicators of coun-
Communities. try performance. OECD Publishing.
CAI-Asia. (2011). Walkability in Indi- Galanis, A. and Eliou, N. (2011). Evaluation of the pe-
an Cities, the Clean Air Initiative for destrian infrastructure using walkability indicators.
Asian Cities. Clean Air Initiative for Asian Cities WSEAS Transactions on Environment and Develop-
Center (http://www.cleanairinitiative.org). ment. 7(12): 385-394. DOI: https://doi.org/10.1016/
Castillo, H. and Pitfield, D.E. (2010). ELASTIC–A meth- S0950-0618 (02)00004-1
odological framework for identifying and selecting Gallin, N. (2001). Quantifying pedestrian friendliness-
sustainable transport indicators. Transportation Re- guidelines for assessing pedestrian level of service.
search Part D: Transport and Environment, 15(4): 179- Road and Transport Research, 10(1): 47-55. DOI:
188. DOI: https://doi.org/10.1016/j.trd.2014.11.017 https://doi.org/10.1007/s11116-006-9107-3
Cherchye, L. Moesen, W. and Puyenbroeck, T. (2004). Giles-Corti, B. and Donovan, R.J. (2003). Relative in-
Legitimately diverse, yet comparable: on synthesizing fluences of individual, social environmental, and
social inclusion performance in the EU. JCMS: Jour- physical environmental correlates of walking. Amer-
nal of Common Market Studies, 42(5): 919-955. DOI: ican journal of public health, 93(9): 1583-1589. DOI:
https://doi.org/10.1111/jcms.12519 https://doi.org/10.2105/AJPH.2018.304585
Christian, H.E. Bull, F.C. Middleton, N.J. Knuiman, Goh, E.G. and Noborio, K. (2015). Sensitivity Analysis
M.W. Divitini, M.L. Hooper, P. and Giles-Corti, B. and Validation for Numerical Simulation of Water In-
(2011). How important is the land use mix measure filtration into Unsaturated Soil. International scholar-
in understanding walking behaviour? Results from ly research notices.
the RESIDE study. International Journal of Behav- Görener, A. (2012). Comparing AHP and ANP: an ap-
plication of strategic decisions making in a manufac-

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21 19

turing company. International Journal of Business and Motamed, B. and Bitaraf, A. (2016). An empirical as-
Social Science, 3(11). DOI: https://doi.org/10.30845/ sessment of the walking environment in a megaci-
ijbss ty: Case study of Valiasr Street ArchNet-IJAR, 10(3).
Haghshenas, H. and Vaziri, M. (2012). Urban sustain- DOI: https://doi.org/10.26687/archnet-ijar.v10i3.1024
able transportation indicators for global comparison. Mu, E. and Pereyra-Rojas, M. (2016). Practical Deci-
Ecological Indicators, 15(1): 115-121. DOI: https://doi. sion Making: An Introduction to the Analytic Hier-
org/10.1016/j.ecolind.2006.01.003 archy Process (ahp) Using Super Decisions (Vol. 2):
Hawthorne, W. (1989). Why Ontarians Walk, Why On- Springer.
tarians Don’t Walk More: A Study into the Walking Nardo, M., Saisana, M., Saltelli, A. and Tarantola, S.
Habits of Ontarians: Energy Probe Research Foun- (2005). Tools for composite indicators building.
dation. National Census. (2017). Traffic accident census. http://
Humpel, N., Owen, N. and Leslie, E. (2002). Environ- transportsafety.ir/
mental factors associated with adults’ participation in Owen, N., Humpel, N., Leslie, E., Bauman, A. and
physical activity: a review. American Journal of pre- Sallis, J.F. (2004). Understanding environmental in-
ventive medicine, 22(3):188-199. DOI: https://doi. fluences on walking: review and research agenda.
org/10.5455/ajpmph.281781 American Journal of preventive medicine, 27(1): 67-
Jacobs, A.B. (1993). Great streets. ACCESS Magazine, 76. DOI: https://doi.org/10.1016/j.amepre.2004.03.006
1(3). Pikora, T., Giles-Corti, B., Bull, F., Jamrozik, K. and
Jollands, N. (2003). The usefulness of aggregate indica- Donovan, R. (2003). Developing a framework for as-
tors in policy making and evaluation: a discussion sessment of the environmental determinants of walk-
with application to eco-efficiency indicators in New ing and cycling. Social science & medicine, 56(8):
Zealand. 1693-1703. DOI: 10.1016/S0277-9536(02)00163-6.
Juwana, I. (2012). Development of a water sustainabili- Powell, K.E., Martin, L.M. and Chowdhury, P.P. (2003).
ty index for West Java,Indonesia. Victoria university. Places to walk: convenience and regular physical ac-
Krajnc, D. and Glavič, P. (2005). A model for integrat- tivity. American Journal of public health, 93(9): 1519-
ed assessment of sustainable development. Resourc- 1521. DOI: https://doi.org/10.2105/AJPH.93.9.1519
es, Conservation and Recycling, 43(2): 189-208. DOI: Rebecchi, A., Buffoli, M., Dettori, M., Appolloni, L.,
https://doi.org/10.1016/j.resconrec.2004.06.002 Azara, A., Castiglia, P., D’Alessandro, D., Capolon-
Krambeck, H.V. (2006). The global walkability index. go, S. (2019). Walkable Environments and Healthy
(Master’s), Massachusetts Institute of Technology. Urban Moves: Urban Context Features Assessment
Leslie, E., Coffee, N., Frank, L. and Owen, N. (2007). Framework Experienced in Milan. Sustainability, 11
Walkability of local communities: Using geograph- (10): 1-18. DOI: https://doi.org/10.3390/su11102778
ic information systems to objectively assess rele- Rikalovic, A., Cosic, I., Labati, R.D. and Piuri, V.
vant environmental attributes. Health & Place, 13: (2015). A Comprehensive Method for Industrial Site
111-122. DOI: https://doi.org/10.1016/j.healthplace. Selection: The Macro-Location Analysis. IEEE Sys-
2005.11.001 tems Journal, 11(4): 2971-2980. DOI: https://doi.
Li, F., Liu, X., Hu, D., Wang, R., Yang, W., Li, D. and org/10.1109/JSYST.2015.2444471
Zhao, D. (2009). Measurement indicators and an Saaty, T.L. (1990a). Decision making for leaders: the an-
evaluation approach for assessing urban sustainable alytic hierarchy process for decisions in a complex
development: A case study for China’s Jining City. world: RWS publications.
Landscape and Urban Planning, 90(3): 134-142. DOI: Saaty, T.L. (1990b). How to make a decision: the analytic
https://doi.org/10.1016/j.landurbplan.2008.10.022 hierarchy process. European Journal of operational re-
Mahini, A.S. and Gholamalifard, M. (2006). Siting search, 48(1): 9-26. DOI: https://doi.org/10.1016/0377-
MSW landfills with a weighted linear combination 2217(90)90057
methodology in a GIS environment. Internation- Saelens, B.E. Sallis, J.F. and Frank, L.D. (2003). Envi-
al Journal of Environmental Science & Technology, ronmental correlates of walking and cycling: findings
3(4), 435-445. DOI: https://doi.org/10.1016/j.was- from the transportation, urban design, and planning
man.2010.01.015 literatures. Annals of behavioral medicine, 25(2): 80-91.
DOI: https://doi.org/10.1207/S15324796ABM2502_03

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


20 Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21

Saghayi, M. Sadeghi, Z. (2013). Mid-term planning for tectural Research, 10(2): 113-129. DOI: https://doi.
using bicycle to achieve sustainable development: Is- org/10.26687/archnet-ijar.v10i2.956
fahan center case study, Urban Planning and Research, Zhou, P. Ang, B. and Poh, K. (2007). A mathematical
4(12): 95-116. programming approach to constructing composite in-
Saisana, M. (2011). Weighting methods, Seminar on dicators. Ecological economics, 62(2): 291-297. DOI:
composite Indicators: From Theory to Practice, Ispra. https://doi.org/10.1016/j.ecolecon.2006.12.020
Sallis, J. Bauman, A. and Pratt, M. (1998). Environ- Zito, P. and Salvo, G. (2011). Toward an urban transport
mental and policy interventions to promote physi- sustainability index: an European comparison. Euro-
cal activity. American Journal of preventive medicine, pean Transport Research Review, 3(4): 179-195. DOI:
15(4): 379-397. DOI: https://doi.org/10.1016/S0749- https://doi.org/10.1007/s12544-011-0059-0
3797(98)00076-2
San Francisco Department of Public Health. (2008). The
Pedestrian Environmental Quality Index (PEQI): An Appendix
assessment of the physical condition of streets and in-
tersections, San Francisco: San Francisco Department
of Public Health. The pairwise comparison matrix of the indicators
Sharma, M.J. Moon, I. and Bae, H. (2008). Analyt- involved in the decision was completed by experts
ic hierarchy process to assess and optimize distribu- using a value from 1 to 9 to reflect the experts’ rel-
tion network. Applied Mathematics and Computation, ative preference (also called “judgment”) in each of
202(1): 256-265. DOI: https://doi.org/10.1016/j. the compared pairs. For example, according to ex-
procs.2015.07.005 perts’ judgments, safety is very much more impor-
Southworth, M. (2005). Designing the walkable city. tant than quality (i.e. the intersection of the row
Journal of urban planning and development, 131(4): “safety” and column “quality” contains the value 7).
246-257. DOI: https://doi.org/10.1061/(ASCE)0733- For reverse comparison, the importance of “quali-
9488 ty” relative to the importance of “safety” would be
Tal, G. and Handy, S. (2012). Measuring nonmotorized the reciprocal of this value (quality/safety = 1/7) as
accessibility and connectivity in a robust pedestrian shown in Table A1. When the importance of an in-
network. Transportation Research Record: Journal of dicator is compared with itself, the input value is
the Transportation Research Board, 2299(1): 48-56. 1, which corresponds to an intensity of equal im-
DOI: https://doi.org/10.3141/2299-06 portance.
Tribby, C.P. Miller, H.J. Brown, B.B. Werner, C.M. and It is worth noting that for combining the judg-
Smith, K.R. (2016). Assessing Built Environment ments of various experts, the geometric mean was
Walkability using Activity-Space Summary Measures. applied to form one single pairwise comparison ma-
J Transp Land Use, 9(1): 187–207. DOI: https://doi. trix (Saaty, 1990).
org/10.5198/jtlu.2015.625 TA1
US Department of Health and Human Services. (1996).
Physical activity and health: a report of the Surgeon The procedure of calculating indicator weights
General: Diane Publishing. from a pairwise comparison matrix has several
USDG. (2007). Urban Street Design Guidelines. City of steps including:
Charlotte, Charlotte, NC. 1) Adding the values in each column,
Walkscore. (2019). www.walkscore.com. 2) Dividing each cell by the total of the column,
Wright, C. MacDougall, C. Atkinson, R. and Booth, 3) Calculating the average value of each row to
B. (1996). Exercise in daily life: supportive environ- find final weights (Mu, Pereyra-Rojas, 2016).
ments. Canberra, Commonwealth Government of
Australia. After calculating the weights, it is necessary to
Zayed, M.A.A. (2016). The effect of landscape elements check the consistency of the judgments. In a com-
on walkability in Egyptian gated communities. Arch- parison matrix, if a value of 2 is provided as the im-
Net-IJAR, Archnet International Journal of Archi- portance of the first indicator over the second and
a value of 3 is assigned as the importance of the

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC


Marzieh Reisi, Mozhgan Ahmadi Nadoushan and Lu Aye / Bulletin of Geography. Socio-economic Series / 46 (2019): 7–21 21

Table A1. Pair wise comparison matrix for categories

Safety Quality Attractiveness


Safety 1 7 8
Quality 1/7 1 2
Attractiveness 1/8 1/2 1

second indicator with respect to the third, the im- 4) The consistency index (CI) was calculated us-
portance of the first indicator with respect to the ing Eq. A1:
third should be 2×3 = 6, for a consistent judgment.
Assigning a value other than 6 would result in a
level of inconsistency in the matrix of judgments.
Therefore, some inconsistency is expected in AHP Where n = number of compared indicators
analysis as the result of the subjective preferences
of experts (Mu, Pereyra-Rojas, 2016). To calculate 5) Consistency ratio was obtained using Eq. A2:
consistency for the study, several steps were under-
taken:
1) Each value in the first column of the com- Where RI = Random consistent index presented by
parison matrix was multiplied by the first indicator (Alonso, Lamata, 2006) based on number of select-
weight. This process was continued for all columns ed indicators.
of the comparison matrix, resulting in a weighted
matrix. As mentioned earlier, all steps for weights and
2) The values in each row of the weighted matrix consistency calculation were conducted using Ex-
were added to obtain values called “weighted sum”. pert Choice software in this study.
3) The elements of the weighted sum were divid-
ed by the corresponding indicator weight and aver-
aged to extract λmax.

The proofreading of articles, positively reviewed and approved for publishing in the ‘Bulletin of
Geography. Socio-economic Series’, was financed from the funds of the Ministry of Science and Higher
Education earmarked for activities popularizing science, in line with Agreement No 695/P-DUN/2018.

Unauthentifiziert | Heruntergeladen 21.12.19 12:56 UTC

You might also like