Availability of Mobile Augmented Reality System for Urban Landscape Simulation

Availability of Mobile Augmented Reality System
for Urban Landscape Simulation

Tomohiro Fukuda, Tian Zhang, and Nobuyoshi Yabuki
Division of Sustainable Energy and Environmental Engineering,
Graduate School of Engineering, Osaka University, Japan

Contents
1. Introduction
2. Developed Mobile AR
3. Comparative verification of landscape
simulation methods
1. Experimental Outline
2. Differences between Cloud-VR and mobile AR in Evaluation
3. Results and Discussion

4. Conclusion

2

Contents
1. Introduction
simulation methods

4. Conclusion

3

1. Introduction
1.1 Motivation -1
 In recent years, the need for landscape simulation has been
growing. A review meeting of future landscape is carried out on a
planned construction site in addition to being carried out in a
room.
 It is difficult for stakeholders to imagine concretely such an image
that is three-dimensional and does not exist. A landscape
visualization method using Computer Graphics (CG) and Virtual
Reality (VR) has been developed.
 However, this method requires much time and expense to make a
3D model such as the present terrain and artificial material in
addition to the subject of the landscape assessment. Moreover,
since consistency with real space is not achieved when using VR
on a planned construction site, it has the problem that a reviewer
cannot get an immersive experience.

A landscape study on site VR capture 4

1. Introduction
1.1 Motivation -2
 In this research, the authors focus Augmented Reality (AR) which
can superimpose an actual landscape acquired with a video
camera and 3DCG. When AR is used, a landscape assessment
object will be included in the present surroundings. Thereby, a
drastic reduction of the time and expense involved in carrying out
3DCG modeling of the present surroundings can be expected.
 A smartphone is widely available on the market level.

Sekai Camera Web Smartphone Market in Japan
http://sekaicamera.com/ 5

1. Introduction
モバイル型景観ARの進化
1.2 Previous Study
 In AR, realization of geometric
consistency with a video image of
an actual landscape and CG is an
important feature
1. Use of physical sensors such as GPS
(Global Positioning System) and gyroscope. To
realize highly precise geometric
consistency, special hardware which is
expensive is required.
Image Sketch (2005)

2006
©2012 Tomohiro Fukuda, Osaka-U 6

1. Introduction
1.2 Previous Study
2. Use of an artificial marker. Since an artificial marker needs to be always
visible by the AR camera, the movable span of a user is limited. Moreover,
to realize high precision, it is necessary to use a large artificial marker.

Yabuki, N., et al.: 2011, An invisible height evaluation
system for building height regulation to preserve good
landscapes using augmented reality, Automation in
Construction, Volume 20, Issue 3, 228-235.
artificial marker

7

1. Introduction
1.3 Aim
 The authors have developed and verified SOAR (Sensor Oriented Mobile AR)
system which realizes geometric consistency using GPS, a gyroscope and
a video camera which are mounted in a smartphone [1]. The authors have
also developed and verified GOAR (GIS Oriented Mobile AR) system which uses
GIS to obtain position data instead of GPS [2]. A low cost AR system with
high flexibility is realizable.
 In this research, the availability of landscape simulation method of a
mobile AR is considered, comparing with a photo montage and VR which
are existing methods.

1. Fukuda, et al.: SOAR: Sensor oriented Mobile Augmented Reality for Urban Landscape Assessment, Proceedings of the 17th International Conference on
Computer Aided Architectural Design Research in Asia (CAADRIA2012), pp. 387-396 (2012)
2. Fukuda, et al.: GOAR: GIS oriented Mobile Augmented Reality for Urban Landscape Assessment, 4th International Conference on Communications,
Mobility, and Computing (CMC 2012), pp. 183-186 (2012) 8

Contents
1. Introduction
simulation methods

4. Conclusion

9

2.1 Developed Mobile AR System
 Standard Spec Smartphone: GALAPAGOS 003SH (Softbank Mobile Corp.)
 Development Language: OpenGL-ES（Ver.2.0），Java（Ver.1.6）
 Development Environment: Eclipse Galileo（Ver.3.5）
 Location Estimation Technology: GIS includes Google Maps API and Digital
Elevation Model (DEM) which is 10 m mesh size (GOAR)

Video Camera
Spec of 003SH

OS Android™ 2.2
Qualcomm®MSM8255
CPU
Snapdragon® 1GHz
ROM：1GB
Memory
RAM：512MB
Weight ≒140g
Size ≒W62×H121×D12mm
Display Size 3.8 inch
Resolution 480×800 pixel

003SH

10

2.2 System Flow -1

While the CG model realizes
Calibration of a video camera
ideal rendering by the
Definition of landscape assessment 3DCG model perspective drawing method,
rendering of a video camera
Activation of AR system produces distortion.

Selection of 3DCG model

Starting of Activation of Activation of
Google Maps gyroscope video camera

Input of DEM
Angle information Capture of live
acquisition video image Distortion Calibration
Position
information
acquisition

Definition of position and angle
information on CG virtual camera

Superposition to live video image and 3DCG model

Display of AR image

Save of AR image

Calibration of the video camera
11
using Android NDK-OpenCV

2.2 System Flow -2
3DCG Model


Definition of landscape assessment 3DCG model
Geometry, Texture, Unit
Activation of AR system

3DCG model allocation file


Input of DEM
acquisition video image 3DCG model name, File name,
Position Position data (longitude, latitude,
information
acquisition orthometric height), Degree of
rotation angle, and Zone
Definition of position and angle number of the rectangular plane

Superposition to live video image and 3DCG model 3DCG model arrangement information file

Display of AR image

Save of AR image

Number of the 3DCG model
allocation information file, 12
Each name

2.2 System Flow -3






Input of DEM
acquisition video image
Position
information
acquisition



Display of AR image

Save of AR image

GUI of the Developed System
13

2.2 System Flow -4


yaw



Input of DEM
acquisition video image roll
Position pitch
information
acquisition

Coordinate System of Developed
AR system

Display of AR image

Save of AR image

14

2.2 System Flow -5




Selection of 3DCG model 1. The user tap the current
location on Google Maps

Input of DEM
Position
information
acquisition
2. The position data (longitude,
Definition of position and angle latitude) on the current location
information on CG virtual camera is obtained


Display of AR image

Save of AR image

3. Altitude is created using position
data (longitude, latitude) and
DEM
15

2.2 System Flow -6






Input of DEM
Position
information
acquisition



Display of AR image

Save of AR image

16

Contents
1. Introduction
simulation methods

4. Conclusion

17

3. Comparative verification of landscape simulation methods

3.1 Experimental Outline -1
The landscape simulation method of a mobile AR was verified through
comparative experiments using photo montage and VR, which are existing
methods. In order to use the same conditions as mobile AR, a cloud
computing type VR (cloud-VR) which can run Android OS was applied.

 Experimental Methodology
1. A 3D model of a virtual project was created. In this research, a high-rise
building (width: 40m, depth: 40m, height: 150m) and a wind power generator (height:
104m) were selected at varying distances (100m and 1200m) from a viewpoint.
Moreover, the Tokyo Sky Tree (height: 634m) was selected at a distance of
1500m from the viewpoint.
2. The operation of photo montage, Cloud-VR, and mobile AR was explained
to the subjects.
3. The subjects carried out the landscape study using photo montage for
about two minutes, using Cloud-VR for about five minutes, and using a
mobile AR for five minutes, in that order.
4. After the experiment, a questionnaire about the three landscape
simulation methods was implemented. The themes of the questionnaire
were the reproducibility of the landscape, the operability of the system,
and cost.


 Experimental photos and outputs

Photo montage Cloud-VR Mobile AR 19


 The viewpoint was the West Park (longitude: 34.672501111,
latitude: 135.20194833, altitude: 4m) in Port Island, Kobe city.

Regulation of building heights

viewpoint
Present state
21


 There were 28 subjects, of which 75% were male (N=21) and
25% were female (N=7).
 Regarding age, 50% were in their 20s (N=14), 14% were in their
30s (N=4), 22% were in their 40s (N=6), and 14% were in their
50s (N=4).
 54% subjects (N=15) had experience of using photo montage
and/or VR for landscape study before and 46% subjects (N=13)
had no such experience.

4, 14% 0, 0%

S
20代
S
30代
6, 22% 14, 50% S
40代
S
50代
S
60代
4, 14%

22


 The question items on the reproducibility of a landscape
were "reality", "reproducibility", "scale grasp", "immersion",
and "intuitiveness". The question items on operability were
"easiness", "feedback", and "interactivity". The question
items on cost were "expense", "creation time".
 The questionnaire result was scored using a 5-point scale.
Five points was the best value. An independent t-test was
performed according to simulation methods.

Question items
Large classification Small classification
Reality
Reproducibility
Reproducibility Scale grasp
Immersion
Intuitiveness
Easiness
Operability Feedback
Interactivity
Expense
Cost 23
Creation time


3.2 Differences between Cloud-VR and
mobile AR in Evaluation

 In regard to operability, mobile AR acquires the position data of
CG virtual camera by GPS or GIS, and acquires the angle data of
one with a gyroscope in real-time. Cloud-VR defines beforehand
the position data and the angle data of view-points. Features such
as fly-through, walk-through, parallel translation, rotation, etc. are
operated via a GUI (Graphical User Interface) on a screen.
 The screen size of the Cloud-VR is 10.1 inches, and the screen
size of the mobile AR differs from 3.8 inches. However, the
subjects considered the screens to be the same size.
 At the time of the experiment, although texture mapping was
used in the Cloud-VR, it was not used in the mobile AR. Since it is
technically possible, the mobile AR was evaluated as if the texture
mapping had been used.

24


3.3 Results and Discussion
 As for the mobile AR, all the user groups gave a score of four or
more points for "scale grasp", "immersion", "intuitiveness",
"easiness", "feedback" and "interactivity". The score of 3.2 or
more points was given for "reality", "reproducibility", "expense"
and "creation time" which were the remaining items.
 The items, “immersion", "feedback", and "interactivity” of photo
montage and the items "expense", "creation time" of Cloud-VR
were lower than three points. That is, mobile AR was given a high
evaluation for all items.

25


3.3 Results and Discussion: AR vs. Photo Montage
 In all the user groups, a significant difference was obtained for the
items "immersion", "easiness", "feedback", and "interactivity". In
the experienced subjects, a further significant difference was
obtained for "intuitiveness".
 Why "feedback" and "interactivity" were given a high evaluation is
considered. Both photo montage and mobile AR create a 3DCG
model superimposed on a photo or live video. A photo montage is
a two-dimensional picture and cannot respond to changes in the
viewpoint position or direction during study. On the other hand,
mobile AR can change the position and direction of the viewpoint
corresponding to the user's intention.

Compa Rea Reprod Scale Immer Intuitiv Easin Feedb Intera Expen Creati
on
rison lity ucibility grasp sion eness ess ack ctivity se time
Whole AR PM △△△ △△ △△△ △△△
(N=28) AR VR ▼ △ △△ △ △△△ △△△
Experie AR PM △△△ △ △△ △△△ △△△
nced
(N=15) AR VR ▼ △△ △△ △△
Inexperi AR PM △ △ △△△ △△△
enced
(N=13) AR VR △ △△△ △△△
△/▼: significant difference 5%, △△/▼▼: significant difference 1%, △△△/▼▼▼: significant
26
difference 0.1%, △: Left conditions have a large value., ▼: Right conditions have a large value.


3.3 Results and Discussion: AR vs. VR
 In all the user groups, a significant difference was obtained for the
items "expense" and "creation time". VR needs to create all 3DCG
models. AR creates only the subject in the 3DCG model. Therefore,
when an object for landscape assessment created using a 3D
model is not large, the cost performance of AR is high.
 About "reproducibility", the reason the significant difference was
obtained for the Cloud-VR may be associated with a problem of
the optical integrity of AR. Since VR is created using a full 3DCG
model, optical integrity is realized within the VR virtual space. On
the other hand, AR differs in the influence of light on the 3DCG
model and live video, and also differs in shade expression.

Compa Rea Reprod Scale Immer Intuitiv Easin Feedb Intera Expen Creati
on
rison lity ucibility grasp sion eness ess ack ctivity se time
Whole AR PM △△△ △△ △△△ △△△
(N=28) AR VR ▼ △ △△ △ △△△ △△△
Experie AR PM △△△ △ △△ △△△ △△△
nced
(N=15) AR VR ▼ △△ △△ △△
Inexperi AR PM △ △ △△△ △△△
enced
(N=13) AR VR △ △△△ △△△
△/▼: significant difference 5%, △△/▼▼: significant difference 1%, △△△/▼▼▼: significant
27
difference 0.1%, △: Left conditions have a large value., ▼: Right conditions have a large value.

Contents
1. Introduction
simulation methods

4. Conclusion

28

4. Conclusion

4.1 Conclusion
 For mobile AR, which is used as a smartphone
platform, a score of 3.2 or more points was
obtained for reproducibility of a landscape,
operability, and cost. When comparing it with
existing methods, mobile AR is evaluated as
being better than equivalent.
 When mobile AR was compared with photo
montage, a significant difference was obtained
for "immersion" and "intuitiveness" of landscape
reproducibility, and for "easiness", "feedback"
and "interactivity" of operability. This was
because mobile AR can respond to changes in
the user's viewpoint position or orientation,
whereas photo montage cannot.
 When mobile AR was compared with Cloud-VR, a
significant difference was obtained for "expense"
and "creation time" of cost. VR needs to create
all 3DCG models. AR creates only the subject
using a 3DCG model. Therefore, when an object
for landscape assessment created using a 3D
model is not large, the cost performance of AR is
high.
29

4. Conclusion

4.2 Future Work
 A future work should attempt to improve the optical integrity of
the AR system.

30

Thank you for your attention!

E-mail: fukuda@see.eng.osaka-u.ac.jp
Twitter: fukudatweet
Facebook: Tomohiro Fukuda
Linkedin: Tomohiro Fukuda

Availability of Mobile Augmented Reality System for Urban Landscape Simulation

More Related Content

What's hot

Viewers also liked

Similar to Availability of Mobile Augmented Reality System for Urban Landscape Simulation

More from Tomohiro Fukuda

Recently uploaded

Availability of Mobile Augmented Reality System for Urban Landscape Simulation