Geospatial Visualization In Place:
 New Opportunities with Augmented Reality

             Mark Billinghurst
               HIT Lab NZ
         University of Canterbury
Christchurch Earthquakes
Christchurch Before and After
  Professional solutions available
    Autodesk REVIST, ESRI ArcGIS, Grass,etc
Emerging Novel User Interfaces
  Stereo Visualization
  Multi-touch screens
  Smartphones
VisionSpace




Next Generation Immersive Visualization
GeoSpatial Data
Limitations
  Complex interfaces
  Unintuitive interaction
  2D display for 3D content
  Separation from the real world
  Remote visualization
Augmented Reality (Azuma 97)
  Combines Real and Virtual Images
   -  Both can be seen at the same time
  Interactive in real-time
   -  The virtual content can be interacted with
  Registered in 3D
   -  Virtual objects appear fixed in space
Augmented Reality Examples
  Put AR pictures here
AR and Geospatial Visualization
  Enhanced Maps
    AR overlay on printed material
  Outdoor AR
    In place visualization
  Mobile AR
  Sensor Visualization
    Sensor data shown in place
AR Markerless Tracking




  OPIRA Library
    Uses natural features for tracking
    Fast, robust performance
Interactive AR Maps
  Markerless tracking
  3D model overlay
  Gesture input
Outdoor AR
  Highly accurate outdoor AR
   tracking system
    GPS, Inertial, RTK system
    HMD
  First prototype
    Laptop based
    2-3 cm accuracy
Image Registration




AR Stakeout Application
Mobile AR - Hardware

    GPS
                                       Example self-built working
   Antenna
                                       solution with PCI-based 3D graphics


                                               PCI 3D Graphics Board

                           Tracker
                          Controller
                                                                       PC104 Sound Card

                        DC to DC
 Wearable
                        Converter              CPU
 Computer
                                                               PC104 PCMCIA

             Battery


              GPS         RTK                           Hard Drive
                       correction
                         Radio

                                                                        Serial
                                                                        Ports

Columbia Touring Machine
2008 - Location Aware Phones




Motorola Droid       Nokia Navigator
Real World Information Overlay
  Tag real world locations
    GPS + Compass input
    Overlay graphics data on live video
  Applications
    Travel guide, Advertising, etc
  Eg: Mobilizy Wikitude
    Android based, Public API released
  Other companies
    Layar, AcrossAir, Tochnidot, RobotVision, etc
Layar – www.layar.com
  iPhone, Android
  > 2 million downloads
  1500+ information layers
HIT Lab NZ Outdoor AR Platform
  Cross platform
    Android, iPhone
  3D onsite visualization
    Intuitive user interface
  Positions content in space
    Camera, GPS, compass
  Client/Server software architecture
  Targeting museum guide/outdoor site applications
Prototype: Earthquake Reconstruction




  See past, present and future building designs
  Earthquake survivor stories shown on map view
  Collect user comments
  Android platform
Client/Server Architecture
 Web Interface	

Add models	




Web application java
 and php server	

                                      Android	

                                     application	

Database server	

         Postgres
Hydrosys Project (TU Graz)
  http://www.hydrosysonline.eu/
  Smart Earth visualization
  Sensor network
  Handheld AR display
AR View
Hydrosys Interface
Looking to the Future
Research Directions
  User Interface
  Wide Area Tracking
  Information Filtering
  Application Evaluation
  Social Networking
AR Navigation
  Many commercial AR browsers
    Information in place
    How to navigate to POI
2D vs. AR Navigation?




        VS
AR Navigation Study
  Users navigate between Points of Interest
  Three conditions
     AR: Using only an AR view
     2D-map: Using only a top down 2D map view
     AR+2D-map: Using both an AR and 2D map view
  Experiment Measures
     Quantitative
       -  Time taken, Distance travelled
     Qualitative
       -  Experimenter observations, Navigation behavior, Interviews
       -  User surveys, workload (NASA TLX)
HIT Lab NZ Test Platform – AR View
HIT Lab NZ Platform – Map View
Distance and Time




No significant differences
Paths Travelled
                    Red – AR
                    Blue – AR + Map
                    Yellow - Map
Navigation Behaviour
            Depends on interface
              Map doesn’t show
               short cuts
Survey Responses
User Comments
  AR
    “you don't know exactly where you are all of the time.”
    “using AR I found it difficult to see where I was going”
  Map
    “you were able to get a sense of where you were”
    “you are actually able to see the physical objects around you”
  AR+MAP
    “I used the map at the beginning to understand where the
     buildings were and the AR between each point”
    “You can choose a direction with AR and find the shortest way
     using the map.”
Usability Issues
  Screen readability in sunlight
  GPS inaccuracies
  Compass errors
  Touch screen difficulties
  No routing information
Lessons Learned
  User adapt navigation behaviour to guide type
    AR interface shows shortcuts
    Map interface good for planning
  Include map view in AR interface
    2D exocentric, and 3D egocentric
  Allow people to easily change between views
    May use Map far away, AR close
  Difficult to accurately show depth
Conclusions
•    AR allows for GeoSpatial Visualization in Place
•    Hardware and software platforms widely available
•    Many possible applications/commercial possibilities
•    Important research problems need to be solved
        – Wide area tracking
        – User experience
        – Information presentation/filtering
        – Etc..
More Information

•  Mark Billinghurst	

   –  mark.billinghurst@hitlabnz.org	

•  Website	

   –  www.hitlabnz.org

Geospatial Visualization in Place

  • 1.
    Geospatial Visualization InPlace: New Opportunities with Augmented Reality Mark Billinghurst HIT Lab NZ University of Canterbury
  • 3.
  • 4.
  • 6.
      Professional solutionsavailable   Autodesk REVIST, ESRI ArcGIS, Grass,etc
  • 7.
    Emerging Novel UserInterfaces   Stereo Visualization   Multi-touch screens   Smartphones
  • 8.
  • 9.
  • 10.
    Limitations   Complex interfaces  Unintuitive interaction   2D display for 3D content   Separation from the real world   Remote visualization
  • 12.
    Augmented Reality (Azuma97)   Combines Real and Virtual Images -  Both can be seen at the same time   Interactive in real-time -  The virtual content can be interacted with   Registered in 3D -  Virtual objects appear fixed in space
  • 13.
  • 14.
    AR and GeospatialVisualization   Enhanced Maps   AR overlay on printed material   Outdoor AR   In place visualization   Mobile AR   Sensor Visualization   Sensor data shown in place
  • 15.
    AR Markerless Tracking  OPIRA Library   Uses natural features for tracking   Fast, robust performance
  • 16.
    Interactive AR Maps  Markerless tracking   3D model overlay   Gesture input
  • 18.
    Outdoor AR   Highlyaccurate outdoor AR tracking system   GPS, Inertial, RTK system   HMD   First prototype   Laptop based   2-3 cm accuracy
  • 19.
  • 21.
    Mobile AR -Hardware GPS Example self-built working Antenna solution with PCI-based 3D graphics PCI 3D Graphics Board Tracker Controller PC104 Sound Card DC to DC Wearable Converter CPU Computer PC104 PCMCIA Battery GPS RTK Hard Drive correction Radio Serial Ports Columbia Touring Machine
  • 22.
    2008 - LocationAware Phones Motorola Droid Nokia Navigator
  • 23.
    Real World InformationOverlay   Tag real world locations   GPS + Compass input   Overlay graphics data on live video   Applications   Travel guide, Advertising, etc   Eg: Mobilizy Wikitude   Android based, Public API released   Other companies   Layar, AcrossAir, Tochnidot, RobotVision, etc
  • 24.
    Layar – www.layar.com  iPhone, Android   > 2 million downloads   1500+ information layers
  • 27.
    HIT Lab NZOutdoor AR Platform   Cross platform   Android, iPhone   3D onsite visualization   Intuitive user interface   Positions content in space   Camera, GPS, compass   Client/Server software architecture   Targeting museum guide/outdoor site applications
  • 28.
    Prototype: Earthquake Reconstruction  See past, present and future building designs   Earthquake survivor stories shown on map view   Collect user comments   Android platform
  • 29.
    Client/Server Architecture WebInterface Add models Web application java and php server Android application Database server Postgres
  • 30.
    Hydrosys Project (TUGraz)   http://www.hydrosysonline.eu/   Smart Earth visualization   Sensor network   Handheld AR display
  • 31.
  • 32.
  • 33.
  • 34.
    Research Directions   UserInterface   Wide Area Tracking   Information Filtering   Application Evaluation   Social Networking
  • 35.
    AR Navigation   Manycommercial AR browsers   Information in place   How to navigate to POI
  • 36.
    2D vs. ARNavigation? VS
  • 37.
    AR Navigation Study  Users navigate between Points of Interest   Three conditions   AR: Using only an AR view   2D-map: Using only a top down 2D map view   AR+2D-map: Using both an AR and 2D map view   Experiment Measures   Quantitative -  Time taken, Distance travelled   Qualitative -  Experimenter observations, Navigation behavior, Interviews -  User surveys, workload (NASA TLX)
  • 38.
    HIT Lab NZTest Platform – AR View
  • 39.
    HIT Lab NZPlatform – Map View
  • 40.
    Distance and Time Nosignificant differences
  • 41.
    Paths Travelled   Red – AR   Blue – AR + Map   Yellow - Map
  • 42.
    Navigation Behaviour   Depends on interface   Map doesn’t show short cuts
  • 43.
  • 44.
    User Comments   AR   “you don't know exactly where you are all of the time.”   “using AR I found it difficult to see where I was going”   Map   “you were able to get a sense of where you were”   “you are actually able to see the physical objects around you”   AR+MAP   “I used the map at the beginning to understand where the buildings were and the AR between each point”   “You can choose a direction with AR and find the shortest way using the map.”
  • 45.
    Usability Issues   Screenreadability in sunlight   GPS inaccuracies   Compass errors   Touch screen difficulties   No routing information
  • 46.
    Lessons Learned   Useradapt navigation behaviour to guide type   AR interface shows shortcuts   Map interface good for planning   Include map view in AR interface   2D exocentric, and 3D egocentric   Allow people to easily change between views   May use Map far away, AR close   Difficult to accurately show depth
  • 47.
    Conclusions •  AR allows for GeoSpatial Visualization in Place •  Hardware and software platforms widely available •  Many possible applications/commercial possibilities •  Important research problems need to be solved – Wide area tracking – User experience – Information presentation/filtering – Etc..
  • 48.
    More Information •  MarkBillinghurst –  [email protected] •  Website –  www.hitlabnz.org