Object Tracking with the
ARMedia SDK
ALESSANDRO TERENZI
CTO @ INGLOBE TECHNOLOGIES SRL
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
INTRODUCTION
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
INTRODUCTION
• ARMedia: Company’s Brand for…
• Augmented Reality
• Virtual Reality
• Perceptual Computing
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
INTRODUCTION
• The Offer:
• ARMedia Plugins: 3ds Max, Maya, Cinema 4D, SketchUp, …
• HyperSpaces: General Purpose Authoring Platform
• HyperIndustry: Authoring Platform for the Industry
• HyperPublisher: Authoring Platform for Publishing
• ARMedia SDK: Software Developers
• Custom Solutions…
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
INTRODUCTION
• The Market:
• Architecture, Engineering & Construction (AEC)
• Healthcare
• Manufacturing
• Aerospace
• Automotive
• Cultural Heritage
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
INTRODUCTION
• The Clients:
• 40 Countries
• 2.000+ Customers
• 50.000+ Registered Users
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG
• Version 0.0 (June 2013): Object Tracking
• iOS/Android/Windows/MacOS: Not a SDK!
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG
• Version 1.0 (June 2014): Object Tracking
• iOS/Android/Unity3D
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG
• Version 1.1.0 (June 2015):
• + 360 Panorama Viewer, 3D Viewer, “Motion” Tracking
• + more “ways” to load 3D models…
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG
• Version 2.0.0 (March 2016):
• + Planar Tracking
• + Location Tracking
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG
• Version 2.1.0 (October 2016):
• + Performances Improvements
• + APIs Documentation
• + Multiple Object Tracking
• + Improved & Simplified Trackable Data Creation
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: OVERVIEW
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: OVERVIEW
• Tools
• Target Creation
• ARMedia Toolset (iOS/Android)
• ARMedia Developer Portal
• 3rd Party Components (Optional)
• Content Related
• Conversion Utilities
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: OVERVIEW
• Tracking Technologies —> Tracking Module
• Planar (2D)
• Object (3D)
• Location (GPS)
• Motion (IMUs)
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: OVERVIEW
• Content & Visualisation (Rendering)
• 3rd Party Engines or Native APIs
• SDK APIs —> Rendering Module
• Unity3D Plugin
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: OVERVIEW
• Licensing & Subscriptions
• 1, 6, 12 Months…
• Application Keys…
• Targets Creation & Management…
• …the Developer Portal, Helpdesk & Support
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Multiple Object Tracking
• Object?
• Multiple?
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
1 (*.JPG)
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
2 (PSZ)
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
Have CAD model or reconstructed mesh?
(OBJ, MTL, PNG/JPG)
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
3
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
4
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
5
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
6
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
7
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
8
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
9 (ZIP)
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
10
(ZIP) ARMedia Toolset
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs…
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• Simplicity: as few APIs as possible (Objective-C, Java) …
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• Supported Platforms
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• License/Application Keys
• Every single app must have a valid license key, generated on
the Developer Portal
• The role of the bundle identifier…
• Set using the SDK APIs…
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• Tracking Module
• Just tracking APIs (and related events)…
• Camera capturing (not mandatory)
• Just need an OpenCV::Mat object
• Modularity:
• Capturing (Input)
• Rendering (Output)
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• Rendering Module
• Simple graphics APIs (few high level methods)
• Loading models (“as is”)
• Managing animations (rigid body + bones systems)
• Components-based (ViewControllers/Activities)
• Not only AR… VR as well…
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
NEW!
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• Unity3D Plugin
• Simple APIs (C#)
• Ready-to-use components (prefabs)
• Examples for each tracking method
• Interactions with ‘real’ objects: easy!
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: HANDS-ON…
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: HANDS-ON…
• Unity3D Plugin:
• Import SDK Package
• Import Trackable Data
• Setup Scene & App Logic
• Configure Player Settings
• Build & Run
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: COMING SOON…
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: COMING SOON…
• GearVR
• Smart Glasses & Headsets…
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
CONTACTS
Alessandro Terenzi
a.terenzi@inglobetechnologies.com
Skype: a.terenzi

Alessandro Terenzi (InGlobe Technologies): Object Tracking with ARMedia SDK

  • 1.
    Object Tracking withthe ARMedia SDK ALESSANDRO TERENZI CTO @ INGLOBE TECHNOLOGIES SRL #AWE2017 Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
  • 2.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) INTRODUCTION
  • 3.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) INTRODUCTION • ARMedia: Company’s Brand for… • Augmented Reality • Virtual Reality • Perceptual Computing
  • 4.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) INTRODUCTION • The Offer: • ARMedia Plugins: 3ds Max, Maya, Cinema 4D, SketchUp, … • HyperSpaces: General Purpose Authoring Platform • HyperIndustry: Authoring Platform for the Industry • HyperPublisher: Authoring Platform for Publishing • ARMedia SDK: Software Developers • Custom Solutions…
  • 5.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) INTRODUCTION • The Market: • Architecture, Engineering & Construction (AEC) • Healthcare • Manufacturing • Aerospace • Automotive • Cultural Heritage
  • 6.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) INTRODUCTION • The Clients: • 40 Countries • 2.000+ Customers • 50.000+ Registered Users
  • 7.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: CHANGE LOG
  • 8.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: CHANGE LOG • Version 0.0 (June 2013): Object Tracking • iOS/Android/Windows/MacOS: Not a SDK!
  • 9.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: CHANGE LOG • Version 1.0 (June 2014): Object Tracking • iOS/Android/Unity3D
  • 10.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: CHANGE LOG • Version 1.1.0 (June 2015): • + 360 Panorama Viewer, 3D Viewer, “Motion” Tracking • + more “ways” to load 3D models…
  • 11.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: CHANGE LOG • Version 2.0.0 (March 2016): • + Planar Tracking • + Location Tracking
  • 12.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: CHANGE LOG • Version 2.1.0 (October 2016): • + Performances Improvements • + APIs Documentation • + Multiple Object Tracking • + Improved & Simplified Trackable Data Creation
  • 13.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: OVERVIEW
  • 14.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: OVERVIEW • Tools • Target Creation • ARMedia Toolset (iOS/Android) • ARMedia Developer Portal • 3rd Party Components (Optional) • Content Related • Conversion Utilities
  • 15.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: OVERVIEW • Tracking Technologies —> Tracking Module • Planar (2D) • Object (3D) • Location (GPS) • Motion (IMUs)
  • 16.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: OVERVIEW • Content & Visualisation (Rendering) • 3rd Party Engines or Native APIs • SDK APIs —> Rendering Module • Unity3D Plugin
  • 17.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: OVERVIEW • Licensing & Subscriptions • 1, 6, 12 Months… • Application Keys… • Targets Creation & Management… • …the Developer Portal, Helpdesk & Support
  • 18.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW?
  • 19.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Multiple Object Tracking • Object? • Multiple?
  • 20.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Trackable Data Creation Process (Objects): 1 (*.JPG)
  • 21.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Trackable Data Creation Process (Objects): 2 (PSZ)
  • 22.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Trackable Data Creation Process (Objects): Have CAD model or reconstructed mesh? (OBJ, MTL, PNG/JPG)
  • 23.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Trackable Data Creation Process (Objects): 3
  • 24.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Trackable Data Creation Process (Objects): 4
  • 25.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Trackable Data Creation Process (Objects): 5
  • 26.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Trackable Data Creation Process (Objects): 6
  • 27.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Trackable Data Creation Process (Objects): 7
  • 28.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Trackable Data Creation Process (Objects): 8
  • 29.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Trackable Data Creation Process (Objects): 9 (ZIP)
  • 30.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: WHAT’S NEW? • Trackable Data Creation Process (Objects): 10 (ZIP) ARMedia Toolset
  • 31.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs…
  • 32.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs • Simplicity: as few APIs as possible (Objective-C, Java) …
  • 33.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs • Supported Platforms
  • 34.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs • License/Application Keys • Every single app must have a valid license key, generated on the Developer Portal • The role of the bundle identifier… • Set using the SDK APIs…
  • 35.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs • Tracking Module • Just tracking APIs (and related events)… • Camera capturing (not mandatory) • Just need an OpenCV::Mat object • Modularity: • Capturing (Input) • Rendering (Output)
  • 36.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs
  • 37.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs
  • 38.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs
  • 39.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs
  • 40.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs • Rendering Module • Simple graphics APIs (few high level methods) • Loading models (“as is”) • Managing animations (rigid body + bones systems) • Components-based (ViewControllers/Activities) • Not only AR… VR as well…
  • 41.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs
  • 42.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs
  • 43.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs NEW!
  • 44.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs
  • 45.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs • Unity3D Plugin • Simple APIs (C#) • Ready-to-use components (prefabs) • Examples for each tracking method • Interactions with ‘real’ objects: easy!
  • 46.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs
  • 47.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs
  • 48.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs
  • 49.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: APIs
  • 50.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: HANDS-ON…
  • 51.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: HANDS-ON… • Unity3D Plugin: • Import SDK Package • Import Trackable Data • Setup Scene & App Logic • Configure Player Settings • Build & Run
  • 52.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: COMING SOON…
  • 53.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) SDK: COMING SOON… • GearVR • Smart Glasses & Headsets…
  • 54.
    #AWE2017 Augmented World Expo(USA) - June 1, 2017 - Santa Clara (CA) CONTACTS Alessandro Terenzi [email protected] Skype: a.terenzi