FERMIUM - A Framework for Real-time Procedural Point Cloud Animation & Morphing

FERMIUM: A Framework for Real-time Procedural Point Cloud Animation and Morphing
Ole Wegen, Florence Böttger, Jürgen Döllner and Matthias Trapp
Hasso Plattner Institute, Faculty of Digital Engineering, University of Potsdam, Germany
VMV 2021: 26th International Symposium on Vision, Modeling, and Visualization

Motivation
What are point clouds/point-based geometry?
 A set comprising a (usually) high number of points
 Each point has attributes (e.g., position, color and normal)
 Compact and flexible representation of geometry
Applications in the field of...
 … computer animations and particle effects
 … physics-based simulations
 … real-world scene representation and analysis
FERMIUM: A Framework for Real-time Procedural Point Cloud Animation and Morphing 2
~3.3 million points - scanned using mobile device

Point Cloud Acquisition
LiDAR Scanning (e.g., on Mobile Devices) Sampling of Polygonal Geometry
Vertex-based Triangle-based Density-based
Position Color Normal Tangent Bi-Tangent
Exemplary point attributes obtained from polygonal geometry.

Types of Attribute Animation
Stateless Attribute-Animations Stateful Attribute-Animations

Animation to Visualize Acquisition Time

Interactive Explosion Views
September 27th FERMIUM: A Framework for Real-time Procedural Point Cloud Animation and Morphing 7
false-colored segmented point cloud (1,577,299 points) textured segmented point cloud (7,335,109 points)

Random Mapping Axis-based Mapping Octree-based Mapping Distance-based Mapping
Point Cloud Morphing
9
Comparison of mapping approaches for morphing two point clouds (Ghandi, ~200,000 points to Einstein ~100,000 points)

Point Cloud Morphing
10

Per-Attribute Easing Functions
11
Color
Easing Function
Position
Easing Function

Mesh-2-Mesh Morphing via Point Clouds
12

Combination of Animation Effects

Animated Point Clouds
14
point cloud animation using morphing point cloud animation using attribute animation

F E R M I U M
Pre-Processing
Client-side (CPU) Processing Server-side (GPU) Per-frame Processing
Camera & Object
Transform Animation
Point Animation
(Compute Shader)
Geometry Processing
(Transform Shader)
Rasterization
(Fixed Func.|Compute)
Post-Processing
(Compute Shader)
Final Frame
Geometry Buffer
Point Buffers
Spatial Data Structure
Animation Parameters and Data Buffers
Point Cloud Data
Raster Buffer

VBO-based Approach – Overview
17

VBO-based Approach – Shader
18
Vertex Buffer Attributes
Interpolation
in vec3 vertexPosition;
in vec3 vertexColor;
in vec3 vertexNormal;
in vec3 targetPosition;
in vec3 targetColor;
in vec3 targetNormal;
out Data{
vec3 position;
vec3 color;
vec3 normal;
};
void main(void)
{
//calculate morphingProgress
…
position = interpolate(targetPosition, vertexPosition, morphingProgressPosition);
color = interpolate(targetColor, vertexColor, morphingProgressColor);
normal = interpolate(targetNormal, vertexNormal, morphingProgressNormal);
…
}

SSBO-based Approach – Overview

SSBO-based Approach – Shader
Input Vertex Attributes
Interpolation
SSBO for target points
struct PointData{
vec4 position;
vec4 color;
vec4 normal;
};
in PointData vertex;
in uint targetPointID;
layout (std430) readonly buffer targetData
{
PointData point[];
};
out Data{
vec3 position;
vec3 color;
vec3 normal;
};
void main(void)
{
…
position = interpolate(point[targetPointID].position, vertex.position, morphingProgressPosition);
color = interpolate(point[targetPointID].color, vertex.color, morphingProgressColor);
normal = interpolate(point[targetPointID].normal, vertex.normal, morphingProgressNormal);
…
}

CS-based Approach – Shader
SSBOs for all input data
Interpolation
Target ID from invocation ID
layout (local_size_x = 1024) in;
struct PointData{
vec4 position;
vec4 color;
vec4 normal;
};
layout (std430, binding=0) buffer resultData{ PointData resultPoint[]; };
layout (std430, binding=1) readonly buffer sourceData{ PointData sourcePoint[]; };
layout (std430, binding=2) readonly buffer targetData{ PointData targetPoint[]; };
layout (std430, binding=3) readonly buffer assignmentData{ uint assignmentID[]; };
void main(void)
{
if(gl_GlobalInvocationID.x >= assignmentID.length())
return;
uint sourceID = gl_GlobalInvocationID.x;
uint targetID = assignmentID[sourceID];
resultPoint[sourceID].position =
interpolate(targetPoint[targetID].position, sourcePoint[sourceID].position, morphingProgressPosition);
resultPoint[sourceID].color =
interpolate(targetPoint[targetID].color, sourcePoint[sourceID].color, morphingProgressColor);
resultPoint[sourceID].normal =
interpolate(targetPoint[targetID].normal, sourcePoint[sourceID].normal, morphingProgressNormal);
}

Rendering Performance
23
Test Machine:
 Intel Core i5-4460 processor (4 cores, 3.2 GHz), 16GB RAM
 NVIDIA GeForce RTX 2080 Ti GPU, 16GB VRAM

Test Data & Setup
24
Point cloud morphing of different complexity:
 T.1: 10⁶ to 10⁶
 T.2: 10⁷ to 10⁶
 T.3: 10⁷ to 10⁷
 T.4: 2x 10⁷ to 10⁶
 T.5: 2x 10⁷ to 10⁷
 T.6: 2x 10⁷ to 2x 10⁷

Initialization Times

Mapping Computation Times

VRAM Consumption

Average Frame Times

Comparison of Approaches
Implementation Aspect VBO-based SSBO-based CS-based
Initialization Time Low Medium Medium
Memory Update/Transfer High Low Low
Mapping Computation Time High Medium Medium
Average Frame Time Stable Stable Unstable
VRAM Consumption Low Medium High
Modularity Low Low High

Conclusions & Future Work
Contributions:
 A holistic, extensible, real-time animation and morphing system for point clouds
 Evaluation of different point mapping and morphing implementation approaches
Future Work:
 Evaluate compute-shader-based rasterization for performance improvements
 Enable and evaluate clustering-based mapping approaches
 Combine with non-photorealistic rendering techniques
Acknowledgements:
 Maximilian Mayer and Daniel Atzberger for their support
 The 3D assets were obtained from free3d.com and open3dmodel.com
This work was funded by the German Federal Ministry of Education and Research
(BMBF) through grant 01IS15041 (“mdViProject”) and 01IS19006 (“KILabor ITSE”).
Exemplary cross-hatch rendering of a point cloud
(courtesy of Ronja Wagner).
Funded by: In Cooperation with:

FERMIUM - A Framework for Real-time Procedural Point Cloud Animation & Morphing

More Related Content

Similar to FERMIUM - A Framework for Real-time Procedural Point Cloud Animation & Morphing (20)

More from Matthias Trapp (20)

Recently uploaded (20)

FERMIUM - A Framework for Real-time Procedural Point Cloud Animation & Morphing