3D Scanning & Reconstruction with Kinect
Download as PPTX, PDF0 likes415 views
1. The document describes how a 3D scanner uses a Kinect sensor to capture depth and color frames and convert them into a 3D point cloud representation of the real world. It explains how the Kinect coordinate system relates to the real world coordinate system and how spatial transformations are applied to convert between the two. 2. It also details how the anchor point between the Kinect and real world can be detected by recognizing a tennis ball in the color frame and using its corresponding depth value. 3. With the anchor point known, all other points captured by the Kinect can be accurately mapped to locations in the real world coordinate system, allowing for 3D modeling of the captured scene.
3D Scanning & Reconstruction with Kinect
- 1. 3D Scanner With Kinect MINH NGUYEN 08/2013
- 4. Depth Frame A depth frame = array of pixels 640 px 480px 13 bits = depth 3 bits = player 1 pixel = 16 bits
- 5. Color Frame • A color frame: Array of color pixels 640 px 480px Blue 1 color pixel = 32 bits Green Red 8 bits
- 6. Kinect Space & Real World Space Sensor direction z y x • Right-handed coordinate system North direction y z x Upward Kinect skeleton coordinate system Real world coordinate system
- 7. Depth Frame To Point Cloud • Pixel(x, y) Point (x1, y1, z, color) – Map pixel (x, y) to (x1, y1) in Kinect space. – Z = depth at pixel (x, y) – Color = map to color frame to get color at pixel (x, y)
- 8. Spatial sensor • Rotate by counter clockwise in Real World coordinate system North direction y z x Upward Yaw = Rotate about Z axis Roll = Rotate about X axis Pitch = Rotate about Y axis
- 9. Kinect To RealWorld • Point (X, Y, Z) in Kinect = Point (X?, Y?, Z?) in Real World ? • Need the real world origin (anchor point) ! • Algorithm: Transform = Rotate + Translate • Step 1: Rotate to bring current Kinect coordinate system back to normal Kinect position: • Rotate about X a ‘roll degree’ to get roll to zero • Rotate about Z a ‘-pitch degree’ o get pitch to zero • Rotate about Y a ‘-yaw degree’ to get yaw to zero • Step 2: Rotate Kinect coordinate system to Real world coordinate system : • Rotate about X ‘90 degree’ to bring Z upward • Rotate about Z ‘180 degree’ to bring Y to North • Step 3: Translate to anchor point IMPORTANT: The order of the rotation: (Roll -> Pitch -> Yaw) # (Yaw -> Pitch -> Roll)
- 10. Detect Anchor Point • Image processing to recognize tennis ball Ball (x, y) in color frame • Ball (x, y) Ball In Kinect (x1, y1, z) – Map color frame to depth frame to get (x1, y1) – Z = depth value at (x1, y1) • Kinect in Ball ? – Rotate to real world – Inverse sign
- 11. 3D Model Construction y z x z y x P(x, y, z) Kinect Origin Real World Origin Attitude (0, 0, 0) Ball in Kinect (0.1, 0. 0.3) P in Kinect (0, 0, 0.2) P in Real World (0.1, -0.1, 0) Attitude (40, 0, 0) Ball in Kinect (0.1, 0, 0.25) P in Kinect (-0.1, 0, 0.3) P in Real World (?, ?, ?)