Azure Kinect DK C/C++ 開発概要(仮)
Download as PPTX, PDF2 likes3,784 views
This document provides an overview and code samples for using the Azure Kinect SDK and Azure Kinect Body Tracking SDK in C/C++. It explains how to install the SDKs, generate projects, capture and process sensor data, playback recordings, and track body poses. Code samples are provided for initializing devices, capturing images, transforming data between coordinate systems, and drawing body skeletons overlaid on images.
{
const int32_t x = position[1], y = position[0];
const uint32_t index = body_index_map.at<uint8_t>( y, x );
if( index != K4ABT_BODY_INDEX_MAP_BACKGROUND ){
pixel = colors[index % colors.size()];
}
}
);
- Background Index is 255 (K4ABT_BODY_INDEX_MAP_BACKGROUND)](https://image.slidesharecdn.com/j7xhacixskyx3a4dhz01-signature-e51c0e2af7879bfe18f47025c40ae96fa03b59dd3b3d0196307a01ef0f777793-poli-190820123000/85/Azure-Kinect-DK-C-C-16-320.jpg)
![Azure Kinect Body Tracking SDK Programming
// Get Body Skeleton
std::vector<k4abt_body_t> bodies = body_frame.get_bodies();
// Get Image that used for Inference
k4a::capture body_capture = body_frame.get_capture();
k4a::image skeleton_image = body_capture.get_color_image();
cv::Mat skeleton = k4a::get_mat( skeleton_image );
// Draw Body Skeleton
for( const k4abt_body_t& body : bodies ){
for( const k4abt_joint_t& joint : body.skeleton.joints ){
k4a_float2_t position;
const bool result = calibration.convert_3d_to_2d( joint.position, K4A_CALIBRATION_TYPE_DEPTH, K4A_CALIBRATION_TYPE_COLOR, &position );
if( !result ){
continue;
}
const int32_t id = body.id;
const cv::Point point( static_cast<int32_t>( position.xy.x ), static_cast<int32_t>( position.xy.y ) );
cv::circle( skeleton, point, 5, colors[id % colors.size()], -1 );
}
}
- Get Image that used for Inference
- Convert Joint Position (3D) to 2D for Draw Circle on Image](https://image.slidesharecdn.com/j7xhacixskyx3a4dhz01-signature-e51c0e2af7879bfe18f47025c40ae96fa03b59dd3b3d0196307a01ef0f777793-poli-190820123000/85/Azure-Kinect-DK-C-C-17-320.jpg)
Azure Kinect DK C/C++ 開発概要(仮)
- 1. Azure Kinect DK C/C++開発概要(仮)
- 2. Self Introduction 杉浦 司 (Tsukasa Sugiura) - Freelance Programmer - Microsoft MVP for Windows Development - Point Cloud Library Maintainer - @UnaNancyOwen
- 3. Azure Kinect DK Sensor
- 4. Azure Kinect SDK Introduction Azure Kinect Sensor SDK (k4a / k4arecord) Azure Kinect Body Tracking SDK (k4abt) - Color, Depth, Infrared, IMU, Point Cloud - Open Source Library hosted on GitHub (Depth Engine is Closed Source) - Cross Platform (Windows, Linux) - C API, C++ and C# Wrapper - Body Index Map, Skeleton (26 Joints/Person) - Deep Learning based Pose Estimation - Closed Source Library - Cross Platform (Windows, Linux) - C API, (C++ Wrapper)
- 5. How to Install Azure Kinect SDK? Azure Kinect Sensor SDK - Install Pre-Built SDK using Installer - Build and Install SDK from Source Code Azure Kinect Body Tracking SDK - Install SDK using Installer - Install NVIDIA GPU Driver and Visual C++ 2015 Runtime for ONNX Runtime (CUDA Backend) Download Azure Kinect Body Tracking SDK | Microsoft Docs https://docs.microsoft.com/en-us/azure/Kinect-dk/body-sdk-download About Azure Kinect Sensor SDK | Microsoft Docs https://docs.microsoft.com/en-us/azure/Kinect-dk/about-sensor-sdk
- 6. How to Generate Project with Azure Kinect SDK? cmake_minimum_required( VERSION 3.6 ) project( Solution ) add_executable( Project main.cpp ) # Azure Kinect Sensor SDK (Official Support) find_package( k4a REQUIRED ) find_package( k4arecord REQUIRED ) # Azure Kinect Body Tracking SDK (Un-Official Support) set( CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}" ) find_package( k4abt REQUIRED ) if( k4a_FOUND AND k4arecord_FOUND AND k4abt_FOUND ) target_link_libraries( Project k4a::k4a ) target_link_libraries( Project k4a::k4arecord ) target_link_libraries( Project k4a::k4abt ) endif() CMake Module for Azure Kinect Sensor SDK | GitHub Gists https://gist.github.com/UnaNancyOwen/5ce9115ba8b71d12982e5aa9f99788f1#file-findk4abt-cmake Include CMake and MS Build files in the MSI #370 | GitHub/Azure-Kinect-Sensor-SDK https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/370
- 7. Azure Kinect Sensor SDK Overview Capture Image Playback Device
- 8. Azure Kinect Sensor SDK Programming // Azure Kinect Sensor SDK #include <k4a/k4a.h> #include <k4a/k4a.hpp> /* C++ Wrapper */ // Get Connected Devices const int32_t device_count = k4a::device::get_installed_count(); if( device_count == 0 ){ throw k4a::error( "Failed to found device!" ); } // Open Default Device k4a::device device = k4a::device::open( K4A_DEVICE_DEFAULT ); // Start Cameras with Configuration k4a_device_configuration_t configuration = K4A_DEVICE_CONFIG_INIT_DISABLE_ALL; configuration.color_format = K4A_IMAGE_FORMAT_COLOR_BGRA32; configuration.color_resolution = K4A_COLOR_RESOLUTION_1080P; configuration.depth_mode = K4A_DEPTH_MODE_NFOV_UNBINNED; configuration.wired_sync_mode = K4A_WIRED_SYNC_MODE_STANDALONE; configuration.synchronized_images_only = true; device.start_cameras( &configuration );
- 9. Azure Kinect Sensor SDK Programming // Get Capture k4a::capture capture; const std::chrono::milliseconds timeout( K4A_WAIT_INFINITE ); const bool result = device.get_capture( &capture, timeout ); if( !result ){ break; } // Get Color Image k4a::image color_image = capture.get_color_image(); cv::Mat color = k4a::get_mat( color_image ); // Get Depth Image k4a::image depth_image = capture.get_depth_image(); cv::Mat depth = k4a::get_mat( depth_image ); - k4a::get_mat() is utility function that defined in util.h Utility for Azure Kinect Sensor SDK | GitHub Gists https://gist.github.com/UnaNancyOwen/9f16ce7ea4c2673fe08b4ce4804fc209#file-util-h
- 10. Azure Kinect Sensor SDK Programming // Show Image depth.convertTo( depth, CV_8U, -255.0 / 5000.0, 255.0 ); cv::imshow( "color", color ); cv::imshow( "depth", depth ); // Wait Key const int32_t key = cv::waitKey( 30 ); if( key == 'q' ){ break; } // Close Device device.close(); // Clear Handle color_image.reset(); depth_image.reset(); capture.reset();
- 11. Azure Kinect Sensor SDK Programming // Initialize Transformation k4a::calibration calibration = device.get_calibration( configuration.depth_mode, configuration.color_resolution ); k4a::transformation transformation = k4a::transformation( calibration ); // Transform Color Image to Depth Camera k4a::image transformed_color_image = transformation.color_image_to_depth_camera( depth_image, color_image ); cv::Mat transformed_color = k4a::get_mat( transformed_color_image ); // Transform Depth Image to Color Camera k4a::image transformed_depth_image = transformation.depth_image_to_color_camera( depth_image ); cv::Mat transformed_depth = k4a::get_mat( transformed_depth_image ); // Transform Depth Image to Point Cloud k4a::image xyz_image = transformation.depth_image_to_point_cloud( depth_image, K4A_CALIBRATION_TYPE_DEPTH ); //k4a::image xyz_image = transformation.depth_image_to_point_cloud( transformed_depth_image, K4A_CALIBRATION_TYPE_COLOR ); cv::Mat xyz = k4a::get_mat( xyz_image ); - Transform Image to Other Coordinate System Color Image → Depth Camera Depth Image → Color Camera Depth Image → Point Cloud Add overload functions that returns k4a::image in k4a::transformation #596 | GitHub/Azure-Kinect-Sensor-SDK https://github.com/microsoft/Azure-Kinect-Sensor-SDK/pull/596 // Close Transformation transformation.destroy();
- 12. Azure Kinect Sensor SDK Programming // Azure Kinect Sensor SDK #include <k4arecord/playback.h> #include <k4arecord/playback.hpp> /* C++ Wrapper from Azure Kinect Sensor SDK v1.2.0 */ // Open Playback File k4a::playback playback = k4a::playback::open( "./path/to/file.mkv" ); // Get Capture k4a::capture capture; bool result = playback.get_next_capture( &capture ); if( !result ){ break; } // Close Playback playback.close(); - NOTE: k4arecorder (Azure Kinect Sensor SDK v1.1.x) has a bug that first 2-frames are empty. Move C++ wrapper for playback #493 | GitHub/Azure-Kinect-Sensor-SDK https://github.com/microsoft/Azure-Kinect-Sensor-SDK/pull/493
- 13. Azure Kinect Sensor SDK Programming Azure Kinect Office Sample Recordings | Microsoft Download Center https://www.microsoft.com/en-us/download/details.aspx?id=58385
- 14. Azure Kinect Body Tracking SDK Overview Capture Queue Playback Device Frame Body
- 15. Azure Kinect Body Tracking SDK Programming // Azure Kinect Body Tracking SDK #include <k4abt.h> #include "k4abt.hpp" /* Un-Official C++ Wrapper */ //#include <k4abt.hpp> /* Official C++ Wrapper Provide from Next Release */ // Create Tracker k4abt::tracker tracker = k4abt::tracker::create( calibration ); if( !tracker ){ throw k4a::error( "Failed to create tracker!" ); } // Enqueue Capture tracker.enqueue_capture( capture ); // Pop Tracker Result k4abt::frame body_frame = tracker.pop_result(); - Official C++ wrapper is coming soon … C++Wrapper for Azure Kinect Body Tracking SDK | GitHub Gists https://gist.github.com/UnaNancyOwen/41b578f5d272aa6f22cf8ff6565fb71b#file-k4abt-hpp
- 16. Azure Kinect Body Tracking SDK Programming // Get Body Index Map k4a::image body_index_map_image = body_frame.get_body_index_map(); cv::Mat body_index_map = k4a::get_mat( body_index_map_image ); // Draw Body Index Map cv::Mat body_index = cv::Mat::zeros( body_index_map.size(), CV_8UC3 ); body_index.forEach<cv::Vec3b>( [&]( cv::Vec3b& pixel, const int32_t* position ){ const int32_t x = position[1], y = position[0]; const uint32_t index = body_index_map.at<uint8_t>( y, x ); if( index != K4ABT_BODY_INDEX_MAP_BACKGROUND ){ pixel = colors[index % colors.size()]; } } ); - Background Index is 255 (K4ABT_BODY_INDEX_MAP_BACKGROUND)
- 17. Azure Kinect Body Tracking SDK Programming // Get Body Skeleton std::vector<k4abt_body_t> bodies = body_frame.get_bodies(); // Get Image that used for Inference k4a::capture body_capture = body_frame.get_capture(); k4a::image skeleton_image = body_capture.get_color_image(); cv::Mat skeleton = k4a::get_mat( skeleton_image ); // Draw Body Skeleton for( const k4abt_body_t& body : bodies ){ for( const k4abt_joint_t& joint : body.skeleton.joints ){ k4a_float2_t position; const bool result = calibration.convert_3d_to_2d( joint.position, K4A_CALIBRATION_TYPE_DEPTH, K4A_CALIBRATION_TYPE_COLOR, &position ); if( !result ){ continue; } const int32_t id = body.id; const cv::Point point( static_cast<int32_t>( position.xy.x ), static_cast<int32_t>( position.xy.y ) ); cv::circle( skeleton, point, 5, colors[id % colors.size()], -1 ); } } - Get Image that used for Inference - Convert Joint Position (3D) to 2D for Draw Circle on Image
- 18. Azure Kinect Body Tracking SDK Programming // Show Image cv::imshow( "body index", body_index ); cv::imshow( "skeleton", skeleton ); // Wait Key const int32_t key = cv::waitKey( 30 ); if( key == 'q' ){ break; } // Close Tracker tracker.destroy(); // Clear Handle body_index_map_image.reset(); skeleton_image.reset(); body_capture.reset(); body_frame.reset();