Pose Landmark Detection

usePoseDetection

The usePoseDetection hook initializes and manages a pose detection instance using the MediaPipe library. It provides a convenient way to set up pose detection in a React Native application using the Vision Camera library.

Usage

import { usePoseDetection } from 'path/to/poseDetection';

const poseDetection = usePoseDetection(callbacks, runningMode, model, options);

Parameters

• callbacks: An object containing callback functions for handling detection results and errors.

  • onResults: A function that is called when pose detection results are available.
  • onError: A function that is called when an error occurs during pose detection.

• runningMode: The mode in which the pose detection should run. It can be one of the following:

  • RunningMode.IMAGE: For detecting poses in static images.
  • RunningMode.VIDEO: For detecting poses in video frames.
  • RunningMode.LIVE_STREAM: For detecting poses in a live camera stream.

• model: A string specifying the model to be used for pose detection.

• options (optional): An object containing additional configuration options for the pose detection. All properties are optional.

  • numPoses: The maximum number of poses to detect. Default is 1.
  • minPoseDetectionConfidence: The minimum confidence score for pose detection. Default is 0.5.
  • minPosePresenceConfidence: The minimum confidence score for pose presence. Default is 0.5.
  • minTrackingConfidence: The minimum confidence score for pose tracking. Default is 0.5.
  • shouldOutputSegmentationMasks: A boolean indicating whether to output segmentation masks. Default is false.
  • delegate: The delegate to be used for pose detection. It can be one of the following:
    • Delegate.GPU: Use GPU for pose detection.
    • Delegate.CPU: Use CPU for pose detection.
  • mirrorMode: The mirror mode for the camera. It can be one of the following:
    • "no-mirror": No mirroring.
    • "mirror": Mirror the camera output.
    • "mirror-front-only": Mirror the camera output only for the front camera. Default is "mirror-front-only".
  • forceOutputOrientation: The orientation to force for the output image. It can be one of the following:
    • "portrait"
    • "portrait-upside-down"
    • "landscape-left"
    • "landscape-right"
  • forceCameraOrientation: The orientation to force for the camera image. It can be one of the following:
    • "portrait"
    • "portrait-upside-down"
    • "landscape-left"
    • "landscape-right"
  • fpsMode: The frame rate mode for the camera. It can be "none" or a number specifying the target frame rate.

Returns

An object containing the following properties:

• cameraViewLayoutChangeHandler: A function to handle layout changes for the camera view.
• cameraDeviceChangeHandler: A function to handle changes in the camera device.
• cameraOrientationChangedHandler: A function to handle changes in the camera orientation.
• resizeModeChangeHandler: A function to handle changes in the resize mode.
• cameraViewDimensions: An object containing the dimensions of the camera view.
• frameProcessor: A function to process frames from the camera.

Example

import React from 'react';
import { usePoseDetection } from 'path/to/poseDetection';
import { Camera } from 'react-native-vision-camera';

const PoseDetectionComponent = () => {
  const callbacks = {
    onResults: (results) => {
      console.log('Pose detection results:', results);
    },
    onError: (error) => {
      console.error('Pose detection error:', error);
    },
  };

  const poseDetection = usePoseDetection(callbacks, 'LIVE_STREAM', 'pose_model', {
    // numPoses: 1,
    // minPoseDetectionConfidence: 0.5,
    // minPosePresenceConfidence: 0.5,
    // minTrackingConfidence: 0.5,
    // shouldOutputSegmentationMasks: false,
    // delegate: 'GPU',
    // mirrorMode: 'mirror-front-only',
    // forceOutputOrientation: 'portrait',
    // forceCameraOrientation: 'portrait',
    // fpsMode: 30,
  });

  return (
    <Camera
      style={{ flex: 1 }}
      device={poseDetection.cameraDevice}
      onLayout={poseDetection.cameraViewLayoutChangeHandler}
      frameProcessor={poseDetection.frameProcessor}
      frameProcessorFps={poseDetection.fpsMode}
    />
  );
};

export default PoseDetectionComponent;

This documentation provides a comprehensive overview of the usePoseDetection hook, including its parameters, return values, and an example of how to use it in a React Native component.