Face Landmark Detection

useFaceLandmarkDetection

The useFaceLandmarkDetection hook initializes and manages a Face Landmarking detection instance using the MediaPipe library. It provides a convenient way to set up face landmarking in a React Native application using the Vision Camera library.

For more details or to demo it, visit MediaPipe - Face Landmark Detection

Usage

import { useFaceLandmarkDetection } from 'path/to/faceLandmarkDetection';

const faceDetection = useFaceLandmarkDetection(callbacks, runningMode, model, options);

Parameters

• callbacks: An object containing callback functions for handling detection results and errors.

  • onResults: A function that is called when face landmarking results are available.
  • onError: A function that is called when an error occurs during face landmarking.

• runningMode: The mode in which the face landmarking should run. It can be one of the following:

  • RunningMode.IMAGE: For detecting face landmarks in static images.
  • RunningMode.VIDEO: For detecting face landmarks in video frames.
  • RunningMode.LIVE_STREAM: For detecting face landmarks in a live camera stream.

• model: A string specifying the model to be used for face landmarking.

• options (optional): An object containing additional configuration options for the face landmarking. All properties are optional.

  • numFaces: The maximum number of faces to detect. Default is 1.
  • minFaceDetectionConfidence: The minimum confidence score for face detection. Default is 0.5.
  • minFacePresenceConfidence: The minimum confidence score for face presence. Default is 0.5.
  • minTrackingConfidence: The minimum confidence score for tracking. Default is 0.5.
  • delegate: The delegate to be used for face landmarking. It can be one of the following:
    • Delegate.GPU: Use GPU for face landmarking.
    • Delegate.CPU: Use CPU for face landmarking.
  • mirrorMode: The mirror mode for the camera. It can be one of the following:
    • "no-mirror": No mirroring.
    • "mirror": Mirror the camera output.
    • "mirror-front-only": Mirror the camera output only for the front camera. Default is "mirror-front-only".
  • forceOutputOrientation: The orientation to force for the output image. It can be one of the following:
    • "portrait"
    • "portrait-upside-down"
    • "landscape-left"
    • "landscape-right"
  • forceCameraOrientation: The orientation to force for the camera image. It can be one of the following:
    • "portrait"
    • "portrait-upside-down"
    • "landscape-left"
    • "landscape-right"
  • fpsMode: The frame rate mode for the camera. It can be "none" or a number specifying the target frame rate.

Returns

An object containing the following properties:

• cameraViewLayoutChangeHandler: A function to handle layout changes for the camera view.
• cameraDeviceChangeHandler: A function to handle changes in the camera device.
• cameraOrientationChangedHandler: A function to handle changes in the camera orientation.
• resizeModeChangeHandler: A function to handle changes in the resize mode.
• cameraViewDimensions: An object containing the dimensions of the camera view.
• frameProcessor: A function to process frames from the camera.

Example

import React from 'react';
import { useFaceLandmarkDetection } from 'path/to/faceLandmarkDetection';
import { Camera } from 'react-native-vision-camera';

const FacelandmarkDetectionComponent = () => {
  const callbacks = {
    onResults: (results) => {
      console.log('Face Landmarking results:', results);
    },
    onError: (error) => {
      console.error('Face Landmarking error:', error);
    },
  };

  const faceLandmarkDetection = useFacelandmarkDetection(callbacks, 'LIVE_STREAM', 'face_landmarking.task', {
    // numFaces: 1,
    // minFaceDetectionConfidence: 0.5,
    // minFacePresenceConfidence: 0.5,
    // minTrackingConfidence: 0.5,
    // shouldOutputSegmentationMasks: false,
    // delegate: 'GPU',
    // mirrorMode: 'mirror-front-only',
    // forceOutputOrientation: 'portrait',
    // forceCameraOrientation: 'portrait',
    // fpsMode: 30,
  });

  return (
    <Camera
      style={{ flex: 1 }}
      device={faceLandmarkDetection.cameraDevice}
      onLayout={faceLandmarkDetection.cameraViewLayoutChangeHandler}
      frameProcessor={faceLandmarkDetection.frameProcessor}
      frameProcessorFps={faceLandmarkDetection.fpsMode}
    />
  );
};

export default FaceLandmarkDetectionComponent;

This documentation provides a comprehensive overview of the useFacelandmarkDetection hook, including its parameters, return values, and an example of how to use it in a React Native component.