Using OpenCV to detect face key points with C++

This post is a follow-up on my first post about building a face detector with OpenCV in C++. In this post we will build on the existing code and detect face key points. The result will look like this.

Detected face keypoints

Since we will work with a relatively new version of OpenCV (4.2.0), you might want to go back to the previous post to read more on how to install the necessary packages.

The code is on my github.

Let’s get going!

Detecting face key points

After detecting faces in the last post, we now want to detect face key points. We use the cv::face::FacemarkLBF model to find key points in the face rectangles we identified in the last tutorial.

Adding the key point detection model file

As for the face detection model, we have to add a model file for the LBF model. I put the model file in the assets folder of this posts git repo, so you can go there and download it.

For passing the location of this model file to our code, we will use the same CMake target_compile_definitions trick we used in the first post. So make sure you have the model file in the right place and add the following lines to your CMakeLists.txt.

# Introduce preprocessor variables to keep paths of asset files

A class for the key point detector

We start by adding a class for the key point detector. This way we have the code for initializing and calling the model in one place.


In the include folder, we create a file KeyPointDetector.h. This will be the header file for the key point detector.

The KeyPointDetector will have two public methods. The first is a constructor. We will use the constructor to initialize the underlying LBF model.

The second method detect_key_points detects face key points within a given rectangle in an image. As the key points for each face are of type std::vector<cv::Point2f>, this function will return a vector of std::vector<cv::Point2f>.

The header file for the key point detector will look like this.


#include <opencv4/opencv2/face.hpp>

class KeyPointDetector {
    /// Constructor
    explicit KeyPointDetector();

    /// Detect face key points within a rectangle inside an image
    /// \param face_rectangles Rectangles that contain faces
    /// \param image Image in which we want to detect key points
    /// \return List of face keypoints for each face rectangle
    detect_key_points(const std::vector<cv::Rect> &face_rectangles,
                      const cv::Mat &image) const;

    cv::Ptr<cv::face::Facemark> facemark_;



Next, we implement those methods in src/KeyPointDetector.cpp.

First, let’s look at the constructor. We create a new cv::face::FacemarkLBF model. Then we load the model configuration from the KEY_POINT_DETECTION_MODEL variable we passed in via CMake.

KeyPointDetector::KeyPointDetector() {
    facemark_ = cv::face::FacemarkLBF::create();

Following, we implement detect_key_points.

To adhere to the API of cv::face::Facemark::fit(), we transform our input to a cv::InputArray. Then we call the models fit function and return the detected points.

        const std::vector<cv::Rect> &face_rectangles, 
        const cv::Mat &image) const 

    cv::InputArray faces_as_input_array(face_rectangles);
    std::vector<std::vector<cv::Point2f> > key_points;

    return key_points;

Using the key point detector

Now we jump to our main.cpp to use the key point detector we defined. We use the face detector from the previous post. Then we feed the detected rectangles to our key point detector.

#include <opencv4/opencv2/opencv.hpp>
#include "FaceDetector.h"
#include "KeyPointDetector.h"

int main(int argc, char **argv) {

    cv::VideoCapture video_capture;
    if (! {
        return 0;

    FaceDetector face_detector;
    KeyPointDetector keypoint_detector;

    cv::Mat frame;
    while (true) {
        video_capture >> frame;
        auto rectangles = face_detector

        auto keypoint_faces = keypoint_detector
                .detect_key_points(rectangles, frame);

Instead of displaying the rectangles, we display the detected points.

        const auto red = cv::Scalar(0, 0, 255);
        for (const auto &face :keypoint_faces) {
            for (const cv::Point2f &keypoint : face) {
                cv::circle(frame, keypoint,
                           8, red, -1);

        imshow("Image", frame);
        const int esc_keycode = 27;
        if (cv::waitKey(10) == esc_keycode) {
    return 0;

You should see a result similar to the image below.

Detected face keypoints


In this post we used a face detection model to find faces in an image. Then we found key points in those images using OpenCV.

I hope this helps you to build interesting stuff! Here is a link to the code. Let me know if you run into any errors!

Follow me on twitter (@bewagner_) for more content on C++ and machine learning!

Related Posts

The state of DevOps

Analyzing developer sentiment towards DevOps based on the Stack Overflow 2020 Developer Survey

C++ dependency management with CMake's FetchContent

How to replace git submodules with a built-in CMake feature



Building a face detector with OpenCV in C++

How to detect faces in an image with OpenCV