Face Recognition with Micropython (K210 chip boards: Sipeed Maix series, M5StickV, HuskyLens)

Hello and welcome to my channel, Hardware.ai!
Today’s video is going to be a quick review of face recognition model and firmware for K210 chip, developed and published by Sipeed. I already made the videos about image recognition and object detection models for this chip. Now, face recognition neural network model architecture is quite different from these two and I spent quite some time studying it from the information available on the internet. So, let me gently break the facial recognition voodoo for you.

Essentially, for facial recognition pipeline we’re using 3 different models - first one to detect the face, second one to detect the landmarks (eyes, nose, mouth, etc) and then align the face and finally the third one to calculate the face feature vector, which is a “simplified” representation of person’s unique face features. It is basically a long line of numbers, corresponding to that person’s face. If we have two feature vectors, we can calculate the euclidean distance between them and if it sufficiently small, we conclude this is in fact the same person! Really, it’s that simple. Let’s have a look at the Micropython code now.

After usual boilerplate stuff (lcd init, camera init, blah, blah) we have an infinite loop where we a) detect a face, crop and resize it to b) input to landmark detection network, which gives us the coordinates of face landmarks, which we use to “straighten” the face by applying affine transform, then we c)calculate the face feature vector with the third network and compare it with other vectors in the memory to find if the face in camera now is sufficiently resembles the face feature vectors stored in memory.

The models are available to download at maixhub.com, link in the description. They are free for personal use, but business users need to pay license fee. The models and face encoding/face feature vector comparison code are not open source. Despite I myself don’t really like that, but it is understandable. Sipeed is a small Chinese company and they want to earn some money by licensing as well.
If you are applying it for personal use, that doesn’t matter for you much. If you have a commercial project where you would like to use face recognition with K210, then there are two ways to do it - buy license from Sipeed and use their model and API, possible even with MF1 OEM module (this one supports IR camera and liveliness detection and is generally more geared towards business applications) OR train the model and write the low-level C code yourself. Out of three models, the face detection model is freely available, it is fairly easy to train and use landmark detection model. The most difficult part is face feature vector encoding model and face feature vector comparison C code. My educated guess is that they use Mobilefacenet model for encoding face features, link to Github in the description. If you are going in that direction and think you have enough technical skills, good luck!

That’s about it for today’s video, hope you enjoyed it and found the information useful for your projects. Do you have any interesting ideas of face recognition projects with microcontrollers? If yes, please do share in the comments! Until the next time!

The board for this video was kindly provided by Seeed studio, check out this board and other hardware for machine learning acceleration at Seeed studio store!
https://www.seeedstudio.com/Sipeed-MAix-BiT-for-RISC-V-AI-IoT-1-p-2873.html

Implementations of Mobilefacenet(I don't know why I said "Facemobilenet" xD):
https://github.com/JaydenGG/Mobilefacenet Tensorflow
https://github.com/JaydenGG/Mobilefacenet Keras

Download the models at:
https://www.maixhub.com/

Add me on LinkedIn if you have any questions:
https://www.linkedin.com/in/dmitry-maslov-ai/

Credits for music:

Dreams - A Synthwave Mix (Juno Dreams - Paradise)