OpenMV4 Plus uses Edge Impulse to self-train neural networks for face mask recognition.

Last April, our new product OpenMV4 H7 Plus was launched. Today, I will introduce to you the new feature of OpenMV4 H7 Plus—using the EdgeImpulse online website to independently train neural networks for classification and recognition.

In this tutorial, you will use machine learning to build a recognition system that can identify whether a person is wearing a face mask through the OpenMV4 H7 Plus smart camera, achieving image classification.

EdgeImpulse is a website that provides online training services for neural network models for embedded devices. It is our partner for OpenMV and also an official partner of STMicroelectronics. Currently, EdgeImpulse is free for our OpenMV users. OpenMV users can use EdgeImpulse to train neural network models suitable for OpenMV online for free.

Integrating neural networks into embedded devices like OpenMV enables functions such as distinguishing poachers from elephants, performing quality control on factory production lines, and enabling remote-controlled model cars to drive autonomously.

In this tutorial, you will learn how to collect images to build a high-quality dataset, how to apply transfer learning to train a neural network, and how to deploy the system to OpenMV.

Additionally, we have provided video tutorials which can be viewed by searching for "OpenMV4 Plus Training Neural Network for Mask Recognition" on the official website.

You can view the entire project content, including all code and models, at the following address: https://book.openmv.cc/project/mask.html

Using EdgeImpulse to train neural network models online for OpenMV mainly involves the following four steps: dataset collection, uploading, training, and deployment.

01、Collecting Data Set

In this tutorial, we will build a model that can distinguish whether a face is wearing a mask or not. Of course, you can also choose to classify other items. To enable the operation of the machine learning model, you need to collect a large number of example images of faces both with and without masks. During training, these example images are used for the model's differentiation practice.

We need to use OpenMV IDE to collect our dataset. The steps for capturing images are as follows:

Create two categories:

Open the "Tools" in the OpenMV IDE menu bar, select "Dataset Editor," click "New Dataset," then create a new folder and open it. Inside the dataset, create two more folders named "mask" (for saving photos of people wearing masks) and "face" (for saving photos of people without masks).

Collect images using OpenMV

First, we connect the OpenMV, click "Connect" in the IDE, then click "Run." In the Framebuffer, we can see the real-time image from the OpenMV. By clicking the photo button in the left menu bar, the OpenMV will automatically save this image, which becomes one data point in the dataset.

First, we need to save the face of a person wearing a mask. Click the photo button, and the IDE will capture an image below, automatically naming it 00000. The next one will be 00001, and so on.

Collect 200 photos of faces wearing masks (100 each for males and females) and 200 photos of faces not wearing masks (100 each for males and females) using OpenMV.

Note: Ensure that photos are captured from various angles to guarantee the diversity of our training and learning.

If during the collection process you find that a particular image is not captured perfectly, you can right-click on the image and select delete.

02. Upload

After capturing images with OpenMV, you need to register an account and log in to the EdgeImpulse official website (https://edgeimpulse.com/) to start uploading images. The steps for uploading images are as follows:

Create a new project on EdgeImpulse, click on "keys", select "API Key" and copy the "API Key". Use the API Key to establish connectivity between OpenMV IDE and Edge Impulse.

Select "Tools" from the top menu bar of OpenMV IDE, then choose "Data Editor," followed by "Export," "Upload," and "Upload via API Key." Copy the "API Key" and proceed with the upload.

During upload, the data will be automatically divided into training and test sets. We recommend using the default ratio of "80% and 20%." We have collected over 400 facial images in total. Here, 80% of the Training Date means that 80% of your data is used for training by default, while the remaining 20% is allocated for the test set.

03. Training Dataset

The dataset is ready, you can now train the dataset on the EdgeImpulse website interface.

Configuration Processing Module:

First, select "Impulse Design" and configure the processing module:

Set the default image dimensions to "96 x 96".

Select the "Images" module, indicating that we are conducting classification training on images (you can also use EdgeImpulse to classify sounds, videos, etc.);

Select "Transfer Learning (Images)", set the learning model;

Select "Save impulse", if it shows Successfully, then the configuration is successful.

Image Preprocessing:

Click "Image" in the left menu, set the color format to "RGB", then click "Save".

Next, select "Generate Features" to initiate the process. This will perform image preprocessing on over 400 data points, and a 3D visualization of the complete dataset will be displayed on the right side.

Configure the transfer learning model

Click on "Transfer learning" in the left menu, you can choose all default settings for the parameters, or adjust them according to your needs:

1. Set the default value for the Number of training cycles to 10

2. Set the learning rate to 0.0005

3. You may choose to tick or leave unticked the "Data augmentation" option.

4. Set the default Minimum confidence rating to 0.8.

Click "Start training", the training process takes about 4 to 5 minutes.

After the model training is completed, you can view the accuracy, confusion matrix, and expected device performance.

Testing Model:

After training is complete, we will use test data to evaluate the model.

Select "Model testing", tick the checkbox beside "Sample name", then click "Classify selected". The accuracy displayed here has reached 97%, which is quite remarkable for a model with such limited data.

The image displayed in red is uncertain. Click the three dots on the right side of this image, select "Show classification," and you will enter the "Real-time Classification" interface, which contains more details about the file. This interface will help you determine the reason for the image classification error.

If the data falls outside all known clusters (with mask/without mask), this may be data that does not match any previously seen classification—possibly due to hair covering most of the face.

04. Running the Model on OpenMV

After completing the steps of training design, model training, and model validation, you can export this model to your OpenMV. Select "OpenMV," choose "Build" to generate, and it will automatically create three files:

trained.tflite trained neural network model

Two classification labels (face and mask) in labels.txt

ei_image_classification.py is the code to be run on OpenMV

Connect OpenMV IDE, save these three files to the built-in Flash of OpenMV. Open the ei_image_classification.py file in OpenMV IDE to see the code we just generated in EdgeImpulse. Click run, and the running results will be displayed in the serial terminal.

Alright, you have successfully added the feature for OpenMV to train neural networks on its own.

We look forward to your results!