Real Time Smile Detection in Webcam using Python 3

Introduction

Real time smile detection in webcam using Python 3 will show you how your working webcam detects your face and smile and draws a rectangle around your smile on your face. In my previous tutorial we have seen Real Time Eye Detection in Webcam using Python 3.

The similar tutorial we will use here to detect your face and smile and draw a rectangle around your smile to indicated your each smile.

In this tutorial also we will detect your face but we won’t draw any rectangle around it. If you want to draw a rectangle around your face also then uncomment the line that draws a rectangle around your face in the Python script.

Make sure you do have the camera installed in your system in order to see yourself in webcam.

Here in this example we will use OpenCv to capture the video and display into a frame. When you wish to close the video window you need to press either ESC key or ‘q’ from your keyboard.

Recommended reading: See yourself in webcam using Python 3

Real time face detection in webcam using Python 3

Prerequisites

Python 3.8.1, OpenCV 4.2.0, imutils 0.5.3

You need two XML file for detecting face and smile – haarcascade_frontalface_alt.xml and haarcascade_smile.xml, respectively. To download the file please go to the URL https://opencv.org/releases.html. Make sure you download the zip archive (Sources).

Example with Source Code

Preparing your workspace

Preparing your workspace is one of the first things that you can do to make sure that you start off well. The first step is to check your working directory.

When you are working in the Python terminal, you need first navigate to the directory, where your file is located and then start up Python, i.e., you have to make sure that your file is located in the directory where you want to work from.

In the below image you see I have opened a cmd prompt and navigated to the directory where I have to create Python script for implementing the example “real time smile detection in webcam using Python 3”.

Creating Python Script

Now we will create the Python script and see how to implement real time smile detection in webcam using Python 3.

In the below Python script we first import the required module OpenCv as cv2. Then we grab the reference to the webcam. Note that here I am using Laptop’s built-in webcam. Make sure your webcam works fine.

Notice we create the cascade and initialize it with our face cascade and smile cascade. This loads the face cascade and smile cascade into memory so it’s ready for use. Remember, the cascade is just XML files that contain the data to detect face and smiles.

To initialize you must download these XML files from the given link above. Once you download the zip file, extract and look for the desired files into opencv\data\haarcascades directory and copy the XML files under the C:\py_scripts where you have created below python script smile_detection.py.

If you do not copy those XML files into the desired directory then you may get below error while you try to execute the below python script:

faces = faceCascade.detectMultiScale(frame)
cv2.error: OpenCV(4.2.0) \opencv\modules\objdetect\src\cascadedetect.cpp:1698: error: (-215:Assertion failed) !empty() in function ‘cv::CascadeClassifier::detectMultiScale’

Then we grab the reference to the webcam. Next we read frame by frame the video captures.

The detectMultiScale function is a general function that detects objects. Since we are calling it on the face cascade, that’s what it detects.

The function returns a list of rectangles in which it believes it found a face. Next, we will loop over where it thinks it found something.

This function returns 4 values: the x and y location of the rectangle, and the rectangle’s width and height (w , h).

We use these values to draw a rectangle using the built-in rectangle() function.

In the similar way we define the region with respect to the face area and we draw a rectangle around each smile to indicate your smiles. First we need to detect face in order to detect smiles on your face otherwise we won’t be able to get the relative region or area for your smiles.

Finally we show it in frame. When you press either ESC key or “q” from keyboard, frame will disappears.

import cv2

faceCascade = cv2.CascadeClassifier('haarcascade_frontalface_alt.xml')
smileCascade= cv2.CascadeClassifier('haarcascade_smile.xml')

# grab the reference to the webcam
vs = cv2.VideoCapture(0)

# keep looping
while True:
	# grab the current frame
	ret, frame = vs.read()
  
	# if we are viewing a video and we did not grab a frame,
	# then we have reached the end of the video
	if frame is None:
		break
		
	gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
	gray = cv2.equalizeHist(gray)
		
	faces = faceCascade.detectMultiScale(frame, scaleFactor=1.05, minNeighbors=5, minSize=(45, 45))

	for (x,y,w,h) in faces:
		#cv2.rectangle(frame,(x,y),(x+w,y+h),(255,0,0),2)
		face_gray = gray[y:y+h, x:x+w]
		face_color = frame[y:y+h, x:x+w]
		smiles = smileCascade.detectMultiScale(face_gray, scaleFactor=1.7, minNeighbors=3, minSize=(15, 15))
		for (ex,ey,ew,eh) in smiles:
			cv2.rectangle(face_color,(ex,ey),(ex+ew,ey+eh),(0,255,0),1)
	
	# show the frame to our screen
	cv2.imshow("Video", frame)
	key = cv2.waitKey(1) & 0xFF

	# if the 'q' or ESC key is pressed, stop the loop
	if key == ord("q") or key == 27:
		break
 
# close all windows
cv2.destroyAllWindows()

Testing the application

Now it’s time to test our application by executing the following command in the cmd prompt:

C:\py_scripts>python <python script name>.py

Once the script gets executed successfully, you will be able to see yourself in the frame and a rectangle is drawn around smile as shown in below image: