Detecting ArUco markers with OpenCV and Python

In this tutorial you will learn how to detect ArUco markers in images and real-time video streams using OpenCV and Python.

This blog post is part two in our three-part series on ArUco markers and fiducials:

1. Generating ArUco markers with OpenCV and Python (last week’s post)
2. Detecting ArUco markers in images and video with OpenCV (today’s tutorial)
3. Automatically determining ArUco marker type with OpenCV (next week’s post)

Last week we learned:

• What an ArUco dictionary is
• How to select an ArUco dictionary appropriate to our task
• How to generate ArUco markers using OpenCV
• How to create ArUco markers using online tools

Today we’re going to learn how to actually detect ArUco markers using OpenCV.

To learn how to detect ArUco markers in images and real-time video with OpenCV, just keep reading.

Detecting ArUco markers with OpenCV and Python

From there we’ll review our project directory structure and implement two Python scripts:

1. One Python script to detect ArUco markers in images
2. And another Python script to detect ArUco markers in real-time video streams

We’ll wrap up this tutorial on ArUco marker detection using OpenCV with a discussion of our results.

OpenCV ArUCo marker detection

As I discussed in last week’s tutorial, the OpenCV library comes with built-in ArUco support, both for generating ArUco markers and for detecting them.

Understanding the “cv2.aruco.detectMarkers” function

We can define an ArUco marker detection procedure in, essentially, only 3-4 lines of code:

arucoDict = cv2.aruco.Dictionary_get(cv2.aruco.DICT_6X6_50)
arucoParams = cv2.aruco.DetectorParameters_create()
(corners, ids, rejected) = cv2.aruco.detectMarkers(image, arucoDict,
	parameters=arucoParams)

Configuring your development environment

In order to generate and detect ArUco markers, you need to have the OpenCV library installed.

Luckily, OpenCV is pip-installable:

$ pip install opencv-contrib-python

If you need help configuring your development environment for OpenCV 4.3+, I highly recommend that you read my pip install opencv guide — it will have you up and running in a matter of minutes.

Having problems configuring your development environment?

All that said, are you:

• Short on time?
• Learning on your employer’s administratively locked system?
• Wanting to skip the hassle of fighting with the command line, package managers, and virtual environments?
• Ready to run the code right now on your Windows, macOS, or Linux system?

Then join PyImageSearch Plus today!

Gain access to Jupyter Notebooks for this tutorial and other PyImageSearch guides that are pre-configured to run on Google Colab’s ecosystem right in your web browser! No installation required.

And best of all, these Jupyter Notebooks will run on Windows, macOS, and Linux!

Project structure

Before we can learn how to detect ArUco tags in images, let’s first review our project directory structure so you have a good idea on how our project is organized and what Python scripts we’ll be using.

Start by using the “Downloads” section of this tutorial to download the source code and example images.

From there, we can inspect the project directory:

$ tree . --dirsfirst
.
├── images
│   ├── example_01.png
│   └── example_02.png
├── detect_aruco_image.py
└── detect_aruco_video.py

2 directories, 9 files

With our project directory structure reviewed, we can move on to implementing ArUco tag detection with OpenCV!

Detecting ArUco markers with OpenCV in images

Ready to learn how to detect ArUco tags in images using OpenCV?

Open up the detect_aruco_image.py file in your project directory, and let’s get to work:

# import the necessary packages
import argparse
import imutils
import cv2
import sys

# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--image", required=True,
	help="path to input image containing ArUCo tag")
ap.add_argument("-t", "--type", type=str,
	default="DICT_ARUCO_ORIGINAL",
	help="type of ArUCo tag to detect")
args = vars(ap.parse_args())

Therefore, you must make absolutely certain that the type used to generate the ArUco tags is the same type you are using for the detection phase.

Note: Don’t know what ArUco dictionary was used to generate the tags in your input images? Don’t worry, I’ve got you covered. Next week I’ll be showing you one of the Python scripts in my personal arsenal that I break out when I can’t identify what type a given ArUco tag is. This script automatically identifies the ArUco tag type. Stay tuned for next week’s tutorial, where I’ll review it in detail.

Next up comes our ARUCO_DICT, which enumerates each of the ArUco tag types that OpenCV supports:

# define names of each possible ArUco tag OpenCV supports
ARUCO_DICT = {
	"DICT_4X4_50": cv2.aruco.DICT_4X4_50,
	"DICT_4X4_100": cv2.aruco.DICT_4X4_100,
	"DICT_4X4_250": cv2.aruco.DICT_4X4_250,
	"DICT_4X4_1000": cv2.aruco.DICT_4X4_1000,
	"DICT_5X5_50": cv2.aruco.DICT_5X5_50,
	"DICT_5X5_100": cv2.aruco.DICT_5X5_100,
	"DICT_5X5_250": cv2.aruco.DICT_5X5_250,
	"DICT_5X5_1000": cv2.aruco.DICT_5X5_1000,
	"DICT_6X6_50": cv2.aruco.DICT_6X6_50,
	"DICT_6X6_100": cv2.aruco.DICT_6X6_100,
	"DICT_6X6_250": cv2.aruco.DICT_6X6_250,
	"DICT_6X6_1000": cv2.aruco.DICT_6X6_1000,
	"DICT_7X7_50": cv2.aruco.DICT_7X7_50,
	"DICT_7X7_100": cv2.aruco.DICT_7X7_100,
	"DICT_7X7_250": cv2.aruco.DICT_7X7_250,
	"DICT_7X7_1000": cv2.aruco.DICT_7X7_1000,
	"DICT_ARUCO_ORIGINAL": cv2.aruco.DICT_ARUCO_ORIGINAL,
	"DICT_APRILTAG_16h5": cv2.aruco.DICT_APRILTAG_16h5,
	"DICT_APRILTAG_25h9": cv2.aruco.DICT_APRILTAG_25h9,
	"DICT_APRILTAG_36h10": cv2.aruco.DICT_APRILTAG_36h10,
	"DICT_APRILTAG_36h11": cv2.aruco.DICT_APRILTAG_36h11
}

The key to this dictionary is a human-readable string (i.e., the name of the ArUco tag type). The key then maps to the value, which is OpenCV’s unique identifier for the ArUco tag type.

Using this dictionary we can take our input --type command line argument, pass it through ARUCO_DICT, and then obtain the unique identifier for the ArUco tag type.

The following Python shell block shows you a simple example of how this lookup operation is performed:

>>> print(args)
{'type': 'DICT_5X5_100'}
>>> arucoType = ARUCO_DICT[args["type"]]
>>> print(arucoType)
5
>>> 5 == cv2.aruco.DICT_5X5_100
True
>>> 

I covered the types of ArUco dictionaries, including their name conventions in my previous tutorial, Generating ArUco markers with OpenCV and Python.

If you would like more information on ArUco dictionaries, please refer there; otherwise, simply understand that this dictionary lists out all possible ArUco tags that OpenCV can detect.

Next, let’s move on to loading our input image from disk:

# load the input image from disk and resize it
print("[INFO] loading image...")
image = cv2.imread(args["image"])
image = imutils.resize(image, width=600)

# verify that the supplied ArUCo tag exists and is supported by
# OpenCV
if ARUCO_DICT.get(args["type"], None) is None:
	print("[INFO] ArUCo tag of '{}' is not supported".format(
		args["type"]))
	sys.exit(0)

# load the ArUCo dictionary, grab the ArUCo parameters, and detect
# the markers
print("[INFO] detecting '{}' tags...".format(args["type"]))
arucoDict = cv2.aruco.Dictionary_get(ARUCO_DICT[args["type"]])
arucoParams = cv2.aruco.DetectorParameters_create()
(corners, ids, rejected) = cv2.aruco.detectMarkers(image, arucoDict,
	parameters=arucoParams)

# verify *at least* one ArUco marker was detected
if len(corners) > 0:
	# flatten the ArUco IDs list
	ids = ids.flatten()

	# loop over the detected ArUCo corners
	for (markerCorner, markerID) in zip(corners, ids):
		# extract the marker corners (which are always returned in
		# top-left, top-right, bottom-right, and bottom-left order)
		corners = markerCorner.reshape((4, 2))
		(topLeft, topRight, bottomRight, bottomLeft) = corners

		# convert each of the (x, y)-coordinate pairs to integers
		topRight = (int(topRight[0]), int(topRight[1]))
		bottomRight = (int(bottomRight[0]), int(bottomRight[1]))
		bottomLeft = (int(bottomLeft[0]), int(bottomLeft[1]))
		topLeft = (int(topLeft[0]), int(topLeft[1]))

		# draw the bounding box of the ArUCo detection
		cv2.line(image, topLeft, topRight, (0, 255, 0), 2)
		cv2.line(image, topRight, bottomRight, (0, 255, 0), 2)
		cv2.line(image, bottomRight, bottomLeft, (0, 255, 0), 2)
		cv2.line(image, bottomLeft, topLeft, (0, 255, 0), 2)

		# compute and draw the center (x, y)-coordinates of the ArUco
		# marker
		cX = int((topLeft[0] + bottomRight[0]) / 2.0)
		cY = int((topLeft[1] + bottomRight[1]) / 2.0)
		cv2.circle(image, (cX, cY), 4, (0, 0, 255), -1)

		# draw the ArUco marker ID on the image
		cv2.putText(image, str(markerID),
			(topLeft[0], topLeft[1] - 15), cv2.FONT_HERSHEY_SIMPLEX,
			0.5, (0, 255, 0), 2)
		print("[INFO] ArUco marker ID: {}".format(markerID))

		# show the output image
		cv2.imshow("Image", image)
		cv2.waitKey(0)

The final output visualization is displayed to our screen on Lines 98 and 99.

OpenCV ArUco marker detection results

Let’s put our OpenCV ArUco detector to work!

Use the “Downloads” section of this tutorial to download the source code and example images.

From there, you can execute the following command:

$ python detect_aruco_image.py --image images/example_01.png --type DICT_5X5_100
[INFO] loading image...
[INFO] detecting 'DICT_5X5_100' tags...
[INFO] ArUco marker ID: 42
[INFO] ArUco marker ID: 24
[INFO] ArUco marker ID: 70
[INFO] ArUco marker ID: 66
[INFO] ArUco marker ID: 87

This image contains the ArUco markers that we generated in last week’s blog post. I took each of the five individual ArUco markers and constructed a montage of them in a single image.

As Figure 3 shows, we’ve been able to correctly detect each of the ArUco markers and extract their IDs.

Let’s try a different image, this one containing ArUco markers not generated by us:

$ python detect_aruco_image.py --image images/example_02.png --type DICT_ARUCO_ORIGINAL
[INFO] loading image...
[INFO] detecting 'DICT_ARUCO_ORIGINAL' tags...
[INFO] ArUco marker ID: 241
[INFO] ArUco marker ID: 1007
[INFO] ArUco marker ID: 1001
[INFO] ArUco marker ID: 923

Figure 4 displays the results of our OpenCV ArUco detector. As you can see, I have detected each of the four ArUco markers on my Pantone color matching card (which we’ll be using in a number of upcoming tutorials, so get used to seeing it).

Looking at the command line arguments to the above script, you may be wondering:

“Hey Adrian, how did you know to use DICT_ARUCO_ORIGINAL and not some other ArUco dictionary.”

The short answer is that I didn’t … at least, not initially.

I actually have a “secret weapon” up my sleeve. I’ve put together a Python script that can automatically infer ArUco marker type, even if I don’t know what type of marker is in an image.

I’ll be sharing that script with you next week, so be on the lookout for it.

Detecting ArUco markers in real-time video streams with OpenCV

In our previous section we learned how to detect ArUco markers in images …

… but is it possible to detect ArUco markers in real-time video streams?

The answer is yes, it absolutely is — and I’ll be showing you how to do so in this section.

Open up the detect_aruco_video.py file in your project directory structure, and let’s get to work:

# import the necessary packages
from imutils.video import VideoStream
import argparse
import imutils
import time
import cv2
import sys

# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-t", "--type", type=str,
	default="DICT_ARUCO_ORIGINAL",
	help="type of ArUCo tag to detect")
args = vars(ap.parse_args())

# define names of each possible ArUco tag OpenCV supports
ARUCO_DICT = {
	"DICT_4X4_50": cv2.aruco.DICT_4X4_50,
	"DICT_4X4_100": cv2.aruco.DICT_4X4_100,
	"DICT_4X4_250": cv2.aruco.DICT_4X4_250,
	"DICT_4X4_1000": cv2.aruco.DICT_4X4_1000,
	"DICT_5X5_50": cv2.aruco.DICT_5X5_50,
	"DICT_5X5_100": cv2.aruco.DICT_5X5_100,
	"DICT_5X5_250": cv2.aruco.DICT_5X5_250,
	"DICT_5X5_1000": cv2.aruco.DICT_5X5_1000,
	"DICT_6X6_50": cv2.aruco.DICT_6X6_50,
	"DICT_6X6_100": cv2.aruco.DICT_6X6_100,
	"DICT_6X6_250": cv2.aruco.DICT_6X6_250,
	"DICT_6X6_1000": cv2.aruco.DICT_6X6_1000,
	"DICT_7X7_50": cv2.aruco.DICT_7X7_50,
	"DICT_7X7_100": cv2.aruco.DICT_7X7_100,
	"DICT_7X7_250": cv2.aruco.DICT_7X7_250,
	"DICT_7X7_1000": cv2.aruco.DICT_7X7_1000,
	"DICT_ARUCO_ORIGINAL": cv2.aruco.DICT_ARUCO_ORIGINAL,
	"DICT_APRILTAG_16h5": cv2.aruco.DICT_APRILTAG_16h5,
	"DICT_APRILTAG_25h9": cv2.aruco.DICT_APRILTAG_25h9,
	"DICT_APRILTAG_36h10": cv2.aruco.DICT_APRILTAG_36h10,
	"DICT_APRILTAG_36h11": cv2.aruco.DICT_APRILTAG_36h11
}

Refer to the “Detecting ArUco markers with OpenCV in images” section above for a more detailed review of this code block.

We can now load our ArUco dictionary:

# verify that the supplied ArUCo tag exists and is supported by
# OpenCV
if ARUCO_DICT.get(args["type"], None) is None:
	print("[INFO] ArUCo tag of '{}' is not supported".format(
		args["type"]))
	sys.exit(0)

# load the ArUCo dictionary and grab the ArUCo parameters
print("[INFO] detecting '{}' tags...".format(args["type"]))
arucoDict = cv2.aruco.Dictionary_get(ARUCO_DICT[args["type"]])
arucoParams = cv2.aruco.DetectorParameters_create()

# initialize the video stream and allow the camera sensor to warm up
print("[INFO] starting video stream...")
vs = VideoStream(src=0).start()
time.sleep(2.0)

# loop over the frames from the video stream
while True:
	# grab the frame from the threaded video stream and resize it
	# to have a maximum width of 1000 pixels
	frame = vs.read()
	frame = imutils.resize(frame, width=1000)

	# detect ArUco markers in the input frame
	(corners, ids, rejected) = cv2.aruco.detectMarkers(frame,
		arucoDict, parameters=arucoParams)

	# verify *at least* one ArUco marker was detected
	if len(corners) > 0:
		# flatten the ArUco IDs list
		ids = ids.flatten()

		# loop over the detected ArUCo corners
		for (markerCorner, markerID) in zip(corners, ids):
			# extract the marker corners (which are always returned
			# in top-left, top-right, bottom-right, and bottom-left
			# order)
			corners = markerCorner.reshape((4, 2))
			(topLeft, topRight, bottomRight, bottomLeft) = corners

			# convert each of the (x, y)-coordinate pairs to integers
			topRight = (int(topRight[0]), int(topRight[1]))
			bottomRight = (int(bottomRight[0]), int(bottomRight[1]))
			bottomLeft = (int(bottomLeft[0]), int(bottomLeft[1]))
			topLeft = (int(topLeft[0]), int(topLeft[1]))

			# draw the bounding box of the ArUCo detection
			cv2.line(frame, topLeft, topRight, (0, 255, 0), 2)
			cv2.line(frame, topRight, bottomRight, (0, 255, 0), 2)
			cv2.line(frame, bottomRight, bottomLeft, (0, 255, 0), 2)
			cv2.line(frame, bottomLeft, topLeft, (0, 255, 0), 2)

			# compute and draw the center (x, y)-coordinates of the
			# ArUco marker
			cX = int((topLeft[0] + bottomRight[0]) / 2.0)
			cY = int((topLeft[1] + bottomRight[1]) / 2.0)
			cv2.circle(frame, (cX, cY), 4, (0, 0, 255), -1)

			# draw the ArUco marker ID on the frame
			cv2.putText(frame, str(markerID),
				(topLeft[0], topLeft[1] - 15),
				cv2.FONT_HERSHEY_SIMPLEX,
				0.5, (0, 255, 0), 2)

	# show the output frame
	cv2.imshow("Frame", frame)
	key = cv2.waitKey(1) & 0xFF

	# if the `q` key was pressed, break from the loop
	if key == ord("q"):
		break

# do a bit of cleanup
cv2.destroyAllWindows()
vs.stop()

OpenCV ArUco video detection results

Ready to apply ArUco detection to real-time video streams?

Start by using the “Downloads” section of this tutorial to download the source code and example images.

From there, pop open a shell, and execute the following command:

$ python detect_aruco_video.py

As you can see, I’m easily able to detect the ArUco markers in real-time video.

What’s next?

If you’re new to OpenCV and computer vision or want to dig deeper into the fundamentals, I highly recommend you check out the PyImageSearch Gurus course.

To be honest, I wish I’d had access to a course like this in college.

I learned computer vision the hard way, wading through theory and math-heavy textbooks, complex research papers, and the occasional sit-down in my adviser’s office.

It took a while, but by the end of it, I was confident that I knew computer vision well enough to consult for the NIH and build/deploy a couple of iPhone apps to the App Store.

Thankfully, you don’t have to come up with your own examples and projects to learn from.

Inside the PyImageSearch Gurus course, you’ll learn computer vision systematically using practical application and Python code examples. You’ll also have the advantage of learning alongside other developers, researchers, and students just like you who are eager to learn computer vision, level-up their skills, and collaborate on projects.

Enroll in PyImageSearch Gurus and get:

• Highly actionable content on Computer Vision, Deep Learning, and OpenCV with real-world examples.
• The same hands-on, easy to understand teaching style that you expect from PyImageSearch.
• Access to community forums where you can get expert advice from advanced students. I’m in there nearly every day too, answering your questions.

PyImageSearch Gurus is the most comprehensive computer vision course available online today. I guarantee you won’t find a more detailed course online.

Take a look at these success stories to see just what’s possible. If these students can do it, so can you. You just need to take the first step and enroll.

If you’re on the fence, click this link to grab the course syllabus and 10 free sample lessons.

Summary

In this tutorial you learned how to detect ArUco markers in images and real-time video streams using OpenCV and Python.

Detecting ArUco markers with OpenCV is a three-step process:

1. Set what ArUco dictionary you are using.
2. Define the parameters to the ArUco detector (typically the default options suffice).
3. Apply the ArUco detector with OpenCV’s cv2.aruco.detectMarkers function.

OpenCV’s ArUco marker is extremely fast and, as our results showed, is capable of detecting ArUco markers in real-time.

Feel free to use this code as a starting point when using ArUco markers in your own computer vision pipelines.

However, let’s say you are developing a computer vision project to automatically detect ArUco markers in images, but you don’t know what marker type is being used, and therefore, you can’t explicitly set the ArUco marker dictionary — what do you do then?

How are you going to detect ArUco markers if you don’t know what marker type is being used?

I’ll be answering that exact question in next week’s blog post.

To download the source code to this post (and be notified when future tutorials are published here on PyImageSearch), simply enter your email address in the form below!

The post Detecting ArUco markers with OpenCV and Python appeared first on PyImageSearch.