# Finding Where's Waldo using Mask R-CNN

DevFeed: [Finding Where's Waldo using Mask R-CNN](<https://devfeed.tech/articles/finding-where-s-waldo-using-mask-r-cnn-21536.md>)

Original publisher: [Read original article](<http://lifepluslinux.blogspot.com/2018/06/finding-wheres-waldo-using-mask-r-cnn.html>)

Author: Suresh Alse (noreply@blogger.com)

Published: 2018-06-14T05:51:00Z

Content type: tutorial

Language: en

Sources: [Life Plus Linux](<https://devfeed.tech/sources/life-plus-linux.md>)

Topics: [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Algorithm](<https://devfeed.tech/topics/algorithm.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [data](<https://devfeed.tech/topics/data.md>), [Code](<https://devfeed.tech/topics/code.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [code](<https://devfeed.tech/tags/code.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [deeplearning](<https://devfeed.tech/tags/deeplearning.md>), [github](<https://devfeed.tech/tags/github.md>), [ml](<https://devfeed.tech/tags/ml.md>), [paper](<https://devfeed.tech/tags/paper.md>), [quality](<https://devfeed.tech/tags/quality.md>), [training](<https://devfeed.tech/tags/training.md>), [validation](<https://devfeed.tech/tags/validation.md>)

## AI overview

This tutorial explains how to use Mask R-CNN to detect and mask Where's Waldo in images. It covers collecting and annotating a small dataset, training the model, and running predictions. The author reports that performance was better on similar, high-quality images where Waldo was clearly visible.

## Source excerpt

When I was a kid, I really loved solving Where's Waldo. There were few books (it used to be called Where's Wally) in our school library on which I spent hours finding Waldo. For people who do not know what it is, basically Waldo - a unique character is hidden among hundreds of other characters and you have to find him in all the chaos in the image. Now that I am too old to be solving it and too busy to spend hours on such things, I decided to build a system that uses deep learning to automatically solve it and spent weeks to build it. I started off by treating this like a classification problem with two classes - Waldo and not Waldo, similar to Hot dog - not Hot dog . Once we can get the classification problem successfully solved, we can just apply a classification action mapping (CAM) layer to find Waldo's activations in the image and thus finding Waldo. However I couldn't find enough images of Waldo. I found this repo which has about 20 images. And as there are only 20 Waldo vs thousands of not-Waldo characters, there is very high imbalance in the classes. I still tried though. But the results weren't that great. When I looked if someone has already worked on it, I found a medium post which used Tensorflow's Faster R-CNN model to do this. But I didn't want to just find bounding boxes, I wanted to actually mask out Waldo in the image. But I got more images of Where's Waldo from it. Then I came across this paper on Mask R-CNN which sounded promising for this usecase. And it was indeed much better than my earlier approach: Waldo masked out in the image Original Image In this post I would like to share how I was able to get the data, tag it and train a model to be able to solve Where's Waldo. You can checkout my code on github here. Fork deepwaldo on Github Mask R-CNN The main idea here is to: Take the input image and pass it into a set of convolutional layers that sort of generates a feature map for the given image. Now, you take this feature map and pass it into a r