Page 33 -

picture correctly will often inspire you to come up with new categories of errors and
solutions.

The most helpful error categories will be ones that you have an idea for improving. For
example, the Instagram category will be most helpful to add if you have an idea to “undo”
Instagram filters and recover the original image. But you don’t have to restrict yourself only
to error categories you know how to improve; the goal of this process is to build your

intuition about the most promising areas to focus on.

Error analysis is an iterative process. Don’t worry if you start off with no categories in mind.
After looking at a couple of images, you might come up with a few ideas for error categories.
After manually categorizing some images, you might think of new categories and re-examine
the images in light of the new categories, and so on.

Suppose you finish carrying out error analysis on 100 misclassified dev set examples and get
the following:

Image Dog Great cat Blurry Comments

1 ✔ Usual pitbull color

2 ✔
3 ✔ ✔ Lion; picture taken
at zoo on rainy day
4 ✔ Panther behind tree

… … … … ...

% of total 8% 43% 61%

You now know that working on a project to address the Dog mistakes can eliminate 8% of

the errors at most. Working on Great Cat or Blurry image errors could help eliminate more
errors. Therefore, you might pick one of the two latter categories to focus on. If your team
has enough people to pursue multiple directions in parallel, you can also ask some engineers
to work on Great Cats and others to work on Blurry images.

Error analysis does not produce a rigid mathematical formula that tells you what the highest

priority task should be. You also have to take into account how much progress you expect to
make on different categories and the amount of work needed to tackle each one.

Page 33 Machine Learning Yearning-Draft Andrew Ng