Page 31 -

In contrast, if you find that 50% of the mistakes are dogs, then you can be more confident
that the proposed project will have a big impact. It could boost accuracy from 90% to 95% (a
50% relative reduction in error, from 10% down to 5%).

This simple counting procedure of error analysis gives you a quick way to estimate the
possible value of incorporating the 3rd party software for dog images. It provides a
quantitative basis on which to decide whether to make this investment.

Error analysis can often help you figure out how promising different directions are. I’ve seen
many engineers reluctant to carry out error analysis. It often feels more exciting to just jump
in and implement some idea, rather than question if the idea is worth the time investment.
This is a common mistake: It might result in your team spending a month only to realize
afterward that it resulted in little benefit.

Manually examining 100 examples does not take long. Even if you take one minute per
image, you’d be done in under two hours. These two hours could save you a month of wasted
effort.

Error Analysis​ refers to the process of examining dev set examples that your algorithm
misclassified, so that you can understand the underlying causes of the errors. This can help
you prioritize projects—as in this example—and inspire new directions, which we will discuss

next. The next few chapters will also present best practices for carrying out error analyses.

Page 31 Machine Learning Yearning-Draft Andrew Ng