Page 51 -

25 Techniques for reducing avoidable bias

If your learning algorithm suffers from high avoidable bias, you might try the following
techniques:

• Increase the model size ​(such as number of neurons/layers): This technique reduces
bias, since it should allow you to fit the training set better. If you find that this increases

variance, then use regularization, which will usually eliminate the increase in variance.

• Modify input features based on insights from error analysis​: Say your error
analysis inspires you to create additional features that help the algorithm eliminate a
particular category of errors. (We discuss this further in the next chapter.) These new
features could help with both bias and variance. In theory, adding more features could
increase the variance; but if you find this to be the case, then use regularization, which will

usually eliminate the increase in variance.

• Reduce or eliminate regularization​ (L2 regularization, L1 regularization, dropout):
This will reduce avoidable bias, but increase variance.

• Modify model architecture​ (such as neural network architecture) so that it is more

suitable for your problem: This technique can affect both bias and variance.

One method that is not helpful:

• Add more training data​: This technique helps with variance problems, but it usually
has no significant effect on bias.

Page 51 Machine Learning Yearning-Draft Andrew Ng