Ray Sin Learning notes

How to improve Classification and Detection Accuracy?

• Straightforward approach
  • Increase the depth(Add layers) : the number of network levels
    • Increases the number of parameters
    • Makes the network more prone to overfitting
  • Increase the width(Larger Network) : the number of units at each level
    • Increase use of computational resources
    • ex
      1. If two convolutional layers are chained, any uniform increase in the number of their filters results in a quadratic increase of computation
      2. If weights are close to zero, we're wasted compute resources

GoogLeNet Solution : Sparsity

Replacing the fully connected layers by sparse ones, even inside the convolutions

Theoretical breakthroughs : Provable bounds for learning some deep representations, ICML, 2013

• Hebbian Principle : Neurons that fire together, wire together

In Summary

• 22 layers
• Stacked 9 inception modules
• Inception Modules:
  • Use small size filters : 1x1, 3x3, 5x5
  • Decrease dimensions whenever computaion requirements increase via 1x1 Dimension reduction layer
• Counter-Balance Back-Propagation downsides in Deep Network by using intermediate losses in the final loss
• No Fully connected layers is used

reference:http://openaccess.thecvf.com/content_cvpr_2015/papers/Szegedy_Going_Deeper_With_2015_CVPR_paper.pdf