Improving Deep Neural Networks – Ng’s 2nd Course [Nov. 2017]

For my review of the first course, see this post.

In that post I was saying that I’m flirting with the thought of paying my way through the specialization ($49 a months) after the 7-day free trial. And that’s exactly what I did.

Then I enrolled in the second course of this 5-course Deep Learning specialization and it took about 5-6 days to finish it, at a semi-relaxed pace. My final grade was 100%.

After these first two courses, I sort-of have a strategy which seems to be convenient for me. I try to watch all the lectures of a specific week in one or two sittings. This helps me achieve a good overall perspective of the material and it’s also very helpful when I do the practice quizzes. Now, a few words about the course…

Week 1 materials were really dense as we went through a couple of important concepts in detail. First, Professor Ng talked about good ways to choosing the size of the train/test/dev sets and then we talked about bias and variance and the bias-variance tradeoff.

In the second part of week one we dealt with regularization, dropout and other regularization methods, while in the third part Professor Ng discussed about normalizing inputs, weight initialization, gradient checking and also the problem of vanishing and exploding gradients.

Likewise, there were three smaller coding assignments relating to each one of these topics and I enjoyed going through them, despite getting stuck here and there.

In week 2 we learned about mini-batch gradient descent versus gradient descent, when it’s good to choose one over the other and the typical size of the mini-batch for ML/DL projects (power of 2: 64, 128, 512, 1024… samples per mini-batch).

Very important in week 2 were the lectures about optimization algorithms: Exponentially Weighted Averages, GD with momentum, RMSprop, and Adam. We had to implement them in the coding assignment for week 2.

Week 3 lectures were multi-topical as well. Professor Ng talked about hyperparameter tuning. I specifically liked the Pandas vs. Caviar analogy, and when to choose between ‘babysitting’ one algorithm for your DL projects versus training multiple models in parallel.

We also learned about batch normalization and multi-class classification. What I enjoyed most was the lecture on TensorFlow and the coding assignment which took us from introductory concepts of TensorFlow to implementing our own neural network in TF for multi-class classification – recognizing numbers from 0 to 5 in sign language.

I took about 40-45 pages of notes while watching the lectures and, like I said in my earlier posts, I find this an extremely helpful practice to strengthen the concepts learned throughout the materials; notes that I can always revisit at a later time and notes that are helpful while doing the coding assignments.

Now, I’m looking forward to begin working my way through the third course of the specialization, which is about structuring machine learning projects.

Get on my list of friends
More about my book Stress and Adaptation
More about my book Persistent Fat Loss
More about my book Ketone Power
More about my book Periodic Fasting

Related posts:



Leave a Reply

Your email address will not be published. Required fields are marked *