Ng’s Neural Networks and Deep Learning – Some Thoughts [Nov. 2017]

I stopped taking courses on Coursera circa 2016 because I grew dispassionate about their payment models and because they stopped providing statements of accomplishment at the end of the course.

Ironically, the same payment model seems to have reignited my active interest in Coursera. About a few days ago I entered a 7-day free trial in the Deep Learning AI Specialization.

It took about 5 days to finish the first course Neural Networks and Deep Learning taught by Andrew Ng, adjunct professor at Stanford University and co-founder of Coursera. The course is 4 weeks long.

Given the level of complexity of the course, it would have taken me at least a month to get through the lectures, quizzes and programming projects, if there were no-strings attached. I take pride of the fact that I actively motivated myself to spend a couple of hours everyday working through the materials.

In the following lines I’m going to share a few thoughts after taking this first course of the specialization.

Thoughts about Neural Networks and Deep Learning

In week 1, I liked the fact that professor Ng took time to present the layout of the course and also to introduce the concepts that will be learned throughout. This is very helpful orientation for someone new to online learning. Week 1 materials and quiz were relatively introductory and easy, so this should give one enough fuel for things to come in later weeks.

Week 2 was about logistic regression as a neural network. Those familiar with machine learning will connect to the concepts. After a few basic notions, professor Ng starts getting into the mathematics of it, first by discussing the sigmoid activation function, the cost and loss functions, the inputs, the parameters and the output of logistic regression as a neural network.

Things began to precipitate when we got into gradient descent and went through each equation, function, and derivative, one step at a time. To take a short breath from maths, part 2 of week 2 was about vectorization, Jupyter notebooks, and how to implement logistic regression in Python.

While in week 1 I didn’t find it necessary, at this point I was taking notes, doing the equations and derivatives myself and slowly getting through each lecture.

There are two programming assignments in week 2: a practice one that has to do with introducing the student to Python and numpy. If you have no experience coding, this is going to be hard to grasp on the go. The second coding assignment had to do with implementing logistic regression as a simple neural network. I didn’t have too much trouble completing them.

Week 3 kept on getting more complex and difficult compared to prior weeks but I was already getting excited with the thought of maybe pushing through until the end. In week 3 we started getting deeper into neural nets: representation, output, the vectorized implementation and so on.

I was filling 15-20 pages of notes while doing the lectures of week 3. I found it necessary to go through, by hand, the equations, through the derivatives of activation functions, through gradient descent and through backpropagation. I felt it increased my comprehension of the materials and I also found that these notes are helpful for the quiz and coding assignment.

The assignment felt much harder and I got stuck a couple of times. I didn’t manage to finish it in one sitting but I eventually came through. The discussion forums are very helpful.

Week 4 topped it off. The topic was deep neural nets and it nicely built on the concepts learned in week 3. While previously we worked on smaller and more fixed neural network architectures, in week 4 the major challenge was to build an L-layer neural network, sort of like a generalized representation of deep neural networks. This type of representation is very helpful when you don’t have a set/specified net architecture. It provides flexibility and it allows you to scale your deep neural network accordingly.

Very important from the lectures, in my opinion is the discussion about parameters and hyperparamaters. I took a lot of notes during this week’s materials as well.

There were two coding assignments in week 4 (and a quiz). The first assignment was the hardest and probably the longest, while the second was an implementation of the functions we built in the first one. Basically, we had to build a deep neural network step by step, first with a simple architecture and then a L-layer neural network. Unsurprisingly, I got stuck a couple of times, but I came through. The forum discussions were very helpful as well.

Final Words

My final grade for this first course of the specialization is 96.5%.

Overall, I am very pleased and excited with professor Ng’s Deep Learning specialization and with Coursera (again!).

One of the potential not-so-strong points that I could say about this is with regards to the rigidity of the programming assignments. Part of the code is written down for you and you have to work your way through the rest. But, come to think of it, I can understand that if everyone would write their own code from scratch, it would be much harder for the staff to verify and grade the assignments. And, to be honest, I think it would be chaotic, very few would complete the course, and it would be counterproductive for Coursera. So, I guess I don’t have words of criticism…

My deepest appreciation is for how Professor Ng goes through all the hard mathematical concepts, as I think this exponentially improves a student’s understanding of the course.

Finally, the 7-day trial ends tomorrow and I’m seriously thinking of paying my way through ($49 a month) the rest of the courses, so that I can complete the entire Deep Learning specialization.

Get on my list of friends
More about my book Stress and Adaptation
More about my book Persistent Fat Loss
More about my book Ketone Power
More about my book Periodic Fasting

Related posts:



Leave a Reply

Your email address will not be published. Required fields are marked *