From the course: Deep Learning: Model Optimization and Tuning
Join today to access over 24,700 courses taught by industry experts.
Initializing weights
From the course: Deep Learning: Model Optimization and Tuning
Initializing weights
- [Instructor] While building a model, the weights and biases of all the nodes need to be initialized to some value, and then gradient descent will update them to get closer to better accuracy. The initial values of the weights play a huge role in the speed of learning and the final accuracy achieved. Multiple initialization techniques exist. In random normal initialization, random values are drawn from a standard normal distribution and used to initialize the weights. Zeros will initialize all the parameters to zeros, and ones will initialize them to the same value of one. Random uniform will draw values from a random uniform distribution. The difference between a normal and a uniform distribution is that in the case of normal, more values will be closer to the mean, while in uniform distribution, they are evenly spread out. Overall, random normal works best for most use cases, and is almost never changed unless a…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.
