ELLIS Talk by David Tax

With the increasing computing power, the availability of large (image) datasets and the development of machine learning frameworks, anybody with a bit of programming skills can nowadays train complicated neural networks. Unfortunately, not every practitioner is aware of the challenges and pitfalls that are involved in that. Furthermore, it does not seem fashionable to read (older) literature, with the result that mistakes are repeated over and over again. In this talk I would like to discuss a small, but important detail in the training of neural networks: their weight initialisation. I'll discuss some standard approaches, how they are not solving what they are supposed to solve, and I will propose a slight improvement which should allow a reasonable initialisation for even very deep networks.

This will be a hybrid meeting:

• physical: 28 Building Van Mourik Broekmanweg 6, 2.W510 Hilbert
• zoom:       https://tudelft.zoom.us/j/95277941196?pwd=cjZCMlhFQU5tamxkYlNzcHN2Z1A1dz09
                   (Meeting ID: 952 7794 1196, Passcode: 302159)

This meeting is open for all interested researchers.