Watching Neural Networks Learn

Some notes: - A lot of you have pointed out that (tanh(x)+1)/2 == sigmoid(2x). I didn't realize this, so the improvement I was seeing may have been a fluke, I'll have to test it more thoroughly. It is definitely true that UNnormalized tanh outperforms sigmoid. - There are apparently lots of applications of the fourier series in real-world neural nets, many have mentioned NERF and Transformers.

Having a PhD on Neural Networks, I can vouch that this video is a gem and needs more views. Great work.

This is BY FAR the most understandable AI ... that I have ever seen. This is amazing!! Cannot overstate how beautifully this is executed

I've actually done something quite similar – I had the network learn a representation of a 3D scene using a signed distance function. In this context, I found that using a Leaky ReLU gives the models a pseudo-polygonal appearance, while tanh creates smoother models but is somewhat less effective in terms of learning efficiency. Interestingly, the Mish function seems to strike a balance between these two approaches, producing smooth models while maintaining nearly the same learning efficiency as the Leaky ReLU.

When a neural network video feels like watching an Oscar-winning documentary

The tone, the background soothing music, the images, you made something so complicated so easy to digest. Great job. I know you are brilliant!

This is amazing! I’ve been learning the fundamentals over the last few weeks and this is the best video I’ve seen so far. I’m not a math expert by any means, but I actually understood almost everything you said! Thank you so much.

I am currently studying PhD in Applied Mathematics and my research focuses on Mathematical Finance and Machine Learning. This is the best video that explains what artificial neural networks are. This is well executed! Thank you for this.

I've been interested in this field for years but 30 minutes of this explained to me what I couldn't fully understand for years now. 🎉 THANK YOU!

By far the best SoME3 video I’ve seen so far. Great intuitive explanation and stunning visuals.

This video was absolutely amazing. I had some hypotheses about the Fourier Transform being the key to understanding patterns in multi-dimentsional data, but this video beautifully tied all those hypotheses together for me. Absolute hats off. Thank you and hope to see more of this kind of content.

This is an amazing explanation. I'm actually a visual artist and have been deep into image generation for the past year. At this point I have a good basic knowledge and strong intuitive understanding of machine learning and training (I'm familiar with things like Fourier transforms, gradient descent, and overfitting), but this really validated and clarified a lot of those concepts. Many thanks for taking the time to create such an elegant video.

Great Video! This video actually made me cry seeing sorta more viscerally how functions are stitched into EVERYTHING, makes you think that maybe we are a lot like the mandlebrot, the universe recursively calculating itself. Thank you for this video!

Subscribed. Please keep making this type of content. Simple, easily understandable and has pictures.

This video is amazing. The ideas, the animation, the examples, even the voice and narration style. Excellent in every detail.

Wow these videos are INSANELY well made and well explained. You're awesome!

Best ever video on NN with higher level viz. This gave me a vibe of watching Interstellar movie when comparing NN with higher-level math. Also, Kudos to the video editor😄

Wow, this is exceptional. As a semi-retired mechanical engineer studying on my own to better understand neural networks and AI, this is incredibly interesting and educational. Bravo on your excellent presentation on difficult topics. I really enjoy getting the nitty-gritty math behind it all. Subscribed. Thanks and cheers.

Amazing video! Btw, I'd really recommend you to check the original NeRF (Neural Radiance Field) paper. That's a good practical example of using Fourier NNs to represent 4D data

Really incredible video! It is really interesting to see why we use different networks- thank you for making this!