COMPSCI 682: Neural Networks - A Modern Introduction (Grade: A)

Course Overview:
COMPSCI 682 is a 3-credit course focused on modern and practical methods for deep learning. The course began with an introduction to simple classifiers such as perceptrons and logistic regression and progressively moved into more complex topics, including standard neural networks, convolutional neural networks, and elements of recurrent neural networks and transformers. While the course provided a strong emphasis on practical applications, it maintained a solid foundation in the fundamentals of deep learning.

The primary focus was on computer vision applications, but the course also explored aspects of natural language processing (NLP). The programming components were heavily based on Python and its associated packages, such as Numpy and PyTorch, making it essential for students to have a strong background in Linear Algebra, Probability and Statistics, and Multivariate Calculus, along with proficiency in Python.

Key Topics Covered

• Intro to Deep Learning and Historical Context:
  • The course began with a historical overview of deep learning, setting the stage for more complex topics. Early lectures focused on the basics of image classification, k-nearest neighbor, and linear classification.
• Loss Functions and Optimization:
  • A deep dive into loss functions and various optimization techniques was undertaken, including stochastic gradient descent (SGD) and backpropagation. The discussions were supplemented with practical examples and Python implementations.
• Training Neural Networks:
  • Several lectures were dedicated to the intricacies of training neural networks, covering topics such as weight initialization, batch normalization, and hyperparameter optimization. The importance of monitoring and fine-tuning the learning process was emphasized.
• Convolutional Neural Networks (ConvNets):
  • ConvNets were introduced with an exploration of their architecture, followed by discussions on advanced topics such as spatial localization, object detection, image segmentation, and visualizing ConvNets. This section included optional readings on ResNet and Fully Convolutional Networks (FCN).
• Neural Texture Synthesis and Style Transfer:
  • This topic focused on the creative applications of neural networks in generating and manipulating images, particularly through style transfer techniques.
• Generative Modeling and Recurrent Neural Networks (RNNs):
  • The course included guest lectures that provided insights into generative models, including autoregressive models, variational autoencoders (VAEs), and Generative Adversarial Networks (GANs). RNNs and their application in sequential data processing were also covered.
• Transformers and Multi-modal AI:
  • Towards the end of the course, modern architectures such as Transformers were introduced, followed by discussions on multi-modal AI and its applications.
• Neuro-Symbolic AI and Self-Supervised Learning:
  • The final lectures explored cutting-edge topics in AI, including neuro-symbolic AI and the emerging field of self-supervised learning.

Course Format

• Lecture Schedule:
  • In-person Lectures: Tuesdays and Thursdays, covering both fundamental and advanced topics in deep learning.
  • Optional Discussions: Fridays, focusing on specific topics like Python setup, batch normalization, and vector/matrix/tensor derivatives.
• Guest Lectures:
  • The course featured guest lectures from industry experts, offering insights into specialized topics such as scene reconstruction, generative modeling, and urban forest monitoring using AI.
• Programming and Assignments:
  • Assignments were based on Python, leveraging packages like Numpy and PyTorch. These assignments ranged from implementing basic algorithms to developing and fine-tuning complex neural networks.
• Midterm and Project:
  • A midterm exam tested the understanding of key concepts, and a final project provided an opportunity to apply the learned techniques to a real-world problem.

Personal Achievement

I completed this course with an A Grade (4.0/4.0), demonstrating a deep understanding of both the theoretical and practical aspects of neural networks and deep learning. The course challenged me to explore advanced topics, implement cutting-edge algorithms, and engage with the material on a profound level.

Final Thoughts

COMPSCI 682 was a transformative experience that provided me with a strong foundation in deep learning and its applications. The combination of rigorous coursework, insightful lectures, and hands-on assignments equipped me with the skills and knowledge needed to tackle complex problems in the field of machine learning.