The objective of this course is to provide a comprehensive introduction to deep neural networks, which have consistently demonstrated superior performance across diverse domains, notably in processing and generating images, text, and speech.

The course focuses both on theory spanning from the basics to the latest advances, as well as on practical implementations in Python and PyTorch (students implement and train deep neural networks performing image classification, image segmentation, object detection, part of speech tagging, lemmatization, speech recognition, reading comprehension, and image generation). Basic Python skills are required, but no previous knowledge of artificial neural networks is needed; basic machine learning understanding is advantageous.

Students work either individually or in small teams on weekly assignments, including competition tasks, where the goal is to obtain the highest performance in the class.

The course covers the following techniques and tasks:
• Feedforward deep neural networks (basic architectures and activation functions; optimization algorithms)
• Regularization of deep models (L2, dropout, label smoothing, batch normalization)
• Convolutional neural networks (image classification, image segmentation, object detection, fine-tuning pre-trained models)
• Recurrent neural networks (LSTM, GRU, seq2seq)
• Transformer architecture
• Natural language processing (distributed and contextualized word representations, BERT, morphological tagging, named entity recognition, lemmatization, machine translation)
• Deep generative models (variational autoencoders, generative adversarial networks, diffusion models, image and speech generation)
• Structured prediction (CTC and speech recognition, seq2seq)
• Introduction to deep reinforcement learning