Skip to content

Course Description

Organization: University of Minnesota Department of Electrical and Computer Engineering (ECE)
Course Identifier: EE 4940
Sequence Name: Special Topics in Electrical and Computer Engineering (1-4 Cr.)
Offering Course Title: : Introduction to Deep Learning
Offering Date: Spring 2025

Course Instructor, TA and Format

Sang-Hyun Oh, Professor of ECE, graduate faculty in neuroscience, physics, biomedical eng. Email: sang@umn.edu
Office: 5-119 Keller Hall
Teaching Assistant: Ike Goetzman (PhD candidate in ECE; email: goetz219@umn.edu)
Format: Lecture (Jan 22 – May 5, 2024): Monday/Wednesday 11:15am-12:30pm @ Appleby Hall 3

Course Summary

Machine learning, especially deep learning based on neural networks, has shown transformative impacts across life sciences, engineering, and medicine, as recently highlighted by the 2024 Nobel Prizes:

  • The Physics Nobel was awarded to John Hopfield and Geoffrey Hinton for pioneering contributions to neural networks that enable machine learning
  • Demis Hassabis, John Jumper, and David Baker received the Chemistry Nobel for advancing computational protein design.

Course Overview

EE 4940: Introduction to Deep Learning aims to provide students with both fundamental and applied perspectives on these advances. Beginning with an overview of AI, the course introduces deep neural networks (DNNs) through practical, hands-on examples and guides students through specialized neural network architectures, including

  • convolutional neural networks (CNNs) for diagnostic applications and
  • recurrent neural networks (RNNs) for time series analysis.

Emphasis is placed on applying neural networks to tackle real-world problems in classification, regression, image analysis, and time-series prediction. Scientific applications are demonstrated as well, such as using DNNs to solve equations.

The course concludes with dimensionality reduction techniques, followed by advanced topics such as generative AI, and an introduction to reinforcement learning.

Emphasis will be on practical experience with Python-based coding over mathematical theory.

By the end of the course, students will have gained familiarity with Python-driven deep learning, hands-on skills, and a working knowledge of deep learning’s potential for a wide range of problems.

Course Prerequisites

EE 4940 - Special Topics in Electrical and Computer Engineering is recommended but not required.

This course is designed to be accessible and introductory, with essential concepts and coding skills introduced as needed.

Expected Workload

This course (EE 4940) is a 3-credit, lecture format course. The expected workload for this course this semester is, on average, 45 hours per credit, or just over 3 hours per credit hour per week (including lecture). The total workload of 9 hours per week.

Course Goals and Objectives

EE 4940 aims to equip students with a foundation in deep learning, focusing on core architectures like deep neural networks (DNNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs).

Through hands-on applications, students will learn to apply these models to solve practical problems using real-world datasets, such as cancer diagnostics and time-series analysis. The primary goal of the course is to develop students's ability to implement and troubleshoot neural network models, gaining some practical experiences, and be aware of potential pitfalls/artifacts while using DNNs.

By the course's end, students will have a working knowledge of AI’s role in scientific problem-solving, dimensionality reduction techniques, and a simple introduction to generative models and reinforcement learning (RL).

Topics To Be Covered

1. Introduction to AI and deep learning

  • Brief history of AI, neural networks and recent Nobel-winning breakthroughs in physics and chemistry
  • Course setup, tools, and brief introduction to Python for deep learning
  • The universality theorem and the manifold hypothesis

2. Fundamentals of Deep Neural Networks (DNNs)

  • Architecture of DNNs: layers, activation functions, and backpropagation
  • Training, validation, and testing sets;
  • Loss functions, gradient descent, mini-batches
  • Hands-on coding

3. Convolutional Neural Networks (CNNs) - applications in diagnostic imaging

  • CNN architecture
  • Applying CNNs, with MHIST as a case study for cancer diagnosis

4. Recurrent Neural Networks (RNNs) and time series analysis

  • Introduction to RNNs, LSTMs, for sequency data analysis.
  • Hands-on practice with real-world data.

5. Chips and AI

Briefly explore AI chips, neuromorphic devices, and future directions.

6. AI for scientific problem solving: Using neural networks to approximate solutions for

Maxwell’s equations

7. Dimensionality reduction and autoencoders

8. Advanced topics

Course Format

  • Classroom lectures only
  • Canvas LMS link will be provided
  • Textbook and Required Materials: none
  • Based on lecture notes (uploaded in canvas 1-2 days before class)

Homework Assignments and “Mini-projects”:

  • Regularly-scheduled weekly or biweekly homework assignments and 4 multi-week “mini-projects.” The former will be designed to exercise basic concepts and modest extensions of the technical or computational tools introduced in class; the latter will be crafted to allow students to take a “deeper dive” by applying techniques developed in the course to data sets arising in various aspects of ECE and life sciences.
  • You are allowed (in fact, encouraged) to work with others when solving the homework and mini project problems. However, solutions submitted for grading should be formulated by each student independently! Late homework and mini-project submissions will be accepted, but without a legitimate (e.g., illness-based) justification, will result in a maximum grade of 80% for that assignment. Specifically, unjustified late assignments will be graded according to the same rubric as on-time assignments, but the corresponding score will be multiplied by 0.8 when entered into the gradebook.

Exams

There will be one midterm exam and one final exam. The midterm exam will be open book/notes. The final exam will be open book/notes and will be due no earlier than the final exam time specified by the Registrar. Either or both may be take-home exams.

Grading

The grade distributions for the course variations are provided below.

Note: distributions for homework assignments and mini-projects are total; individual submissions will be equally weighted.

Deliverable Weight
Homework Assignments 40%
Mini-projects 40%
Midterm Exam 10%
Final Exam 10%

Software Requirements

Through the homework assignments and mini-projects, students will be required to perform a number of computer exercises to become familiar with the computational and practical aspects of deep learning. This will typically entail utilization of Python, often in concert with existing toolboxes such as NumPy, SciPy, Matplotlib, Scikit-learn, and PyTorch (or TensorFlow), all of which are freely available for Windows, Mac, and Linux operating systems.

As a perhaps preferred alternative, students may wish to perform all (Python) programming computations using a free cloud-based account (i.e., Google Colab), which will be discussed and established at the beginning of the semester. Large-scale deep learning computations necessary for the final “mini-project” will utilize these resources.

Note

The Required University Policy Statements were provided in the original PDF of the course description. They have been removed to allow the generative AI tools to focus on the concept enumeration.