IDS 576 Advanced Predictive Models and Applications for Business Analytics

Edition: Spring 2018

Document version: Apr 26 2018

Overview

The goal of this class is to cover advanced machine learning techniques not covered in IDS 572 and IDS 575. Broadly, we will cover topics spanning graphical models and deep learning. Graphical models are a set of very useful techniques for inferring outcomes and making predictions conditional on preceding events, even when we do not have full information. They have found success in tracking, speech recognition, language modeling (Hidden Markov Models), image segmentation (Markov Random Fields) and many other applications. Similarly, we will study the basics of deep learning architectures, their design choices and how they are trained using gradient methods. We will also study recurrent and convolutional architectures which achieve state of the art in challenging prediction tasks in computer vision and text applications. Time permitting, we will also look at online and reinforcement learning problems.

Logistics

• Lectures: Wednesdays 6.00 PM to 8.30 PM at Douglas Hall 220
• Optional Recitations: Select Wednesdays 5.00 PM - 5.50 PM at F004-2LCF (Check Slack)
• Staff
  • Instructor: Dr. Theja Tulabandhula
  • Teaching Assistant: Mehrnaz Amjadi
• Online communication: Slack
• Offline communication:
  • Instructor Office Hours: Wednesdays 8.30 PM - 9.30 PM (DH220/UH2404) or schedule by Slack
  • TA Office Hours: Tuesdays 2.30 PM - 4.00 PM (UH2401) or schedule by Slack

Textbook and Materials

• There is no official textbook for this course. Several references will be provided within each lecture. For additional reading please refer to these textbooks:
  • Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville (2016).
  • Bayesian Reasoning and Machine Learning by David Barber (2012).
  • Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto (2018).
• Materials (lecture notes, assignments and project details) will be posted on Blackboard.

Software

• Python and Jupyter notebooks will be used. Refer to this set of notes or this to get started in Python.

Timeline

Lectures (tentative)

1. 01/17 : Motivating Applications, Machine Learning Pipeline (Data, Models, Loss, Optimization), Backpropagation
2. 01/24 : Feedforward Networks: Nonlinearities, Convolutional Neural Networks: Convolution, Pooling
3. 01/31 : Jumpstarting Convolutional Neural Networks: Visualization, Transfer, Practical Models (VGG, AlexNet) (video)
4. 02/07 : Text and Embeddings: Introduction to NLP, Word Embeddings, Word2Vec (video)
5. 02/14 : Recurrent Neural Networks: Sequence to Sequence Learning, RNNs and LSTMs (video)
6. 02/21 : Unsupervised Deep Learning: Generative Adversarial Networks, Variational Autoencoders (video)
7. 02/28 : Review of Deep Learning and Recent Advances (video)
8. 03/14 : Graphical Models: Representation: Directed and Undirected Graphical Models, Conditional Independence, D-separation, Local Markov Property (video)
9. 03/21 : Graphical Models: Inference: Variable Elimination, Belief Propagation, Markov Chain Monte Carlo (video)
10. 04/04 : Graphical Models: Learning: Maximum Likelihood Estimation, Expectation Maximization (video)
11. 04/11 : Online Learning: A/B Testing, Multi-armed Bandits, Contextual Bandits (video)
12. 04/18 : Reinforcement Learning: Policies, State-Action Value Functions, Q-Learning (video)
13. 04/25 : Deep Reinforcement Learning: Function Approximation, DQN for Atari Games, AlphaGo Zero (video)
14. 05/02 : Project Presentations

(A concatenated set of slides are available here)

Assignments

1. 01/31: Assignment 1 (lengthy!) out. Due on 02/27
2. 02/28: Assignment 2 out. Due on 03/13
3. 03/21: Assignment 3 out. Due on 04/10
4. 04/11: Assignment 4 out. Due on 04/24

Exams

1. 03/07: Exam I (same venue as lectures, and during class hours)
2. 05/09: Exam II (same venue as lectures, and during class hours)

Project

1. 03/20: Project Report I due
2. 05/01: Project Report II due

Note: Submission deadline for assignments and project reports is BEFORE 2359hrs on the concerned day. Use Blackboard for uploads.

Grades

• Assignments (4): 10% + 5% + 5% + 5%
• Exams (2): 20% (Exam I) + 20% (Exam II)
• Project (2): 10% (Report I) + 25% (Report II)

Assignments

• Always mention sources in your assignment solutions and project writeups.
• Late submissions will have an automatic 20% penalty per day.

Exams

• These are closed book, but one 8.5x11-inch handwritten cheatsheet is allowed.
• No computers and communication devices are allowed.

Project

• This involves working on and documenting a machine learning solution on a dataset of your choice (e.g., reimplementing and verifying the results of any research paper appearing in recent machine learning and data mining conferences). See details on Blackboard.

Miscellaneous Information

• This is a 4 credit graduate level course with CRN 38063, offered by the Information and Decision Sciences department at UIC.
• The semester runs from Jan 16, 2018 - May 04, 2018 (academic calendar).
• Students who wish to observe their religious holidays (http://oae.uic.edu/religious-calendar/) shoud notify the instructor by Jan 20.
• Please contact the instructor at the earliest, if you require accommodations for access to and/or participation in this course.
• Please refer to the academic integrity guidelines set by the university.

Back to the instructor's homepage.