Difference between revisions of "DSC Capstone2020"
From Immersive Visualization Lab Wiki
(→Schedule) |
|||
| (74 intermediate revisions by one user not shown) | |||
| Line 1: | Line 1: | ||
| − | =DSC | + | =DSC 180A Capstone Section A01: Explainable AI= |
* Instructor: [https://jacobsschool.ucsd.edu/faculty/profile?id=360 Jurgen Schulze] | * Instructor: [https://jacobsschool.ucsd.edu/faculty/profile?id=360 Jurgen Schulze] | ||
* Discussion: Wednesdays 12-12:50pm on Zoom at https://ucsd.zoom.us/j/9100475160 | * Discussion: Wednesdays 12-12:50pm on Zoom at https://ucsd.zoom.us/j/9100475160 | ||
| + | * Office hours: Fridays 10-11am on Zoom at https://ucsd.zoom.us/j/97761107672 | ||
* Piazza Discussion Board: https://piazza.com/ucsd/fall2020/dsc180 | * Piazza Discussion Board: https://piazza.com/ucsd/fall2020/dsc180 | ||
| Line 9: | Line 10: | ||
In this capstone domain we are going to study how we can make machine learning systems more user friendly by exploiting additional knowledge we can derive from the system and present it to the user. These types of systems are called Explainable AI. | In this capstone domain we are going to study how we can make machine learning systems more user friendly by exploiting additional knowledge we can derive from the system and present it to the user. These types of systems are called Explainable AI. | ||
| − | The example we are going to use in this class is object recognition in images. We are first going to | + | The example we are going to use in this class is object recognition in images. We are first going to learn about saliency and attention maps, then get to know a large publicly available image data set (COCO) and finally we are going to implement the [http://gradcam.cloudcv.org Grad-CAM algorithm] in [https://pytorch.org PyTorch] and apply it to the [https://cocodataset.org COCO image data set]. |
The quarter will end with a proposal for your capstone project, which you will be working on in the winter quarter. | The quarter will end with a proposal for your capstone project, which you will be working on in the winter quarter. | ||
This class will be entirely remote. | This class will be entirely remote. | ||
| + | |||
| + | You can use the [https://dsc-capstone.github.io/resources/computing/ DSMLP cluster], which already has the COCO data set installed. | ||
==Schedule== | ==Schedule== | ||
| Line 20: | Line 23: | ||
|- | |- | ||
! Week | ! Week | ||
| − | ! Date | + | ! Discussion Date |
| − | ! Discussion | + | ! Discussion Wed 12-12:50pm (Link to Slides) |
| − | ! | + | ! Participation Tasks (Due Tuesdays at noon) |
| + | ! Checkpoints | ||
|- | |- | ||
| 1 | | 1 | ||
| Oct 7 | | Oct 7 | ||
| − | | Overview | + | | [[Media:01_CourseOverviewF20.pdf|Overview]] |
| + | | None for week 1 | ||
| | | | ||
|- | |- | ||
| 2 | | 2 | ||
| Oct 14 | | Oct 14 | ||
| − | | | + | | [[Media:02_SaliencyMapsF20.pdf|Saliency Maps]] |
| + | | [[DSC180F20W2|Literature review]] (due Oct 13) | ||
| | | | ||
|- | |- | ||
| 3 | | 3 | ||
| Oct 21 | | Oct 21 | ||
| − | | | + | | [[Media:03_CocoDatasetF20.pdf|COCO Image Data Set]] |
| − | | | + | | [[DSC180F20W3|COCO Dataset]] (due Oct 20) |
| + | | | ||
|- | |- | ||
| 4 | | 4 | ||
| Oct 28 | | Oct 28 | ||
| − | | | + | | [[Media:04_Grad-CAMF20.pdf|Introduction to Grad-CAM]] |
| − | | Checkpoint #1 due | + | | [[DSC180F20W4|Custom COCO Demo]] (due Oct 27) |
| + | | [[DSC180F20W4|Replication Checkpoint #1]] (due Oct 30) | ||
|- | |- | ||
| 5 | | 5 | ||
| Nov 4 | | Nov 4 | ||
| − | | | + | | [[Media:05_DeepLearningF20.pdf|Deep Learning]] |
| − | | | + | | [[DSC180F20W5|Introduction to Grad-CAM]] (due Nov 3) |
| + | | | ||
|- | |- | ||
| 6 | | 6 | ||
| Nov 11 | | Nov 11 | ||
| Veterans Day (No Discussion) | | Veterans Day (No Discussion) | ||
| + | | [[DSC180F20W6|Deep Learning]] (due Nov 10) | ||
| | | | ||
|- | |- | ||
| 7 | | 7 | ||
| Nov 18 | | Nov 18 | ||
| − | | | + | | [[Media:06_GradCAM_in_ContextF20.pdf|Grad-CAM in Context]] |
| − | | Checkpoint #2 due | + | | [[DSC180F20W7P|PyTorch]] (due Nov 17) |
| + | | [[DSC180F20W7|Replication Checkpoint #2]] (due Nov 22) | ||
|- | |- | ||
| 8 | | 8 | ||
| Nov 25 | | Nov 25 | ||
| − | | | + | | [[Media:07_GradCAM_ForXAI_F20.pdf|Grad-CAM for XAI]] |
| + | | [[DSC180F20W8P|Grad-CAM in Context]] (due Nov 24) | ||
| | | | ||
|- | |- | ||
| 9 | | 9 | ||
| Dec 2 | | Dec 2 | ||
| − | | | + | | [[Media:08_Grad-CAM_F20.pdf|Grad-CAM and Q2 Project]] |
| + | | [[DSC180F20W9P|Grad-CAM for XAI]] (due Dec 1) | ||
| | | | ||
|- | |- | ||
| 10 | | 10 | ||
| Dec 9 | | Dec 9 | ||
| − | | | + | | [[Media:09_ElevatorPitches_F20.pdf|Q2 Project Elevator Pitches]] |
| − | | Final Report due | + | | [[DSC180F20W10P|Grad-CAM Evolution]] (due Dec 8) |
| + | | [[DSC180F20W10|Final Replication Report]] (due Dec 11) | ||
|} | |} | ||
| − | == | + | ==Relevant Publications== |
| + | * Paper for replication: [https://arxiv.org/pdf/1610.02391v1.pdf Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization] | ||
| + | * Paper on COCO image dataset: [https://arxiv.org/abs/1405.0312 Microsoft COCO: Common Objects in Context] | ||
| + | * Original attention map paper: [https://arxiv.org/pdf/1312.6034.pdf Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps] | ||
* [https://arxiv.org/pdf/1512.04150.pdf Learning Deep Features for Discriminative Localization] | * [https://arxiv.org/pdf/1512.04150.pdf Learning Deep Features for Discriminative Localization] | ||
| − | * [https://arxiv.org/pdf/ | + | * [https://arxiv.org/pdf/1311.2901.pdf Visualizing CNNs with deconvolution] |
| + | * [http://faculty.neu.edu.cn/yury/AAI/Textbook/Deep%20Learning%20with%20Python.pdf Deep Learning with Python by François Chollet] | ||
==Useful Links== | ==Useful Links== | ||
| Line 86: | Line 104: | ||
* [http://playground.tensorflow.org Tensorflow Playground] | * [http://playground.tensorflow.org Tensorflow Playground] | ||
* [https://github.com/jacobgil/pytorch-grad-cam Grad-CAM implementation in Pytorch] | * [https://github.com/jacobgil/pytorch-grad-cam Grad-CAM implementation in Pytorch] | ||
| + | * [https://cocodataset.org COCO Image Dataset] | ||
| + | * http://cs-people.bu.edu/jmzhang/excitationbp.html | ||
| + | |||
| + | ==Direct CNN Visualization== | ||
| + | |||
| + | * https://github.com/conan7882/CNN-Visualization | ||
| + | * https://medium.com/@awjuliani/visualizing-neural-network-layer-activation-tensorflow-tutorial-d45f8bf7bbc4 | ||
| + | * drawNet: http://people.csail.mit.edu/torralba/research/drawCNN/drawNet.html | ||
| + | * https://towardsdatascience.com/how-to-visualize-convolutional-features-in-40-lines-of-code-70b7d87b0030 | ||
| + | * https://towardsdatascience.com/understanding-your-convolution-network-with-visualizations-a4883441533b | ||
| + | * http://cs231n.github.io/understanding-cnn/ | ||
Latest revision as of 09:50, 6 January 2021
Contents |
DSC 180A Capstone Section A01: Explainable AI
- Instructor: Jurgen Schulze
- Discussion: Wednesdays 12-12:50pm on Zoom at https://ucsd.zoom.us/j/9100475160
- Office hours: Fridays 10-11am on Zoom at https://ucsd.zoom.us/j/97761107672
- Piazza Discussion Board: https://piazza.com/ucsd/fall2020/dsc180
Overview
In this capstone domain we are going to study how we can make machine learning systems more user friendly by exploiting additional knowledge we can derive from the system and present it to the user. These types of systems are called Explainable AI.
The example we are going to use in this class is object recognition in images. We are first going to learn about saliency and attention maps, then get to know a large publicly available image data set (COCO) and finally we are going to implement the Grad-CAM algorithm in PyTorch and apply it to the COCO image data set.
The quarter will end with a proposal for your capstone project, which you will be working on in the winter quarter.
This class will be entirely remote.
You can use the DSMLP cluster, which already has the COCO data set installed.
Schedule
| Week | Discussion Date | Discussion Wed 12-12:50pm (Link to Slides) | Participation Tasks (Due Tuesdays at noon) | Checkpoints |
|---|---|---|---|---|
| 1 | Oct 7 | Overview | None for week 1 | |
| 2 | Oct 14 | Saliency Maps | Literature review (due Oct 13) | |
| 3 | Oct 21 | COCO Image Data Set | COCO Dataset (due Oct 20) | |
| 4 | Oct 28 | Introduction to Grad-CAM | Custom COCO Demo (due Oct 27) | Replication Checkpoint #1 (due Oct 30) |
| 5 | Nov 4 | Deep Learning | Introduction to Grad-CAM (due Nov 3) | |
| 6 | Nov 11 | Veterans Day (No Discussion) | Deep Learning (due Nov 10) | |
| 7 | Nov 18 | Grad-CAM in Context | PyTorch (due Nov 17) | Replication Checkpoint #2 (due Nov 22) |
| 8 | Nov 25 | Grad-CAM for XAI | Grad-CAM in Context (due Nov 24) | |
| 9 | Dec 2 | Grad-CAM and Q2 Project | Grad-CAM for XAI (due Dec 1) | |
| 10 | Dec 9 | Q2 Project Elevator Pitches | Grad-CAM Evolution (due Dec 8) | Final Replication Report (due Dec 11) |
Relevant Publications
- Paper for replication: Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization
- Paper on COCO image dataset: Microsoft COCO: Common Objects in Context
- Original attention map paper: Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
- Learning Deep Features for Discriminative Localization
- Visualizing CNNs with deconvolution
- Deep Learning with Python by François Chollet
Useful Links
- Main DSC Capstone Website
- Introduction to how CNNs Work
- Tensorflow Playground
- Grad-CAM implementation in Pytorch
- COCO Image Dataset
- http://cs-people.bu.edu/jmzhang/excitationbp.html
Direct CNN Visualization
- https://github.com/conan7882/CNN-Visualization
- https://medium.com/@awjuliani/visualizing-neural-network-layer-activation-tensorflow-tutorial-d45f8bf7bbc4
- drawNet: http://people.csail.mit.edu/torralba/research/drawCNN/drawNet.html
- https://towardsdatascience.com/how-to-visualize-convolutional-features-in-40-lines-of-code-70b7d87b0030
- https://towardsdatascience.com/understanding-your-convolution-network-with-visualizations-a4883441533b
- http://cs231n.github.io/understanding-cnn/
