View source for DSC Capstone2020

=DSC 180 Capstone Section A01: Explainable AI=

* Instructor: [https://jacobsschool.ucsd.edu/faculty/profile?id=360 Jurgen Schulze]
* Discussion: Wednesdays 12-12:50pm on Zoom at https://ucsd.zoom.us/j/9100475160
* Office hours: every 2-3 weeks for each team
* Piazza Discussion Board: https://piazza.com/ucsd/fall2020/dsc180

==Overview==

In this capstone domain we are going to study how we can make machine learning systems more user friendly by exploiting additional knowledge we can derive from the system and present it to the user. These types of systems are called Explainable AI.

The example we are going to use in this class is object recognition in images. We are first going to learn about saliency and attention maps, then get to know a large publicly available image data set (COCO) and finally we are going to implement the [http://gradcam.cloudcv.org Grad-CAM algorithm] in [https://pytorch.org PyTorch] and apply it to the [https://cocodataset.org COCO image data set].

The quarter will end with a proposal for your capstone project, which you will be working on in the winter quarter.

This class will be entirely remote.

All work in this section is to be done in groups of 2-4 people.

You can use the [https://dsc-capstone.github.io/resources/computing/ DSMLP cluster], which already has the COCO data set installed.

==Schedule==

{| border="1" style="text-align:left;"
|-
! Week
! Date
! Discussion Topic
! Homework Tasks
|- 
| 1
| Oct 7
| [[Media:01_CourseOverviewF20.pdf|Overview]]
| None for week 1
|-
| 2
| Oct 14
| [[Media:02_SaliencyMapsF20.pdf|Saliency Maps]]
| [[DSC180F20W2|Literature review]]
|-
| 3
| Oct 21
| Image Data Set
| [[DSC180F20W3|COCO Dataset]]
|-
| 4
| Oct 28
| 
| [[DSC180F20W4|Replication Checkpoint #1]] due
|-
| 5
| Nov 4
| 
| Description of methods
|-
| 6
| Nov 11
| Veterans Day (No Discussion)
| Implementation of methods
|-
| 7
| Nov 18
| 
| [[DSC180F20W7|Replication Checkpoint #2]] due
|-
| 8
| Nov 25
| 
| Implementation of Grad-CAM
|-
| 9
| Dec 2
| 
| Implementation of Grad-CAM
|-
| 10
| Dec 9
| 
| [[DSC180F20W10|Final Replication Report]] due, Capstone Project proposal due
|}

==Papers==

* Paper for replication: [https://arxiv.org/pdf/1610.02391v1.pdf Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization]
* Paper on COCO image dataset: [https://arxiv.org/abs/1405.0312 Microsoft COCO: Common Objects in Context]
* Original attention map paper: [https://arxiv.org/pdf/1312.6034.pdf Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps]
* [https://arxiv.org/pdf/1512.04150.pdf Learning Deep Features for Discriminative Localization]
* [https://arxiv.org/pdf/1311.2901.pdf Visualizing CNNs with deconvolution]

==Useful Links==

* [https://dsc-capstone.github.io Main DSC Capstone Website]
* [https://medium.com/datadriveninvestor/introduction-to-how-cnns-work-77e0e4cde99b Introduction to how CNNs Work]
* [http://playground.tensorflow.org Tensorflow Playground] 
* [https://github.com/jacobgil/pytorch-grad-cam Grad-CAM implementation in Pytorch]
* [https://cocodataset.org COCO Image Dataset]
* http://cs-people.bu.edu/jmzhang/excitationbp.html

==Direct CNN Visualization==

* https://github.com/conan7882/CNN-Visualization
* https://medium.com/@awjuliani/visualizing-neural-network-layer-activation-tensorflow-tutorial-d45f8bf7bbc4
* drawNet: http://people.csail.mit.edu/torralba/research/drawCNN/drawNet.html
* https://towardsdatascience.com/how-to-visualize-convolutional-features-in-40-lines-of-code-70b7d87b0030
* https://towardsdatascience.com/understanding-your-convolution-network-with-visualizations-a4883441533b
* http://cs231n.github.io/understanding-cnn/