Remembering for the Right Reasons: Explanations Reduce
Catastrophic Forgetting

Sayna Ebrahimi, Suzanne Petryk, Akash Gokul, William Gan, Joseph E. Gonzalez, Marcus Rohrbach, trevor darrell

ICLR 2021

[paper] [code]

The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

Jonathan Frankle, Michael Carbin

ICLR 2019

[paper] [code]

1990

Yann LeCun, et al. Optimal brain damage. NIPS. 1990.

1993

Babak Hassibi, et al. Second order derivatives for network pruning: Optimal brain surgeon. NIPS. 1993.

2015

Song Han, et al. Learning both weights and connections for efficient neural network. NIPS. 2015.

2016

Hao Zhou, et al. Less is more: Towards compact cnns. ECCV. 2016.

Song Han, et al. “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding.” ICLR. 2016 

2017

Tien-Ju Yang, et al. Designing energy-efficient convolutional neural networks using energy-aware pruning. CVPR. 2017

Jian-Hao Luo, et al. Thinet: A filter level pruning method for deep neural network compression. ICCV. 2017

Zhuang Liu, et al. Learning efficient convolutional networks through network slimming. ICCV. 2017.

Yihui He, et al. Channel pruning for accelerating very deep neural networks. ICCV. 2017.

Hao Li, et al. Pruning filters for efficient convnets. ICLR. 2017

Pavlo Molchanov, et al. Pruning Convolutional Neural Networks for Resource Efficient Inference. ICLR. 2017

2019

√ Jonathan Frankle, et al. The lottery ticket hypothesis: Finding sparse, trainable neural networks. ICLR. 2019.

Zhuang Liu, et al. Rethinking the value of network pruning. ICLR. 2019.

2020

Chaoqi Wang, et al. Picking winning tickets before training by preserving gradient flow. ICLR. 2020

Jonathan Frankle, et al. The early phase of neural network training. ICLR. 2020.

Haoran You, et al. Drawing early-bird tickets: Toward more efficient training of deep networks. ICLR. 2020.

Alex Renda, et al. Comparing rewinding and fine-tuning in neural network pruning. ICLR. 2020.

Jonathan Frankle, et al. Linear Mode Connectivity and the Lottery Ticket Hypothesis. ICML. 2020.

Pedro Savarese, et al. Winning the lottery with continuous sparsification. NIPS. 2020.

Long Live the Lottery: The Existence of Winning Tickets in
Lifelong Learning

Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, Zhangyang Wang

ICLR 2021

[paper]

Lifelong Learning of Compositional Structures

Jorge A Mendez, ERIC EATON

ICLR 2021

[paper]

Conditional Channel Gated Networks for Task-Aware Continual Learning

Davide Abati, Jakub Tomczak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, Babak Ehteshami Bejnordi

CVPR 2020

[paper] [talk] [lecture]

1
2
3
4
5

# 先进入到如下目录
/root/go/src/github.com/hyperledger

mkdir fabric_lab
cd fabric_lab

之后所有的实验都将在 fabric_lab 目录下进行。

将 cryptogen 、configtxgen 等加入到环境变量

1
2
3
4
5
6
7
8
9
10
11
12

# 使用export命令
export PATH=$PATH:/root/go/src/github.com/hyperledger/fabric/scripts/fabric-samples/bin
# export使用方法为 export PATH=$PATH:路径1:路径2:路径n
# $PATH为系统变量,表示之前所有设置的路径，如果不加则之前所有的路径都失效。所以必须加上。

# 另外一种方法：修改配置文件/etc/profile
# vi /etc/profile
# 在 /etc/profile 最后加入
# export PATH=$PATH:/root/go/src/github.com/hyperledger/fabric/scripts/fabric-samples/bin
# 修改好配置文件后执行
# source /etc/profile
# 使配置文件生效

安装环境

阿里云 轻量应用服务器 Ubuntu 16.04.6

注意

Hyperledger Fabric 依赖的软件版本查看 官方 github 地址 下文件 /docs/source/prereqs.rst，软件版本要求根据安装的 Fabric 的版本差异而略有不同。

PathNet: Evolution Channels Gradient Descent in Super Neural
Networks

Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, Daan Wierstra

arXiv:1701.08734

[paper] [code_1] [code_2]

Overcoming Catastrophic Forgetting with Hard Attention to the Task

Joan Serra, Didac Suris, Marius Miron, Alexandros Karatzoglou

ICML 2018

[paper] [code]