Skip to content

Commit edeafe6

Browse files
author
raschka
committed
gradient checkpointing
1 parent c4147bc commit edeafe6

7 files changed

+245
-397
lines changed

Diff for: README.md

+6-1
Original file line numberDiff line numberDiff line change
@@ -315,10 +315,15 @@ A collection of various deep learning architectures, models, and tips for Tensor
315315
   [PyTorch: [GitHub](pytorch_ipynb/rnn/rnn_lstm_packed_own_csv_imdb.ipynb) | [Nbviewer](https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/rnn/rnn_lstm_packed_own_csv_imdb.ipynb)]
316316

317317

318+
#### Improving Memory Efficiency
319+
320+
- Gradient Checkpointing Demo (Network-in-Network trained on CIFAR-10)
321+
   [PyTorch: [GitHub](pytorch_ipynb/cnn/cnn-vgg16-celeba-data-parallel.ipynb) | [Nbviewer](https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/cnn-vgg16-celeba-data-parallel.ipynb)]
322+
318323
#### Parallel Computing
319324

320325
- Using Multiple GPUs with DataParallel -- VGG-16 Gender Classifier on CelebA
321-
   [PyTorch: [GitHub](pytorch_ipynb/cnn/cnn-vgg16-celeba-data-parallel.ipynb) | [Nbviewer](https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/cnn-vgg16-celeba-data-parallel.ipynb)]
326+
   [PyTorch: [GitHub](pytorch_ipynb/mechanics/gradient-checkpointing-nin.ipynb) | [Nbviewer](https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/mechanics/gradient-checkpointing-nin.ipynb)]
322327

323328
#### Other
324329

Diff for: pytorch_ipynb/helper_train.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -62,15 +62,15 @@ def train_classifier_simple_v1(num_epochs, model, optimizer, device,
6262

6363
train_acc = compute_accuracy(model, train_loader, device)
6464
train_loss = compute_epoch_loss(model, train_loader, device)
65-
print('***Epoch: %03d/%03d | Train. Acc.: %.3f%% | Loss: %.3f' % (
65+
print('***Epoch: %03d/%03d | Train. Acc.: %.3f%% | Loss: %.3f' % (
6666
epoch+1, num_epochs, train_acc, train_loss))
6767
log_dict['train_loss_per_epoch'].append(train_loss.item())
6868
log_dict['train_loss_per_epoch'].append(train_acc.item())
6969

7070
if valid_loader is not None:
7171
valid_acc = compute_accuracy(model, valid_loader, device)
7272
valid_loss = compute_epoch_loss(model, valid_loader, device)
73-
print('***Epoch: %03d/%03d | Valid. Acc.: %.3f%% | Loss: %.3f' % (
73+
print('***Epoch: %03d/%03d | Valid. Acc.: %.3f%% | Loss: %.3f' % (
7474
epoch+1, num_epochs, valid_acc, valid_loss))
7575
log_dict['valid_loss_per_epoch'].append(valid_loss.item())
7676
log_dict['valid_loss_per_epoch'].append(valid_acc.item())

0 commit comments

Comments
 (0)