Repository for implementation of active learning and semi-supervised learning algorithms and applying it to medical imaging datasets

• Least Confidence Sampling [1]
• Margin Sampling [1]
• Ratio Sampling [1]
• Maximum Entropy Sampling [1]
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning [2]
• Learning Loss for Active Learning [3]
• BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning [4]

• Pseudo Labeling [5]
• Autoencoder [5]
• A Simple Framework for Contrastive Learning of Visual Representations [6]
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence [7]

numpy>=1.18.5
torch>=1.4.0
torchvision>=0.5.0
scikit-learn>=0.23.1
pandas>=1.0.4
Pillow>=7.1.2
matplotlib>=3.2.1
toma>=1.1.0
scikit-image>=0.17.2
pytorch-msssim
scikit-learn-extra
dataclasses

The train/test splits for the three datasets can be downloaded here:

A. White Blood Cell: Download
B. Skin Lesion: Download
C. Cell Cycle: Download

usage: python3 train.py [-h] [-n NAME] [-e EPOCHS] [--start-epoch START_EPOCH] [-r]
               [--load-pretrained] [-b BATCH_SIZE] [--lr LR] [--mo MO]
               [--nesterov NESTEROV] [--wd WD] [-p PRINT_FREQ]
               [--layers LAYERS] [--widen-factor WIDEN_FACTOR]
               [--drop-rate DROP_RATE] [--no-augment]
               [--add-labeled-epochs ADD_LABELED_EPOCHS]
               [--add-labeled ADD_LABELED] [--start-labeled START_LABELED]
               [--stop-labeled STOP_LABELED]
               [--labeled-warmup-epochs LABELED_WARMUP_EPOCHS]
               [--unlabeled-subset UNLABELED_SUBSET] [-o] [-m] [--rem]
               [--arch {wideresnet,densenet,lenet,resnet}] [-l {ce,fl}]
               [--log-path LOG_PATH]
               [--al {least_confidence,margin_confidence,ratio_confidence,entropy_based,mc_dropout,learning_loss,augmentations_based}]
               [--mc-dropout-iterations MC_DROPOUT_ITERATIONS]
               [--augmentations_based_iterations AUGMENTATIONS_BASED_ITERATIONS]
               [--root ROOT]
               [--weak-supervision-strategy {active_learning,semi_supervised,random_sampling,fully_supervised}]
               [--ssl {pseudo_labeling,auto_encoder,simclr,fixmatch,auto_encoder_cl,auto_encoder_no_feat,simclr_with_al,auto_encoder_with_al,fixmatch_with_al}]
               [--semi-supervised-uncertainty-method {entropy_based,augmentations_based}]
               [--pseudo-labeling-threshold PSEUDO_LABELING_THRESHOLD]
               [--simclr-train-epochs SIMCLR_TRAIN_EPOCHS]
               [--simclr-temperature SIMCLR_TEMPERATURE] [--simclr-normalize]
               [--simclr-batch-size SIMCLR_BATCH_SIZE]
               [--simclr-arch {lenet,resnet}]
               [--simclr-base-lr SIMCLR_BASE_LR]
               [--simclr-optimizer {adam,lars}] [--simclr-resume] [--weighted]
               [--eval] [-d {cifar10,matek,cifar100,jurkat,plasmodium,isic}]
               [--checkpoint-path CHECKPOINT_PATH] [-s {6666,9999,2323,5555}]
               [--store-logs] [--run-batch] [--reset-model]
               [--fixmatch-mu FIXMATCH_MU]
               [--fixmatch-lambda-u FIXMATCH_LAMBDA_U]
               [--fixmatch-threshold FIXMATCH_THRESHOLD]
               [--fixmatch-k-img FIXMATCH_K_IMG]
               [--fixmatch-epochs FIXMATCH_EPOCHS]
               [--fixmatch-warmup FIXMATCH_WARMUP]
               [--fixmatch-init {None,random,pretrained,simclr,autoencoder}]
               [--learning-loss-weight LEARNING_LOSS_WEIGHT]
               [--dlctcs-loss-weight DLCTCS_LOSS_WEIGHT]
               [--autoencoder-train-epochs AUTOENCODER_TRAIN_EPOCHS]
               [--autoencoder-z-dim AUTOENCODER_Z_DIM] [--autoencoder-resume]
               [--k-medoids] [--k-medoids-n-clusters K_MEDOIDS_N_CLUSTERS]
               [--novel-class-detection] [--gpu-id GPU_ID]

show this help message and exit

name of current running experiment

number of total epochs for AL training

starting epoch number (useful when resuming)

flag to be set if an existing model is to be loaded

load pretrained imagenet weights or not

batch size for AL training (default: 256)

initial learning rate for AL optimizer

momentum

nesterov momentum

weight decay for AL optimizer

print frequency per step

total number of layers for ResNext architecture

widen factor for ResNext architecture

dropout probability for ResNet/LeNet architecture

whether to use standard augmentations or not

if recall doesn't improve perform AL cycle

amount of labeled data to be added during each AL cycle

amount of labeled data to start the AL training

amount of labeled data to stop the AL training

number of warmup epochs before AL training

the subset of the unlabeled data to use for AL algorithms

perform oversampling for labeled dataset or not

to merge certain classes in the dataset (see dataset scripts)

to remove certain classes in the dataset (see dataset scripts)

the architecture to use for AL training

the loss to be used. ce = cross entropy and fl = focal loss

the directory root for storing/retrieving the logs

the AL algorithm to use

number of iterations for mc dropout

number of iterations for augmentations based AL algorithm

the root path for the datasets

the weakly supervised strategy to use

the SSL algorithm to use

the AL algorithm to use in conjunction with a SSL algorithm

the threshold for considering the pseudo label as the actual label

number of total epochs for SimCLR training

the temperature term for simclr loss

normalize the hidden feat vectors in simclr or not

batch size for simclr training (default: 1024)

which encoder architecture to use for simclr

base learning rate for SimCLR optimizer

which optimizer to use for simclr training

flag to be set if an existing simclr model is to be loaded

to use weighted loss or not (only in case of ce)

only perform evaluation and exit

the dataset to train on

the directory root for saving/resuming checkpoints from

the random seed to set

store the logs after training

run all methods in batch mode

reset models after every labels injection cycle

coefficient of unlabeled batch size i.e. mu.B from paper

coefficient of unlabeled loss

pseudo label threshold

number of labeled examples

epochs for SSL or SSL + AL training

warmup epochs with unlabeled data

the semi supervised method to use

the weight for the loss network, loss term in the objective function

the weight for classification loss in dlctcs

number of total epochs for autoencoder training

the bottleneck dimension for the autoencoder architecture

flag to be set if an existing autoencoder model is to be loaded

to perform k medoids init with SimCLR

number of k medoids clusters

turn on novel class detection

the id of the GPU to use

To run supervised training with augmentations-based sampling on white blood cell dataset:
python3 train.py --dataset matek --seed <seed> --root <datasets_root> --al augmentations_based --weak-supervision-strategy active_learning

To run semi-supervised training with augmentations-based sampling on cell cycle dataset:
python3 train.py --dataset jurkat --seed <seed> --root <datasets_root> --ssl fixmatch_with_al --weak-supervision-strategy semi_supervised --semi-supervised-uncertainty-method augmentations_based

To run SimCLR self-supervised pre-training on skin lesions dataset:
python3 train.py --dataset isic --seed <seed> --root <datasets_root> --ssl simclr --simclr-resume 

Results for the three datasets:

A. White Blood Cell
B. Skin Lesion
C. Cell Cycle

All results can be downloaded from here: Download

[1] Settles, B. (2009). Active learning literature survey. University of Wisconsin-Madison Department of Computer Sciences.

[2] Gal, Y., & Ghahramani, Z. (2016). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning (pp. 1050-1059).

[3] Yoo, D., & Kweon, I. S. (2019). Learning loss for active learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 93-102).

[4] Kirsch, A., van Amersfoort, J., & Gal, Y. (2019). Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning. In Advances in Neural Information Processing Systems (pp. 7026-7037).

[5] Van Engelen, J. E., & Hoos, H. H. (2020). A survey on semi-supervised learning. Machine Learning, 109(2), 373-440.

[6] Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709.

[7] Sohn, K., Berthelot, D., Li, C. L., Zhang, Z., Carlini, N., Cubuk, E. D., ... & Raffel, C. (2020). Fixmatch: Simplifying semi-supervised learning with consistency and confidence. arXiv preprint arXiv:2001.07685.