training

Training loop

xmean,xstd = 0.28, 0.35
@inplace
def transformi(b): b['image'] = [(TF.to_tensor(o)-xmean)/xstd for o in b['image']]

_dataset = load_dataset('fashion_mnist').with_transform(transformi)

100%|██████████| 2/2 [00:00<00:00, 345.18it/s]

_dataset = sample_dataset_dict(_dataset)

dls = DataLoaders.from_dataset_dict(_dataset, 1024, num_workers=4)

Core

---

source

OneBatchCB

 OneBatchCB ()

Initialize self. See help(type(self)) for accurate signature.

---

source

BasicTrainCB

 BasicTrainCB ()

Callback for basic pytorch training loop

---

source

DeviceCB

 DeviceCB (device='cpu')

Callback to train on specific device

---

source

MomentumTrainCB

 MomentumTrainCB (momentum)

Callback for basic pytorch training loop

trainer = Trainer(dls,
                  nn.CrossEntropyLoss(), 
                  torch.optim.Adam, 
                  get_model_conv(), 
                  callbacks=[BasicTrainCB(), DeviceCB(),OneBatchCB()])

trainer.fit()

trainer.summarize_callbacks()

	Step	Callback	Doc String
	before_fit	DeviceCB	Moves model to device
	before_batch	DeviceCB	moves batch to device
	predict	BasicTrainCB
	get_loss	BasicTrainCB
	backward	BasicTrainCB
	step	BasicTrainCB
	zero_grad	BasicTrainCB
	after_batch	OneBatchCB

Optimization

---

source

BaseSchedulerCB

 BaseSchedulerCB (scheduler_func)

Initialize self. See help(type(self)) for accurate signature.

---

source

EpochSchedulerCB

 EpochSchedulerCB (scheduler_func)

Steps scheduler

---

source

BatchSchedulerCB

 BatchSchedulerCB (scheduler_func)

Steps scheduler

---

source

OneCycleSchedulerCB

 OneCycleSchedulerCB (pct_start=0.3, anneal_strategy='cos',
                      cycle_momentum=True, base_momentum=0.85,
                      max_momentum=0.95, div_factor=25.0,
                      final_div_factor=10000.0, three_phase=False,
                      last_epoch=-1, verbose=False)

Steps scheduler

Acceleration

---

source

AccelerateCB

 AccelerateCB (mixed_precision='fp16')

Callback for basic pytorch training loop