basicsr.data.prefetch_dataloader¶

class basicsr.data.prefetch_dataloader.CPUPrefetcher(loader)[source]¶

Bases: object

CPU prefetcher.

Parameters:: loader – Dataloader.

next()[source]¶

reset()[source]¶

class basicsr.data.prefetch_dataloader.CUDAPrefetcher(loader, opt)[source]¶

Bases: object

CUDA prefetcher.

Reference: https://github.com/NVIDIA/apex/issues/304#

It may consume more GPU memory.

Parameters:

loader – Dataloader.
opt (dict) – Options.

next()[source]¶

preload()[source]¶

reset()[source]¶

class basicsr.data.prefetch_dataloader.PrefetchDataLoader(num_prefetch_queue, **kwargs)[source]¶

Bases: DataLoader

Prefetch version of dataloader.

Reference: https://github.com/IgorSusmelj/pytorch-styleguide/issues/5#

TODO: Need to test on single gpu and ddp (multi-gpu). There is a known issue in ddp.

Parameters:

num_prefetch_queue (int) – Number of prefetch queue.
kwargs (dict) – Other arguments for dataloader.

batch_size: int | None¶

dataset: Dataset[T_co]¶

drop_last: bool¶

num_workers: int¶

pin_memory: bool¶

pin_memory_device: str¶

prefetch_factor: int¶

sampler: Sampler | Iterable¶

timeout: float¶

class basicsr.data.prefetch_dataloader.PrefetchGenerator(generator, num_prefetch_queue)[source]¶

Bases: Thread

A general prefetch generator.

Reference: https://stackoverflow.com/questions/7323664/python-generator-pre-fetch

Parameters:

generator – Python generator.
num_prefetch_queue (int) – Number of prefetch queue.

run()[source]¶

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.