basicsr.data.prefetch_dataloader
- class basicsr.data.prefetch_dataloader.CPUPrefetcher(loader)[source]
Bases:
object
CPU prefetcher.
- Parameters
loader – Dataloader.
- class basicsr.data.prefetch_dataloader.CUDAPrefetcher(loader, opt)[source]
Bases:
object
CUDA prefetcher.
Ref: https://github.com/NVIDIA/apex/issues/304#
It may consums more GPU memory.
- Parameters
loader – Dataloader.
opt (dict) – Options.
- class basicsr.data.prefetch_dataloader.PrefetchDataLoader(num_prefetch_queue, **kwargs)[source]
Bases:
DataLoader
Prefetch version of dataloader.
Ref: https://github.com/IgorSusmelj/pytorch-styleguide/issues/5#
TODO: Need to test on single gpu and ddp (multi-gpu). There is a known issue in ddp.
- Parameters
num_prefetch_queue (int) – Number of prefetch queue.
kwargs (dict) – Other arguments for dataloader.
- batch_size: Optional[int]
- dataset: Dataset[T_co]
- drop_last: bool
- num_workers: int
- pin_memory: bool
- pin_memory_device: str
- prefetch_factor: int
- sampler: Union[Sampler, Iterable]
- timeout: float
- class basicsr.data.prefetch_dataloader.PrefetchGenerator(generator, num_prefetch_queue)[source]
Bases:
Thread
A general prefetch generator.
Ref: https://stackoverflow.com/questions/7323664/python-generator-pre-fetch
- Parameters
generator – Python generator.
num_prefetch_queue (int) – Number of prefetch queue.
- run()[source]
Method representing the thread’s activity.
You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.