basicsr.utils.lmdb_util

class basicsr.utils.lmdb_util.LmdbMaker(lmdb_path, map_size=1099511627776, batch=5000, compress_level=1)[source]

Bases: object

LMDB Maker.

Parameters:

  • lmdb_path (str) – Lmdb save path.

  • map_size (int) – Map size for lmdb env. Default: 1024 ** 4, 1TB.

  • batch (int) – After processing batch images, lmdb commits. Default: 5000.

  • compress_level (int) – Compress level when encoding images. Default: 1.

close()[source]
put(img_byte, key, img_shape)[source]
basicsr.utils.lmdb_util.make_lmdb_from_imgs(data_path, lmdb_path, img_path_list, keys, batch=5000, compress_level=1, multiprocessing_read=False, n_thread=40, map_size=None)[source]

Make lmdb from images.

Contents of lmdb. The file structure is:

example.lmdb
├── data.mdb
├── lock.mdb
├── meta_info.txt

The data.mdb and lock.mdb are standard lmdb files and you can refer to https://lmdb.readthedocs.io/en/release/ for more details.

The meta_info.txt is a specified txt file to record the meta information of our datasets. It will be automatically created when preparing datasets by our provided dataset tools. Each line in the txt file records 1)image name (with extension), 2)image shape, and 3)compression level, separated by a white space.

For example, the meta information could be: 000_00000000.png (720,1280,3) 1, which means: 1) image name (with extension): 000_00000000.png; 2) image shape: (720,1280,3); 3) compression level: 1

We use the image name without extension as the lmdb key.

If multiprocessing_read is True, it will read all the images to memory using multiprocessing. Thus, your server needs to have enough memory.

Parameters:

  • data_path (str) – Data path for reading images.

  • lmdb_path (str) – Lmdb save path.

  • img_path_list (str) – Image path list.

  • keys (str) – Used for lmdb keys.

  • batch (int) – After processing batch images, lmdb commits. Default: 5000.

  • compress_level (int) – Compress level when encoding images. Default: 1.

  • multiprocessing_read (bool) – Whether use multiprocessing to read all the images to memory. Default: False.

  • n_thread (int) – For multiprocessing.

  • map_size (int | None) – Map size for lmdb env. If None, use the estimated size from images. Default: None

basicsr.utils.lmdb_util.read_img_worker(path, key, compress_level)[source]

Read image worker.

Parameters:

  • path (str) – Image path.

  • key (str) – Image key.

  • compress_level (int) – Compress level when encoding images.

Returns:

Image key. byte: Image byte. tuple[int]: Image shape.

Return type:

str