detectron2.data package

detectron2.data.build_batch_data_loader(dataset, sampler, total_batch_size, *, aspect_ratio_grouping=False, num_workers=0)

Build a batched dataloader for training.

Parameters

• dataset (torch.utils.data.Dataset) – map-style PyTorch dataset. Can be indexed.

• sampler (torch.utils.data.sampler.Sampler) – a sampler that produces indices

• total_batch_size (int) – total batch size across GPUs.

• aspect_ratio_grouping (bool) – whether to group images with similar aspect ratio for efficiency. When enabled, it requires each element in dataset be a dict with keys “width” and “height”.

• num_workers (int) – number of parallel data loading workers

Returns

iterable[list]. Length of each list is the batch size of the current

GPU. Each element in the list comes from the dataset.

detectron2.data.build_detection_test_loader(cfg, dataset_name, mapper=None)

Similar to build_detection_train_loader. But this function uses the given dataset_name argument (instead of the names in cfg), and uses batch size 1.

Parameters

• cfg – a detectron2 CfgNode

• dataset_name (str) – a name of the dataset that’s available in the DatasetCatalog

• mapper (callable) – a callable which takes a sample (dict) from dataset and returns the format to be consumed by the model. By default it will be DatasetMapper(cfg, False).

Returns

DataLoader – a torch DataLoader, that loads the given detection dataset, with test-time transformation and batching.

detectron2.data.build_detection_train_loader(cfg, mapper=None)

A data loader is created by the following steps:

1. Use the dataset names in config to query DatasetCatalog, and obtain a list of dicts.

2. Coordinate a random shuffle order shared among all processes (all GPUs)

3. Each process spawn another few workers to process the dicts. Each worker will: * Map each metadata dict into another format to be consumed by the model. * Batch them by simply putting dicts into a list.

The batched list[mapped_dict] is what this dataloader will yield.

Parameters

• cfg (CfgNode) – the config

• mapper (callable) – a callable which takes a sample (dict) from dataset and returns the format to be consumed by the model. By default it will be DatasetMapper(cfg, True).

Returns

an infinite iterator of training data

detectron2.data.get_detection_dataset_dicts(dataset_names, filter_empty=True, min_keypoints=0, proposal_files=None)

Load and prepare dataset dicts for instance detection/segmentation and semantic segmentation.

Parameters

• dataset_names (list[str]) – a list of dataset names

• filter_empty (bool) – whether to filter out images without instance annotations

• min_keypoints (int) – filter out images with fewer keypoints than min_keypoints. Set to 0 to do nothing.

• proposal_files (list[str]) – if given, a list of object proposal files that match each dataset in dataset_names.

Returns

list[dict] – a list of dicts following the standard dataset dict format.

detectron2.data.load_proposals_into_dataset(dataset_dicts, proposal_file)

Load precomputed object proposals into the dataset.

The proposal file should be a pickled dict with the following keys:

• “ids”: list[int] or list[str], the image ids

• “boxes”: list[np.ndarray], each is an Nx4 array of boxes corresponding to the image id

• “objectness_logits”: list[np.ndarray], each is an N sized array of objectness scores corresponding to the boxes.

• “bbox_mode”: the BoxMode of the boxes array. Defaults to BoxMode.XYXY_ABS.

Parameters

• dataset_dicts (list[dict]) – annotations in Detectron2 Dataset format.

• proposal_file (str) – file path of pre-computed proposals, in pkl format.

Returns

list[dict] – the same format as dataset_dicts, but added proposal field.

detectron2.data.print_instances_class_histogram(dataset_dicts, class_names)

Parameters

• dataset_dicts (list[dict]) – list of dataset dicts.

• class_names (list[str]) – list of class names (zero-indexed).

class detectron2.data.DatasetCatalog

Bases: object

A catalog that stores information about the datasets and how to obtain them.

It contains a mapping from strings (which are names that identify a dataset, e.g. “coco_2014_train”) to a function which parses the dataset and returns the samples in the format of list[dict].

The returned dicts should be in Detectron2 Dataset format (See DATASETS.md for details) if used with the data loader functionalities in data/build.py,data/detection_transform.py.

The purpose of having this catalog is to make it easy to choose different datasets, by just using the strings in the config.

static register(name, func)

Parameters

• name (str) – the name that identifies a dataset, e.g. “coco_2014_train”.

• func (callable) – a callable which takes no arguments and returns a list of dicts. It must return the same results if called multiple times.

static get(name)

Call the registered function and return its results.

Parameters

name (str) – the name that identifies a dataset, e.g. “coco_2014_train”.

Returns

list[dict] – dataset annotations.0

static list() → List[str]

List all registered datasets.

Returns

list[str]

static clear()

Remove all registered dataset.

static remove(name)

Remove the dataset registered by name.

class detectron2.data.MetadataCatalog

Bases: object

MetadataCatalog provides access to “Metadata” of a given dataset.

The metadata associated with a certain name is a singleton: once created, the metadata will stay alive and will be returned by future calls to get(name).

It’s like global variables, so don’t abuse it. It’s meant for storing knowledge that’s constant and shared across the execution of the program, e.g.: the class names in COCO.

static get(name)

Parameters

name (str) – name of a dataset (e.g. coco_2014_train).

Returns

Metadata – The Metadata instance associated with this name, or create an empty one if none is available.

static list()

List all registered metadata.

Returns

list[str] – keys (names of datasets) of all registered metadata

static clear()

Remove all registered metadata.

static remove(name)

Remove the metadata registered by name.

class detectron2.data.Metadata

Bases: types.SimpleNamespace

A class that supports simple attribute setter/getter. It is intended for storing metadata of a dataset and make it accessible globally.

Examples:

# somewhere when you load the data:
MetadataCatalog.get("mydataset").thing_classes = ["person", "dog"]

# somewhere when you print statistics or visualize:
classes = MetadataCatalog.get("mydataset").thing_classes

name: str = 'N/A'

as_dict()

Returns all the metadata as a dict. Note that modifications to the returned dict will not reflect on the Metadata object.

set(**kwargs)

Set multiple metadata with kwargs.

get(key, default=None)

Access an attribute and return its value if exists. Otherwise return default.

class detectron2.data.DatasetFromList(lst: list, copy: bool = True, serialize: bool = True)

Bases: torch.utils.data.dataset.Dataset

Wrap a list to a torch Dataset. It produces elements of the list as data.

__init__(lst: list, copy: bool = True, serialize: bool = True)

Parameters

• lst (list) – a list which contains elements to produce.

• copy (bool) – whether to deepcopy the element when producing it, so that the result can be modified in place without affecting the source in the list.

• serialize (bool) – whether to hold memory using serialized objects, when enabled, data loader workers can use shared RAM from master process instead of making a copy.

class detectron2.data.MapDataset(dataset, map_func)

Bases: torch.utils.data.dataset.Dataset

Map a function over the elements in a dataset.

Parameters

• dataset – a dataset where map function is applied.

• map_func – a callable which maps the element in dataset. map_func is responsible for error handling, when error happens, it needs to return None so the MapDataset will randomly use other elements from the dataset.

class detectron2.data.DatasetMapper(*args, **kwargs)

Bases: object

A callable which takes a dataset dict in Detectron2 Dataset format, and map it into a format used by the model.

This is the default callable to be used to map your dataset dict into training data. You may need to follow it to implement your own one for customized logic, such as a different way to read or transform images. See Use Dataloaders for details.

The callable currently does the following:

1. Read the image from “file_name”

2. Applies cropping/geometric transforms to the image and annotations

3. Prepare data and annotations to Tensor and Instances

__init__(is_train: bool, *, augmentations: List[Union[detectron2.data.transforms.augmentation.Augmentation, fvcore.transforms.transform.Transform]], image_format: str, use_instance_mask: bool = False, use_keypoint: bool = False, instance_mask_format: str = 'polygon', keypoint_hflip_indices: Optional[numpy.ndarray] = None, precomputed_proposal_topk: Optional[int] = None, recompute_boxes: bool = False)

NOTE: this interface is experimental.

Parameters

• is_train – whether it’s used in training or inference

• augmentations – a list of augmentations or deterministic transforms to apply

• image_format – an image format supported by detection_utils.read_image().

• use_instance_mask – whether to process instance segmentation annotations, if available

• use_keypoint – whether to process keypoint annotations if available

• instance_mask_format – one of “polygon” or “bitmask”. Process instance segmentation masks into this format.

• keypoint_hflip_indices – see detection_utils.create_keypoint_hflip_indices()

• precomputed_proposal_topk – if given, will load pre-computed proposals from dataset_dict and keep the top k proposals for each image.

• recompute_boxes – whether to overwrite bounding box annotations by computing tight bounding boxes from instance mask annotations.

classmethod from_config(cfg, is_train: bool = True)

__call__(dataset_dict)

Parameters

dataset_dict (dict) – Metadata of one image, in Detectron2 Dataset format.

Returns

dict – a format that builtin models in detectron2 accept

detectron2.data.detection_utils module

Common data processing utilities that are used in a typical object detection data pipeline.

exception detectron2.data.detection_utils.SizeMismatchError

Bases: ValueError

When loaded image has difference width/height compared with annotation.

detectron2.data.detection_utils.convert_image_to_rgb(image, format)

Convert an image from given format to RGB.

Parameters

• image (np.ndarray or Tensor) – an HWC image

• format (str) – the format of input image, also see read_image

Returns

(np.ndarray) – (H,W,3) RGB image in 0-255 range, can be either float or uint8

detectron2.data.detection_utils.check_image_size(dataset_dict, image)

Raise an error if the image does not match the size specified in the dict.

detectron2.data.detection_utils.transform_proposals(dataset_dict, image_shape, transforms, *, proposal_topk, min_box_size=0)

Apply transformations to the proposals in dataset_dict, if any.

Parameters

• dataset_dict (dict) – a dict read from the dataset, possibly contains fields “proposal_boxes”, “proposal_objectness_logits”, “proposal_bbox_mode”

• image_shape (tuple) – height, width

• transforms (TransformList) –

• proposal_topk (int) – only keep top-K scoring proposals

• min_box_size (int) – proposals with either side smaller than this threshold are removed

The input dict is modified in-place, with abovementioned keys removed. A new key “proposals” will be added. Its value is an Instances object which contains the transformed proposals in its field “proposal_boxes” and “objectness_logits”.

detectron2.data.detection_utils.transform_instance_annotations(annotation, transforms, image_size, *, keypoint_hflip_indices=None)

Apply transforms to box, segmentation and keypoints annotations of a single instance.

It will use transforms.apply_box for the box, and transforms.apply_coords for segmentation polygons & keypoints. If you need anything more specially designed for each data structure, you’ll need to implement your own version of this function or the transforms.

Parameters

• annotation (dict) – dict of instance annotations for a single instance. It will be modified in-place.

• transforms (TransformList or list[Transform]) –

• image_size (tuple) – the height, width of the transformed image

• keypoint_hflip_indices (ndarray[int]) – see create_keypoint_hflip_indices.

Returns

dict – the same input dict with fields “bbox”, “segmentation”, “keypoints” transformed according to transforms. The “bbox_mode” field will be set to XYXY_ABS.

detectron2.data.detection_utils.annotations_to_instances(annos, image_size, mask_format='polygon')

Create an Instances object used by the models, from instance annotations in the dataset dict.

Parameters

• annos (list[dict]) – a list of instance annotations in one image, each element for one instance.

• image_size (tuple) – height, width

Returns

Instances – It will contain fields “gt_boxes”, “gt_classes”, “gt_masks”, “gt_keypoints”, if they can be obtained from annos. This is the format that builtin models expect.

detectron2.data.detection_utils.annotations_to_instances_rotated(annos, image_size)

Create an Instances object used by the models, from instance annotations in the dataset dict. Compared to annotations_to_instances, this function is for rotated boxes only

Parameters

• annos (list[dict]) – a list of instance annotations in one image, each element for one instance.

• image_size (tuple) – height, width

Returns

Instances – Containing fields “gt_boxes”, “gt_classes”, if they can be obtained from annos. This is the format that builtin models expect.

detectron2.data.detection_utils.build_augmentation(cfg, is_train)

Create a list of default Augmentation from config. Now it includes resizing and flipping.

Returns

list[Augmentation]

detectron2.data.detection_utils.create_keypoint_hflip_indices(dataset_names)

Parameters

dataset_names (list[str]) – list of dataset names

Returns

ndarray[int] – a vector of size=#keypoints, storing the horizontally-flipped keypoint indices.

detectron2.data.detection_utils.filter_empty_instances(instances, by_box=True, by_mask=True, box_threshold=1e-05)

Filter out empty instances in an Instances object.

Parameters

• instances (Instances) –

• by_box (bool) – whether to filter out instances with empty boxes

• by_mask (bool) – whether to filter out instances with empty masks

• box_threshold (float) – minimum width and height to be considered non-empty

Returns

Instances – the filtered instances.

detectron2.data.detection_utils.read_image(file_name, format=None)

Read an image into the given format. Will apply rotation and flipping if the image has such exif information.

Parameters

• file_name (str) – image file path

• format (str) – one of the supported image modes in PIL, or “BGR” or “YUV-BT.601”.

Returns

image (np.ndarray) –

an HWC image in the given format, which is 0-255, uint8 for

supported image modes in PIL or “BGR”; float (0-1 for Y) for YUV-BT.601.

detectron2.data.datasets module

detectron2.data.datasets.load_cityscapes_instances(image_dir, gt_dir, from_json=True, to_polygons=True)

Parameters

• image_dir (str) – path to the raw dataset. e.g., “~/cityscapes/leftImg8bit/train”.

• gt_dir (str) – path to the raw annotations. e.g., “~/cityscapes/gtFine/train”.

• from_json (bool) – whether to read annotations from the raw json file or the png files.

• to_polygons (bool) – whether to represent the segmentation as polygons (COCO’s format) instead of masks (cityscapes’s format).

Returns

list[dict] – a list of dicts in Detectron2 standard format. (See Using Custom Datasets )

detectron2.data.datasets.load_coco_json(json_file, image_root, dataset_name=None, extra_annotation_keys=None)

Load a json file with COCO’s instances annotation format. Currently supports instance detection, instance segmentation, and person keypoints annotations.

Parameters

• json_file (str) – full path to the json file in COCO instances annotation format.

• image_root (str or path-like) – the directory where the images in this json file exists.

• dataset_name (str) – the name of the dataset (e.g., coco_2017_train). If provided, this function will also put “thing_classes” into the metadata associated with this dataset.

• extra_annotation_keys (list[str]) – list of per-annotation keys that should also be loaded into the dataset dict (besides “iscrowd”, “bbox”, “keypoints”, “category_id”, “segmentation”). The values for these keys will be returned as-is. For example, the densepose annotations are loaded in this way.

Returns

list[dict] – a list of dicts in Detectron2 standard dataset dicts format. (See Using Custom Datasets )

Notes

1. This function does not read the image files. The results do not have the “image” field.

detectron2.data.datasets.load_sem_seg(gt_root, image_root, gt_ext='png', image_ext='jpg')

Load semantic segmentation datasets. All files under “gt_root” with “gt_ext” extension are treated as ground truth annotations and all files under “image_root” with “image_ext” extension as input images. Ground truth and input images are matched using file paths relative to “gt_root” and “image_root” respectively without taking into account file extensions. This works for COCO as well as some other datasets.

Parameters

• gt_root (str) – full path to ground truth semantic segmentation files. Semantic segmentation annotations are stored as images with integer values in pixels that represent corresponding semantic labels.

• image_root (str) – the directory where the input images are.

• gt_ext (str) – file extension for ground truth annotations.

• image_ext (str) – file extension for input images.

Returns

list[dict] – a list of dicts in detectron2 standard format without instance-level annotation.

Notes

1. This function does not read the image and ground truth files. The results do not have the “image” and “sem_seg” fields.

detectron2.data.datasets.load_lvis_json(json_file, image_root, dataset_name=None)

Load a json file in LVIS’s annotation format.

Parameters

• json_file (str) – full path to the LVIS json annotation file.

• image_root (str) – the directory where the images in this json file exists.

• dataset_name (str) – the name of the dataset (e.g., “lvis_v0.5_train”). If provided, this function will put “thing_classes” into the metadata associated with this dataset.

Returns

list[dict] – a list of dicts in Detectron2 standard format. (See Using Custom Datasets )

Notes

1. This function does not read the image files. The results do not have the “image” field.

detectron2.data.datasets.register_lvis_instances(name, metadata, json_file, image_root)

Register a dataset in LVIS’s json annotation format for instance detection and segmentation.

Parameters

• name (str) – a name that identifies the dataset, e.g. “lvis_v0.5_train”.

• metadata (dict) – extra metadata associated with this dataset. It can be an empty dict.

• json_file (str) – path to the json instance annotation file.

• image_root (str or path-like) – directory which contains all the images.

detectron2.data.datasets.get_lvis_instances_meta(dataset_name)

Load LVIS metadata.

Parameters

dataset_name (str) – LVIS dataset name without the split name (e.g., “lvis_v0.5”).

Returns

dict – LVIS metadata with keys: thing_classes

detectron2.data.datasets.load_voc_instances(dirname: str, split: str, class_names: Union[List[str], Tuple[str, …]])

Load Pascal VOC detection annotations to Detectron2 format.

Parameters

• dirname – Contain “Annotations”, “ImageSets”, “JPEGImages”

• split (str) – one of “train”, “test”, “val”, “trainval”

• class_names – list or tuple of class names

detectron2.data.datasets.register_pascal_voc(name, dirname, split, year, class_names='aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus', 'car', 'cat', 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike', 'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor')

detectron2.data.datasets.register_coco_instances(name, metadata, json_file, image_root)

Register a dataset in COCO’s json annotation format for instance detection, instance segmentation and keypoint detection. (i.e., Type 1 and 2 in http://cocodataset.org/#format-data. instances*.json and person_keypoints*.json in the dataset).

This is an example of how to register a new dataset. You can do something similar to this function, to register new datasets.

Parameters

• name (str) – the name that identifies a dataset, e.g. “coco_2014_train”.

• metadata (dict) – extra metadata associated with this dataset. You can leave it as an empty dict.

• json_file (str) – path to the json instance annotation file.

• image_root (str or path-like) – directory which contains all the images.

detectron2.data.datasets.register_coco_panoptic_separated(name, metadata, image_root, panoptic_root, panoptic_json, sem_seg_root, instances_json)

Register a COCO panoptic segmentation dataset named name. The annotations in this registered dataset will contain both instance annotations and semantic annotations, each with its own contiguous ids. Hence it’s called “separated”.

It follows the setting used by the PanopticFPN paper:

1. The instance annotations directly come from polygons in the COCO instances annotation task, rather than from the masks in the COCO panoptic annotations.

   The two format have small differences: Polygons in the instance annotations may have overlaps. The mask annotations are produced by labeling the overlapped polygons with depth ordering.

2. The semantic annotations are converted from panoptic annotations, where all “things” are assigned a semantic id of 0. All semantic categories will therefore have ids in contiguous range [1, #stuff_categories].

This function will also register a pure semantic segmentation dataset named name + '_stuffonly'.

Parameters

• name (str) – the name that identifies a dataset, e.g. “coco_2017_train_panoptic”

• metadata (dict) – extra metadata associated with this dataset.

• image_root (str) – directory which contains all the images

• panoptic_root (str) – directory which contains panoptic annotation images

• panoptic_json (str) – path to the json panoptic annotation file

• sem_seg_root (str) – directory which contains all the ground truth segmentation annotations.

• instances_json (str) – path to the json instance annotation file

detectron2.data.samplers module

class detectron2.data.samplers.TrainingSampler(size: int, shuffle: bool = True, seed: Optional[int] = None)

Bases: torch.utils.data.sampler.Sampler

In training, we only care about the “infinite stream” of training data. So this sampler produces an infinite stream of indices and all workers cooperate to correctly shuffle the indices and sample different indices.

The samplers in each worker effectively produces indices[worker_id::num_workers] where indices is an infinite stream of indices consisting of shuffle(range(size)) + shuffle(range(size)) + … (if shuffle is True) or range(size) + range(size) + … (if shuffle is False)

__init__(size: int, shuffle: bool = True, seed: Optional[int] = None)

Parameters

• size (int) – the total number of data of the underlying dataset to sample from

• shuffle (bool) – whether to shuffle the indices or not

• seed (int) – the initial seed of the shuffle. Must be the same across all workers. If None, will use a random seed shared among workers (require synchronization among all workers).

class detectron2.data.samplers.InferenceSampler(size: int)

Bases: torch.utils.data.sampler.Sampler

Produce indices for inference. Inference needs to run on the __exact__ set of samples, therefore when the total number of samples is not divisible by the number of workers, this sampler produces different number of samples on different workers.

__init__(size: int)

Parameters

size (int) – the total number of data of the underlying dataset to sample from

class detectron2.data.samplers.RepeatFactorTrainingSampler(repeat_factors, *, shuffle=True, seed=None)

Bases: torch.utils.data.sampler.Sampler

Similar to TrainingSampler, but a sample may appear more times than others based on its “repeat factor”. This is suitable for training on class imbalanced datasets like LVIS.

__init__(repeat_factors, *, shuffle=True, seed=None)

Parameters

• repeat_factors (Tensor) – a float vector, the repeat factor for each indice. When it’s full of ones, it is equivalent to TrainingSampler(len(repeat_factors), ...).

• shuffle (bool) – whether to shuffle the indices or not

• seed (int) – the initial seed of the shuffle. Must be the same across all workers. If None, will use a random seed shared among workers (require synchronization among all workers).

static repeat_factors_from_category_frequency(dataset_dicts, repeat_thresh)

Compute (fractional) per-image repeat factors based on category frequency. The repeat factor for an image is a function of the frequency of the rarest category labeled in that image. The “frequency of category c” in [0, 1] is defined as the fraction of images in the training set (without repeats) in which category c appears. See LVIS: A Dataset for Large Vocabulary Instance Segmentation (>= v2) Appendix B.2.

Parameters

• dataset_dicts (list[dict]) – annotations in Detectron2 dataset format.

• repeat_thresh (float) – frequency threshold below which data is repeated. If the frequency is half of repeat_thresh, the image will be repeated twice.

Returns

torch.Tensor –

the i-th element is the repeat factor for the dataset image

at index i.

detectron2.data.transforms module

class detectron2.data.transforms.ExtentTransform(src_rect, output_size, interp=2, fill=0)

Bases: fvcore.transforms.transform.Transform

Extracts a subregion from the source image and scales it to the output size.

The fill color is used to map pixels from the source rect that fall outside the source image.

See: https://pillow.readthedocs.io/en/latest/PIL.html#PIL.ImageTransform.ExtentTransform

__init__(src_rect, output_size, interp=2, fill=0)

Parameters

• src_rect (x0, y0, x1, y1) – src coordinates

• output_size (h, w) – dst image size

• interp – PIL interpolation methods

• fill – Fill color used when src_rect extends outside image

apply_image(img, interp=None)

apply_coords(coords)

apply_segmentation(segmentation)

class detectron2.data.transforms.ResizeTransform(h, w, new_h, new_w, interp=None)

Bases: fvcore.transforms.transform.Transform

Resize the image to a target size.

__init__(h, w, new_h, new_w, interp=None)

Parameters

• w (h,) – original image size

• new_w (new_h,) – new image size

• interp – PIL interpolation methods, defaults to bilinear.

apply_image(img, interp=None)

apply_coords(coords)

apply_segmentation(segmentation)

inverse()

apply_rotated_box(rotated_boxes)

Apply the resizing transform on rotated boxes. For details of how these (approximation) formulas are derived, please refer to RotatedBoxes.scale().

Parameters

rotated_boxes (ndarray) – Nx5 floating point array of (x_center, y_center, width, height, angle_degrees) format in absolute coordinates.

class detectron2.data.transforms.RotationTransform(h, w, angle, expand=True, center=None, interp=None)

Bases: fvcore.transforms.transform.Transform

This method returns a copy of this image, rotated the given number of degrees counter clockwise around its center.

__init__(h, w, angle, expand=True, center=None, interp=None)

Parameters

• w (h,) – original image size

• angle (float) – degrees for rotation

• expand (bool) – choose if the image should be resized to fit the whole rotated image (default), or simply cropped

• center (tuple (width, height)) – coordinates of the rotation center if left to None, the center will be fit to the center of each image center has no effect if expand=True because it only affects shifting

• interp – cv2 interpolation method, default cv2.INTER_LINEAR

apply_image(img, interp=None)

img should be a numpy array, formatted as Height * Width * Nchannels

apply_coords(coords)

coords should be a N * 2 array-like, containing N couples of (x, y) points

apply_segmentation(segmentation)

create_rotation_matrix(offset=0)

inverse()

The inverse is to rotate it back with expand, and crop to get the original shape.

class detectron2.data.transforms.ColorTransform(op)

Bases: fvcore.transforms.transform.Transform

Generic wrapper for any photometric transforms. These transformations should only affect the color space and

not the coordinate space of the image (e.g. annotation coordinates such as bounding boxes should not be changed)

__init__(op)

Parameters

op (Callable) – operation to be applied to the image, which takes in an ndarray and returns an ndarray.

apply_image(img)

apply_coords(coords)

apply_segmentation(segmentation)

class detectron2.data.transforms.PILColorTransform(op)

Bases: detectron2.data.transforms.transform.ColorTransform

Generic wrapper for PIL Photometric image transforms,

which affect the color space and not the coordinate space of the image

__init__(op)

Parameters

op (Callable) – operation to be applied to the image, which takes in a PIL Image and returns a transformed PIL Image. For reference on possible operations see: - https://pillow.readthedocs.io/en/stable/

apply_image(img)

class detectron2.data.transforms.BlendTransform(src_image: numpy.ndarray, src_weight: float, dst_weight: float)

Bases: fvcore.transforms.transform.Transform

Transforms pixel colors with PIL enhance functions.

__init__(src_image: numpy.ndarray, src_weight: float, dst_weight: float)

Blends the input image (dst_image) with the src_image using formula: src_weight * src_image + dst_weight * dst_image

Parameters

• src_image (ndarray) – Input image is blended with this image

• src_weight (float) – Blend weighting of src_image

• dst_weight (float) – Blend weighting of dst_image

apply_image(img: numpy.ndarray, interp: str = None) → numpy.ndarray

Apply blend transform on the image(s).

Parameters

• img (ndarray) – of shape NxHxWxC, or HxWxC or HxW. The array can be of type uint8 in range [0, 255], or floating point in range [0, 1] or [0, 255].

• interp (str) – keep this option for consistency, perform blend would not require interpolation.

Returns

ndarray – blended image(s).

apply_coords(coords: numpy.ndarray) → numpy.ndarray

Apply no transform on the coordinates.

apply_segmentation(segmentation: numpy.ndarray) → numpy.ndarray

Apply no transform on the full-image segmentation.

inverse() → fvcore.transforms.transform.Transform

The inverse is a no-op.

class detectron2.data.transforms.CropTransform(x0: int, y0: int, w: int, h: int)

Bases: fvcore.transforms.transform.Transform

__init__(x0: int, y0: int, w: int, h: int)

Parameters

y0, w, h (x0,) – crop the image(s) by img[y0:y0+h, x0:x0+w].

apply_image(img: numpy.ndarray) → numpy.ndarray

Crop the image(s).

Parameters

img (ndarray) – of shape NxHxWxC, or HxWxC or HxW. The array can be of type uint8 in range [0, 255], or floating point in range [0, 1] or [0, 255].

Returns

ndarray – cropped image(s).

apply_coords(coords: numpy.ndarray) → numpy.ndarray

Apply crop transform on coordinates.

Parameters

coords (ndarray) – floating point array of shape Nx2. Each row is (x, y).

Returns

ndarray – cropped coordinates.

apply_polygons(polygons: list) → list

Apply crop transform on a list of polygons, each represented by a Nx2 array. It will crop the polygon with the box, therefore the number of points in the polygon might change.

Parameters

polygon (list[ndarray]) – each is a Nx2 floating point array of (x, y) format in absolute coordinates.

Returns

ndarray – cropped polygons.

class detectron2.data.transforms.GridSampleTransform(grid: numpy.ndarray, interp: str)

Bases: fvcore.transforms.transform.Transform

__init__(grid: numpy.ndarray, interp: str)

Parameters

• grid (ndarray) – grid has x and y input pixel locations which are used to compute output. Grid has values in the range of [-1, 1], which is normalized by the input height and width. The dimension is N x H x W x 2.

• interp (str) – interpolation methods. Options include nearest and bilinear.

apply_image(img: numpy.ndarray, interp: str = None) → numpy.ndarray

Apply grid sampling on the image(s).

Parameters

• img (ndarray) – of shape NxHxWxC, or HxWxC or HxW. The array can be of type uint8 in range [0, 255], or floating point in range [0, 1] or [0, 255].

• interp (str) – interpolation methods. Options include nearest and bilinear.

Returns

ndarray – grid sampled image(s).

apply_coords(coords: numpy.ndarray)

Not supported.

apply_segmentation(segmentation: numpy.ndarray) → numpy.ndarray

Apply grid sampling on the full-image segmentation.

Parameters

segmentation (ndarray) – of shape HxW. The array should have integer or bool dtype.

Returns

ndarray – grid sampled segmentation.

class detectron2.data.transforms.HFlipTransform(width: int)

Bases: fvcore.transforms.transform.Transform

Perform horizontal flip.

apply_image(img: numpy.ndarray) → numpy.ndarray

Flip the image(s).

Parameters

img (ndarray) – of shape HxW, HxWxC, or NxHxWxC. The array can be of type uint8 in range [0, 255], or floating point in range [0, 1] or [0, 255].

Returns

ndarray – the flipped image(s).

apply_coords(coords: numpy.ndarray) → numpy.ndarray

Flip the coordinates.

Parameters

coords (ndarray) – floating point array of shape Nx2. Each row is (x, y).

Returns

ndarray – the flipped coordinates.

Note

The inputs are floating point coordinates, not pixel indices. Therefore they are flipped by (W - x, H - y), not (W - 1 - x, H - 1 - y).

inverse() → fvcore.transforms.transform.Transform

The inverse is to flip again

apply_rotated_box(rotated_boxes)

Apply the horizontal flip transform on rotated boxes.

Parameters

rotated_boxes (ndarray) – Nx5 floating point array of (x_center, y_center, width, height, angle_degrees) format in absolute coordinates.

class detectron2.data.transforms.VFlipTransform(height: int)

Bases: fvcore.transforms.transform.Transform

Perform vertical flip.

apply_image(img: numpy.ndarray) → numpy.ndarray

Flip the image(s).

Parameters

img (ndarray) – of shape HxW, HxWxC, or NxHxWxC. The array can be of type uint8 in range [0, 255], or floating point in range [0, 1] or [0, 255].

Returns

ndarray – the flipped image(s).

apply_coords(coords: numpy.ndarray) → numpy.ndarray

Flip the coordinates.

Parameters

coords (ndarray) – floating point array of shape Nx2. Each row is (x, y).

Returns

ndarray – the flipped coordinates.

Note

The inputs are floating point coordinates, not pixel indices. Therefore they are flipped by (W - x, H - y), not (W - 1 - x, H - 1 - y).

inverse() → fvcore.transforms.transform.Transform

The inverse is to flip again

class detectron2.data.transforms.NoOpTransform

Bases: fvcore.transforms.transform.Transform

A transform that does nothing.

apply_image(img: numpy.ndarray) → numpy.ndarray

apply_coords(coords: numpy.ndarray) → numpy.ndarray

inverse() → fvcore.transforms.transform.Transform

apply_rotated_box(x)

class detectron2.data.transforms.ScaleTransform(h: int, w: int, new_h: int, new_w: int, interp: str = None)

Bases: fvcore.transforms.transform.Transform

Resize the image to a target size.

__init__(h: int, w: int, new_h: int, new_w: int, interp: str = None)

Parameters

• w (h,) – original image size.

• new_w (new_h,) – new image size.

• interp (str) – interpolation methods. Options includes nearest, linear (3D-only), bilinear, bicubic (4D-only), and area. Details can be found in: https://pytorch.org/docs/stable/nn.functional.html

apply_image(img: numpy.ndarray, interp: str = None) → numpy.ndarray

Resize the image(s).

Parameters

• img (ndarray) – of shape NxHxWxC, or HxWxC or HxW. The array can be of type uint8 in range [0, 255], or floating point in range [0, 1] or [0, 255].

• interp (str) – interpolation methods. Options includes nearest, linear (3D-only), bilinear, bicubic (4D-only), and area. Details can be found in: https://pytorch.org/docs/stable/nn.functional.html

Returns

ndarray – resized image(s).

apply_coords(coords: numpy.ndarray) → numpy.ndarray

Compute the coordinates after resize.

Parameters

coords (ndarray) – floating point array of shape Nx2. Each row is (x, y).

Returns

ndarray – resized coordinates.

apply_segmentation(segmentation: numpy.ndarray) → numpy.ndarray

Apply resize on the full-image segmentation.

Parameters

segmentation (ndarray) – of shape HxW. The array should have integer or bool dtype.

Returns

ndarray – resized segmentation.

inverse() → fvcore.transforms.transform.Transform

The inverse is to resize it back.

class detectron2.data.transforms.Transform

Bases: object

Base class for implementations of deterministic transformations for image and other data structures. “Deterministic” requires that the output of all methods of this class are deterministic w.r.t their input arguments. Note that this is different from (random) data augmentations. To perform data augmentations in training, there should be a higher-level policy that generates these transform ops.

Each transform op may handle several data types, e.g.: image, coordinates, segmentation, bounding boxes, with its apply_* methods. Some of them have a default implementation, but can be overwritten if the default isn’t appropriate. See documentation of each pre-defined apply_* methods for details. Note that The implementation of these method may choose to modify its input data in-place for efficient transformation.

The class can be extended to support arbitrary new data types with its register_type() method.

abstract apply_image(img: numpy.ndarray)

Apply the transform on an image.

Parameters

img (ndarray) – of shape NxHxWxC, or HxWxC or HxW. The array can be of type uint8 in range [0, 255], or floating point in range [0, 1] or [0, 255].

Returns

ndarray – image after apply the transformation.

abstract apply_coords(coords: numpy.ndarray)

Apply the transform on coordinates.

Parameters

coords (ndarray) – floating point array of shape Nx2. Each row is (x, y).

Returns

ndarray – coordinates after apply the transformation.

Note

The coordinates are not pixel indices. Coordinates inside an image of shape (H, W) are in range [0, W] or [0, H]. This function should correctly transform coordinates outside the image as well.

apply_segmentation(segmentation: numpy.ndarray) → numpy.ndarray

Apply the transform on a full-image segmentation. By default will just perform “apply_image”.

Parameters

• segmentation (ndarray) – of shape HxW. The array should have integer

• bool dtype. (or) –

Returns

ndarray – segmentation after apply the transformation.

apply_box(box: numpy.ndarray) → numpy.ndarray

Apply the transform on an axis-aligned box. By default will transform the corner points and use their minimum/maximum to create a new axis-aligned box. Note that this default may change the size of your box, e.g. after rotations.

Parameters

box (ndarray) – Nx4 floating point array of XYXY format in absolute coordinates.

Returns

ndarray – box after apply the transformation.

Note

The coordinates are not pixel indices. Coordinates inside an image of shape (H, W) are in range [0, W] or [0, H].

This function does not clip boxes to force them inside the image. It is up to the application that uses the boxes to decide.

apply_polygons(polygons: list) → list

Apply the transform on a list of polygons, each represented by a Nx2 array. By default will just transform all the points.

Parameters

polygon (list[ndarray]) – each is a Nx2 floating point array of (x, y) format in absolute coordinates.

Returns

list[ndarray] – polygon after apply the transformation.

Note

The coordinates are not pixel indices. Coordinates on an image of shape (H, W) are in range [0, W] or [0, H].

classmethod register_type(data_type: str, func: Optional[Callable] = None)

Register the given function as a handler that this transform will use for a specific data type.

Parameters

• data_type (str) – the name of the data type (e.g., box)

• func (callable) – takes a transform and a data, returns the transformed data.

Examples:

# call it directly
def func(flip_transform, voxel_data):
    return transformed_voxel_data
HFlipTransform.register_type("voxel", func)

# or, use it as a decorator
@HFlipTransform.register_type("voxel")
def func(flip_transform, voxel_data):
    return transformed_voxel_data

# ...
transform = HFlipTransform(...)
transform.apply_voxel(voxel_data)  # func will be called

inverse() → fvcore.transforms.transform.Transform

Create a transform that inverts the geometric changes (i.e. change of coordinates) of this transform.

Note that the inverse is meant for geometric changes only. The inverse of photometric transforms that do not change coordinates is defined to be a no-op, even if they may be invertible.

Returns

Transform

class detectron2.data.transforms.TransformList(transforms: list)

Bases: object

Maintain a list of transform operations which will be applied in sequence. .. attribute:: transforms

type

list[Transform]

__init__(transforms: list)

Parameters

transforms (list[Transform]) – list of transforms to perform.

__add__(other: fvcore.transforms.transform.TransformList) → fvcore.transforms.transform.TransformList

Parameters

other (TransformList) – transformation to add.

Returns

TransformList – list of transforms.

__iadd__(other: fvcore.transforms.transform.TransformList) → fvcore.transforms.transform.TransformList

Parameters

other (TransformList) – transformation to add.

Returns

TransformList – list of transforms.

__radd__(other: fvcore.transforms.transform.TransformList) → fvcore.transforms.transform.TransformList

Parameters

other (TransformList) – transformation to add.

Returns

TransformList – list of transforms.

__len__() → int

Returns

Number of transforms contained in the TransformList.

inverse() → fvcore.transforms.transform.TransformList

Invert each transform in reversed order.

class detectron2.data.transforms.Augmentation

Bases: object

Augmentation defines policies/strategies to generate Transform from data. It is often used for pre-processing of input data. A policy typically contains randomness, but it can also choose to deterministically generate a Transform.

A “policy” that generates a Transform may, in the most general case, need arbitrary information from input data in order to determine what transforms to apply. Therefore, each Augmentation instance defines the arguments needed by its get_transform() method with the input_args attribute. When called with the positional arguments defined by the input_args, the get_transform() method executes the policy.

Examples:

# if a policy needs to know both image and semantic segmentation
assert aug.input_args == ("image", "sem_seg")
tfm: Transform = aug.get_transform(image, sem_seg)
new_image = tfm.apply_image(image)

To implement a custom Augmentation, define its input_args and implement get_transform().

Note that Augmentation defines the policies to create a Transform, but not how to apply the actual transform to those data.

input_args: Tuple[str] = ('image',)

Attribute of class instances that defines the argument(s) needed by get_transform(). Default to only “image”, because most policies only require knowing the image in order to determine the transform.

Users can freely define arbitrary new args and their types in custom Augmentation. In detectron2 we use the following convention:

• image: (H,W) or (H,W,C) ndarray of type uint8 in range [0, 255], or floating point in range [0, 1] or [0, 255].

• boxes: (N,4) ndarray of float32. It represents the instance bounding boxes of N instances. Each is in XYXY format in unit of absolute coordinates.

• sem_seg: (H,W) ndarray of type uint8. Each element is an integer label of pixel.

We do not specify convention for other types and do not include builtin Augmentation that uses other types in detectron2.

abstract get_transform(*args) → fvcore.transforms.transform.Transform

Execute the policy to use input data to create transform(s).

:param arguments must follow what’s defined in input_args.:

Returns

Return a Transform instance, which is the transform to apply to inputs.

__repr__()

Produce something like: “MyAugmentation(field1={self.field1}, field2={self.field2})”

__str__()

Produce something like: “MyAugmentation(field1={self.field1}, field2={self.field2})”

detectron2.data.transforms.TransformGen

Alias for Augmentation, since it is something that generates :class:`Transform`s

alias of detectron2.data.transforms.augmentation.Augmentation

class detectron2.data.transforms.AugInput

Bases: object

A base class for anything on which a list of Augmentation can be applied. This class provides input arguments for Augmentation to use, and defines how to apply transforms to these data.

An instance of this class must satisfy the following:

• Augmentation declares some data it needs as arguments. A AugInput must provide access to these data in the form of attribute access (getattr). For example, if a Augmentation to be applied needs “image” and “sem_seg” arguments, this class must have the attribute “image” and “sem_seg” whose content is as required by the :class:`Augmentation`s.

• This class must have a transform(tfm: Transform) -> None() method which in-place transforms all attributes stored in the class.

transform(tfm: fvcore.transforms.transform.Transform) → None

apply_augmentations(augmentations: List[Union[detectron2.data.transforms.augmentation.Augmentation, fvcore.transforms.transform.Transform]]) → fvcore.transforms.transform.TransformList

Apply a list of Transform/Augmentation in-place and returned the applied transform. Attributes of this class will be modified.

Returns

TransformList – returns transformed inputs and the list of transforms applied. The TransformList can then be applied to other data associated with the inputs.

class detectron2.data.transforms.StandardAugInput(image: numpy.ndarray, *, boxes: Optional[numpy.ndarray] = None, sem_seg: Optional[numpy.ndarray] = None)

Bases: detectron2.data.transforms.augmentation.AugInput

A standard implementation of AugInput for the majority of use cases. This class provides the following standard attributes that are common to use by Augmentation (augmentation policies). These are chosen because most Augmentation won’t need anything more to define a augmentation policy. After applying augmentations to these special attributes, the returned transforms can then be used to transform other data structures that users have.

image

image in HW or HWC format. The meaning of C is up to users

Type

ndarray

boxes

Nx4 boxes in XYXY_ABS mode

Type

ndarray or None

sem_seg

HxW semantic segmentation mask

Type

ndarray or None

Examples:

input = StandardAugInput(image, boxes=boxes)
tfms = input.apply_augmentations(list_of_augmentations)
transformed_image = input.image
transformed_boxes = input.boxes
transformed_other_data = tfms.apply_other(other_data)

An extended project that works with new data types may require augmentation policies that need more inputs. An algorithm may need to transform inputs in a way different from the standard approach defined in this class. In those situations, users can implement new subclasses of AugInput with differnt attributes and the transform() method.

__init__(image: numpy.ndarray, *, boxes: Optional[numpy.ndarray] = None, sem_seg: Optional[numpy.ndarray] = None)

Parameters

• image – (H,W) or (H,W,C) ndarray of type uint8 in range [0, 255], or floating point in range [0, 1] or [0, 255].

• boxes – (N,4) ndarray of float32. It represents the instance bounding boxes of N instances. Each is in XYXY format in unit of absolute coordinates.

• sem_seg – (H,W) ndarray of type uint8. Each element is an integer label of pixel.

transform(tfm: fvcore.transforms.transform.Transform) → None

In-place transform all attributes of this class.

detectron2.data.transforms.apply_augmentations(augmentations: List[Union[detectron2.data.transforms.augmentation.Augmentation, fvcore.transforms.transform.Transform]], inputs)

Use AugInput.apply_augmentations() instead.

class detectron2.data.transforms.RandomApply(transform, prob=0.5)

Bases: detectron2.data.transforms.augmentation.Augmentation

Randomly apply the wrapper transformation with a given probability.

__init__(transform, prob=0.5)

Parameters

• transform (Transform, Augmentation) – the transform to be wrapped by the RandomApply. The transform can either be a Transform or Augmentation instance.

• prob (float) – probability between 0.0 and 1.0 that the wrapper transformation is applied

get_transform(img)

class detectron2.data.transforms.RandomBrightness(intensity_min, intensity_max)

Bases: detectron2.data.transforms.augmentation.Augmentation

Randomly transforms image brightness.

Brightness intensity is uniformly sampled in (intensity_min, intensity_max). - intensity < 1 will reduce brightness - intensity = 1 will preserve the input image - intensity > 1 will increase brightness

See: https://pillow.readthedocs.io/en/3.0.x/reference/ImageEnhance.html

__init__(intensity_min, intensity_max)

Parameters

• intensity_min (float) – Minimum augmentation

• intensity_max (float) – Maximum augmentation

get_transform(img)

class detectron2.data.transforms.RandomContrast(intensity_min, intensity_max)

Bases: detectron2.data.transforms.augmentation.Augmentation

Randomly transforms image contrast.

Contrast intensity is uniformly sampled in (intensity_min, intensity_max). - intensity < 1 will reduce contrast - intensity = 1 will preserve the input image - intensity > 1 will increase contrast

See: https://pillow.readthedocs.io/en/3.0.x/reference/ImageEnhance.html

__init__(intensity_min, intensity_max)

Parameters

• intensity_min (float) – Minimum augmentation

• intensity_max (float) – Maximum augmentation

get_transform(img)

class detectron2.data.transforms.RandomCrop(crop_type: str, crop_size)

Bases: detectron2.data.transforms.augmentation.Augmentation

Randomly crop a subimage out of an image.

__init__(crop_type: str, crop_size)

Parameters

• crop_type (str) – one of “relative_range”, “relative”, “absolute”, “absolute_range”. See config/defaults.py for explanation.

• crop_size (tuple[float]) – the relative ratio or absolute pixels of height and width

get_transform(img)

get_crop_size(image_size)

Parameters

image_size (tuple) – height, width

Returns

crop_size (tuple) – height, width in absolute pixels

class detectron2.data.transforms.RandomExtent(scale_range, shift_range)

Bases: detectron2.data.transforms.augmentation.Augmentation

Outputs an image by cropping a random “subrect” of the source image.

The subrect can be parameterized to include pixels outside the source image, in which case they will be set to zeros (i.e. black). The size of the output image will vary with the size of the random subrect.

__init__(scale_range, shift_range)

Parameters

• output_size (h, w) – Dimensions of output image

• scale_range (l, h) – Range of input-to-output size scaling factor

• shift_range (x, y) – Range of shifts of the cropped subrect. The rect is shifted by [w / 2 * Uniform(-x, x), h / 2 * Uniform(-y, y)], where (w, h) is the (width, height) of the input image. Set each component to zero to crop at the image’s center.

get_transform(img)

class detectron2.data.transforms.RandomFlip(prob=0.5, *, horizontal=True, vertical=False)

Bases: detectron2.data.transforms.augmentation.Augmentation

Flip the image horizontally or vertically with the given probability.

__init__(prob=0.5, *, horizontal=True, vertical=False)

Parameters

• prob (float) – probability of flip.

• horizontal (boolean) – whether to apply horizontal flipping

• vertical (boolean) – whether to apply vertical flipping

get_transform(img)

class detectron2.data.transforms.RandomSaturation(intensity_min, intensity_max)

Bases: detectron2.data.transforms.augmentation.Augmentation

Randomly transforms saturation of an RGB image. Input images are assumed to have ‘RGB’ channel order.

Saturation intensity is uniformly sampled in (intensity_min, intensity_max). - intensity < 1 will reduce saturation (make the image more grayscale) - intensity = 1 will preserve the input image - intensity > 1 will increase saturation

See: https://pillow.readthedocs.io/en/3.0.x/reference/ImageEnhance.html

__init__(intensity_min, intensity_max)

Parameters

• intensity_min (float) – Minimum augmentation (1 preserves input).

• intensity_max (float) – Maximum augmentation (1 preserves input).

get_transform(img)

class detectron2.data.transforms.RandomLighting(scale)

Bases: detectron2.data.transforms.augmentation.Augmentation

The “lighting” augmentation described in AlexNet, using fixed PCA over ImageNet. Input images are assumed to have ‘RGB’ channel order.

The degree of color jittering is randomly sampled via a normal distribution, with standard deviation given by the scale parameter.

__init__(scale)

Parameters

scale (float) – Standard deviation of principal component weighting.

get_transform(img)

class detectron2.data.transforms.RandomRotation(angle, expand=True, center=None, sample_style='range', interp=None)

Bases: detectron2.data.transforms.augmentation.Augmentation

This method returns a copy of this image, rotated the given number of degrees counter clockwise around the given center.

__init__(angle, expand=True, center=None, sample_style='range', interp=None)

Parameters

• angle (list[float]) – If sample_style=="range", a [min, max] interval from which to sample the angle (in degrees). If sample_style=="choice", a list of angles to sample from

• expand (bool) – choose if the image should be resized to fit the whole rotated image (default), or simply cropped

• center (list[[float, float]]) – If sample_style=="range", a [[minx, miny], [maxx, maxy]] relative interval from which to sample the center, [0, 0] being the top left of the image and [1, 1] the bottom right. If sample_style=="choice", a list of centers to sample from Default: None, which means that the center of rotation is the center of the image center has no effect if expand=True because it only affects shifting

get_transform(img)

class detectron2.data.transforms.Resize(shape, interp=2)

Bases: detectron2.data.transforms.augmentation.Augmentation

Resize image to a fixed target size

__init__(shape, interp=2)

Parameters

• shape – (h, w) tuple or a int

• interp – PIL interpolation method

get_transform(img)

class detectron2.data.transforms.ResizeShortestEdge(short_edge_length, max_size=9223372036854775807, sample_style='range', interp=2)

Bases: detectron2.data.transforms.augmentation.Augmentation

Scale the shorter edge to the given size, with a limit of max_size on the longer edge. If max_size is reached, then downscale so that the longer edge does not exceed max_size.

__init__(short_edge_length, max_size=9223372036854775807, sample_style='range', interp=2)

Parameters

• short_edge_length (list[int]) – If sample_style=="range", a [min, max] interval from which to sample the shortest edge length. If sample_style=="choice", a list of shortest edge lengths to sample from.

• max_size (int) – maximum allowed longest edge length.

• sample_style (str) – either “range” or “choice”.

get_transform(img)

class detectron2.data.transforms.RandomCrop_CategoryAreaConstraint(crop_type: str, crop_size, single_category_max_area: float = 1.0, ignored_category: int = None)

Bases: detectron2.data.transforms.augmentation.Augmentation

Similar to RandomCrop, but find a cropping window such that no single category occupies a ratio of more than single_category_max_area in semantic segmentation ground truth, which can cause unstability in training. The function attempts to find such a valid cropping window for at most 10 times.

input_args: Tuple[str] = ('image', 'sem_seg')

__init__(crop_type: str, crop_size, single_category_max_area: float = 1.0, ignored_category: int = None)

Parameters

• crop_size (crop_type,) – same as in RandomCrop

• single_category_max_area – the maximum allowed area ratio of a category. Set to 1.0 to disable

• ignored_category – allow this category in the semantic segmentation ground truth to exceed the area ratio. Usually set to the category that’s ignored in training.

get_transform(image, sem_seg)