torchvideo.samplers

Samplers

Different video models use different strategies in sampling frames: some use sparse sampling strategies (like TSN, TRN) whereas others like 3D CNNs use dense sampling strategies. In order to accommodate these different architectures we offer a variety of sampling strategies with the opportunity to implement your own.

FrameSampler

class torchvideo.samplers.FrameSampler

Bases: abc.ABC

Abstract base class that all frame samplers implement.

If you are creating your own sampler, you should inherit from this base class.

sample(video_length)

Generate frame indices to sample from a video of video_length frames.

Parameters

video_length (int) – The duration in frames of the video to be sampled from

Return type

Union[slice, List[int], List[slice]]

Returns

Frame indices

ClipSampler

class torchvideo.samplers.ClipSampler(clip_length, frame_step=1, test=False)

Bases: torchvideo.samplers.FrameSampler

Sample clips of a fixed duration uniformly randomly from a video.

Parameters

• clip_length (int) – Duration of clip in frames

• frame_step (int) – The step size between frames, this controls FPS reduction, a step size of 2 will halve FPS, step size of 3 will reduce FPS to 1/3.

• test (bool) – Whether or not to sample in test mode (in test mode the central clip is sampled from the video)

sample(video_length)

Generate frame indices to sample from a video of video_length frames.

Parameters

video_length (int) – The duration in frames of the video to be sampled from

Return type

Union[slice, List[int], List[slice]]

Returns

Frame indices

FullVideoSampler

class torchvideo.samplers.FullVideoSampler(frame_step=1)

Bases: torchvideo.samplers.FrameSampler

Sample all frames in a video.

Parameters

frame_step – The step size between frames, this controls FPS reduction, a step size of 2 will halve FPS, step size of 3 will reduce FPS to 1/3.

sample(video_length)

Args:

video_length: The duration in frames of the video to be sampled from.

Returns:

a

slice from 0 to video_length with step size frame_step

Return type

Union[slice, List[int], List[slice]]

TemporalSegmentSampler

class torchvideo.samplers.TemporalSegmentSampler(segment_count, snippet_length, *, sample_count=None, test=False)

Bases: torchvideo.samplers.FrameSampler

[TSN] style sampling.

The video is equally divided into a number of segments, segment_count and from within each segment a snippet, a contiguous sequence of frames, snippet_length fr+ames long is sampled.

There are two variants of sampling. One for training and one for testing. During training the snippet location within the segment is uniformly randomly sampled. During testing snippets are sampled centrally within their segment (i.e. deterministically).

[TSN] Uses the following configurations:

Network	Train/Test	segment_count	snippet_length
RGB	Train	3	1
	Test	25	1
Flow	Train	3	5
	Test	25	5

Parameters

• segment_count (int) – Number of segments to split the video into, from which a snippet is sampled.

• snippet_length (int) – The number of frames in each snippet

• sample_count (Optional[int]) – Override the number of samples to be drawn from the segments, by default the sampler will sample a total of segment_count snippets from the video. In some cases it can be useful to sample fewer than this (effectively choosing sample_count snippets from segment_count).

• test (bool) – Whether to sample in test mode or not (see class docstring for training/testing differences)

sample(video_length)

Parameters

video_length (int) – The duration in frames of the video to be sampled from

Return type

Union[List[slice], List[int]]

Returns

Frame indices as list of slices

segment_video(video_length)

Segment a video of video_length frames into self.segment_count segments.

Parameters

video_length (int) – num

Return type

Tuple[ndarray, float]

Returns

(segment_start_idx, segment_length). The segment_start_idx contains the indices of the beginning of each segment in the video. segment_length is the length for all segments.

LambdaSampler

class torchvideo.samplers.LambdaSampler(sampler)

Bases: torchvideo.samplers.FrameSampler

Custom sampler constructed from a user provided function.

Parameters

sampler (Callable[[int], Union[slice, List[slice], List[int]]]) – Function that takes an int, the video length in frames and returns a slice, list of ints, or list of slices representing indices to sample from the video. All the indices should be less than the video length - 1.

sample(video_length)

Generate frame indices to sample from a video of video_length frames.

Parameters

video_length (int) – The duration in frames of the video to be sampled from

Return type

Union[slice, List[int], List[slice]]

Returns

Frame indices