Different video models use different strategies in sampling frames: some use sparse sampling strategies (like TSN, TRN) whereas others like 3D CNNs use dense sampling strategies. In order to accommodate these different architectures we offer a variety of sampling strategies with the opportunity to implement your own.
Abstract base class that all frame samplers implement.
If you are creating your own sampler, you should inherit from this base class.
ClipSampler(clip_length, frame_step=1, test=False)¶
Sample clips of a fixed duration uniformly randomly from a video.
int) – Duration of clip in frames
int) – The step size between frames, this controls FPS reduction, a step size of 2 will halve FPS, step size of 3 will reduce FPS to 1/3.
bool) – Whether or not to sample in test mode (in test mode the central clip is sampled from the video)
Sample all frames in a video.
frame_step – The step size between frames, this controls FPS reduction, a step size of 2 will halve FPS, step size of 3 will reduce FPS to 1/3.
TemporalSegmentSampler(segment_count, snippet_length, *, sample_count=None, test=False)¶
[TSN] style sampling.
The video is equally divided into a number of segments,
segment_countand from within each segment a snippet, a contiguous sequence of frames,
snippet_lengthfr+ames long is sampled.
There are two variants of sampling. One for training and one for testing. During training the snippet location within the segment is uniformly randomly sampled. During testing snippets are sampled centrally within their segment (i.e. deterministically).
[TSN] Uses the following configurations:
int) – Number of segments to split the video into, from which a snippet is sampled.
int) – The number of frames in each snippet
int]) – Override the number of samples to be drawn from the segments, by default the sampler will sample a total of
segment_countsnippets from the video. In some cases it can be useful to sample fewer than this (effectively choosing
bool) – Whether to sample in test mode or not (see class docstring for training/testing differences)
Segment a video of
Custom sampler constructed from a user provided function.
int]]]) – Function that takes an
int, the video length in frames and returns a slice, list of ints, or list of slices representing indices to sample from the video. All the indices should be less than the video length - 1.