facebook/sam2-hiera-large

What it does

SAM 2 is Meta FAIR's unified model for promptable object segmentation in images and videos. A click, box or mask on any image or video frame selects an object; the model returns a mask, and in video it tracks that object through later frames as a masklet. Meta announced the release on 29 July 2024. The published code and checkpoints are licensed Apache 2.0.

The architecture extends the original Segment Anything Model into video by adding a per-session memory. Frames are consumed one at a time. Meta calls this a streaming architecture, a natural generalisation of SAM. The memory stores the target object and earlier prompts, so the model can keep the track when the object leaves the frame and returns. Further prompts on any frame correct the prediction. On a still image the memory is empty and SAM 2 behaves like SAM. That account is from Meta's SAM 2 research page and the paper.

The paper, SAM 2: Segment Anything in Images and Videos (Ravi et al., 2024), names the task promptable visual segmentation. The authors report that SAM 2 produced better video accuracy while using 3× fewer interactions than prior interactive methods, and that on images it was more accurate than SAM and 6× faster. Those figures are the paper's own measurements, not this register's.

Checkpoints and licence

The official repository ships four Hiera image-encoder sizes with the original release: tiny, small, base-plus and large. Meta lists those at 38.9, 46, 80.8 and 224.4 million parameters. This register records the exact safetensors counts from each Hub checkpoint: 38,946,498 (tiny), 46,044,354 (small), 80,834,178 (base-plus) and 224,430,642 (large). There is no language-model context window. The Hub config.json files do not define max_position_embeddings, so that field is blank.

On 29 and 30 September 2024 Meta released SAM 2.1, described in the repository as an improved suite of checkpoints. A footnote in the paper says the published results use this later suite. Meta's blog says SAM 2.1 is stronger on visually similar objects and occlusions. The Hub cards list Apache 2.0 for SAM 2 and SAM 2.1; the licence does not change. The safetensors index for facebook/sam2.1-hiera-large holds 224,447,154 parameters. Meta says SAM 2.1 needs the current repository code, and that an older install should be uninstalled first.

The SA-V dataset

Training used the Segment Anything Video dataset, which Meta released under CC BY 4.0. The paper states 35.5 million masks across 50.9 thousand videos and 642.6 thousand masklets, 53× more masks than any existing video segmentation dataset it compared. The official dataset page rounds the same collection to 51 thousand videos and 643 thousand masklets, gathered in 47 countries.

The repository's dataset readme gives the split: 50,583 training videos with 642,036 masklets (451,720 automatic and 190,316 manual), 155 validation videos with 293 masklets, and 150 test videos with 278 masklets. In the paper the videos average 14 seconds, with 54% indoor scenes and 46% outdoor. The research page notes that annotations cover whole objects, parts, and occlusions.

How to run it

The official code requires Python 3.10 or newer, PyTorch 2.5.1 or newer, and TorchVision 0.20.1 or newer. Clone github.com/facebookresearch/sam2 and install it with pip install -e .. Checkpoints can be fetched with the repo's download script, or loaded from Hugging Face via SAM2ImagePredictor.from_pretrained and SAM2VideoPredictor.from_pretrained. Hugging Face Transformers also exposes these Hub ids through a mask-generation pipeline.

For images, set_image then predict from prompts. For video, init_state opens the clip, add_new_points_or_box adds prompts, and propagate_in_video streams the masklet frame by frame. Those APIs are from the official readme and the Hub model cards. The eight Hub ids on this register are the official facebook/sam2-hiera-* and facebook/sam2.1-hiera-* checkpoints. They are not gated. A later Meta release, SAM 3, is also on this register under mask-generation.

Questions this record can answer

When was sam2-hiera-large released?

The Hugging Face repository was created on 2 Aug 2024, which is the date this register uses.

How many parameters does sam2-hiera-large have?

The safetensors index gives 224,430,642 parameters, shown here as 224M. That figure is exact, not rounded.

What licence is sam2-hiera-large under?

The Hub card lists apache-2.0.

Is sam2-hiera-large gated?

No. The Hugging Face model API does not mark this repository as gated.