LangSAM

Overviews

Lang Segment Anything Model is an open-source project that combines the power of instance segmentation and text prompts to generate masks for specific objects in images.

Original git: https://github.com/luca-medeiros/lang-segment-anything

Configs

opt.json like this:

{
    "model_type": "vit_h",
    "weights_url":"https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth",
    "text_prompt": "",
    "save_masks": true,
    "ckpt_repo_id": "ShilongLiu/GroundingDINO",
    "ckpt_filename":"groundingdino_swinb_cogcoor.pth",
    "ckpt_config_filename":"GroundingDINO_SwinB.cfg.py",
}

model_type: type SAM weights_url: link to download weight text_prompt: text prompt to define object save_masks: true or false to save mask ckpt_repo_id: repo download grouding dino ckpt_filename: file name ckpt ckpt_config_filename: file name config

APIs

`Model`

Bases: BaseModel

`build_model(custom_weight=None)`

Build model in Lang Segment Anything Model

Returns:

Name	Type	Description
`str`		custom weight path. Default: `None`

`get_transform()`

Get image data transform function for preprocessing

Returns:

Name	Type	Description
`transform`	`func`	Transform function. This function will take input as image path and output will return `raw_image` and `image_transform`: image tensor

`predict(image_path)`

Predict from image path

Parameters:

Name	Type	Description	Default
`image_path`	`str`	image path	required

Returns:

Type	Description
	masks, boxes, phrases, logits(Tuple): net output

`process_predictions(net_output, raw_image_path, raw_image_mat, image_transform, save_image_path)`

Post process the output of net

Parameters:

Name	Type	Description	Default
`net_output`	`Tuple`	Net output from LangSAM	required
`raw_image_path`	`str`	raw image path	required
`raw_image_mat`	`(numpy, PIL, tensor)`	image matrix	required
`image_transform`	`tensor`	Image after transform	required
`save_image_path`	`str`	output image path	required

Returns:

Name	Type	Description
`save_image_path`	`str`	output path