TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al. (Optical flow)	Precision@0.5	0.5	# 22
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al. (Optical flow)	Precision@0.9	0.004	# 24
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al. (Optical flow)	IoU overall	0.551	# 24
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al. (Optical flow)	IoU mean	0.426	# 24
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al. (Optical flow)	Precision@0.6	0.376	# 23
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al. (Optical flow)	Precision@0.7	0.231	# 23
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al. (Optical flow)	Precision@0.8	0.094	# 23
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al. (Optical flow)	AP	0.215	# 19
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al.	Precision@0.5	0.475	# 25
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al.	Precision@0.9	0.002	# 25
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al.	IoU overall	0.536	# 25
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al.	IoU mean	0.421	# 25
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al.	Precision@0.6	0.347	# 24
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al.	Precision@0.7	0.211	# 24
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al.	Precision@0.8	0.08	# 24
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al.	AP	0.198	# 20
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al. (Optical flow)	Precision@0.5	0.712	# 16
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al. (Optical flow)	Precision@0.6	0.518	# 17
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al. (Optical flow)	Precision@0.7	0.264	# 18
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al. (Optical flow)	Precision@0.8	0.030	# 19
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al. (Optical flow)	Precision@0.9	0.000	# 11
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al. (Optical flow)	AP	0.267	# 13
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al. (Optical flow)	IoU overall	0.555	# 15
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al. (Optical flow)	IoU mean	0.570	# 15
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al.	Precision@0.5	0.699	# 17
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al.	Precision@0.6	0.460	# 19
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al.	Precision@0.7	0.173	# 19
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al.	Precision@0.8	0.014	# 20
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al.	Precision@0.9	0.000	# 11
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al.	AP	0.233	# 15
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al.	IoU overall	0.541	# 18
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al.	IoU mean	0.542	# 18

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/actor-and-action-video-segmentation-from-a/referring-expression-segmentation-on-j-hmdb)](https://paperswithcode.com/sota/referring-expression-segmentation-on-j-hmdb?p=actor-and-action-video-segmentation-from-a)`
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/actor-and-action-video-segmentation-from-a/referring-expression-segmentation-on-a2d)](https://paperswithcode.com/sota/referring-expression-segmentation-on-a2d?p=actor-and-action-video-segmentation-from-a)`

Actor and Action Video Segmentation from a Sentence

CVPR 2018 · Kirill Gavrilyuk, Amir Ghodrati, Zhenyang Li, Cees G. M. Snoek ·

This paper strives for pixel-level segmentation of actors and their actions in video content. Different from existing works, which all learn to segment from a fixed vocabulary of actor and action pairs, we infer the segmentation from a natural language input sentence. This allows to distinguish between fine-grained actors in the same super-category, identify actor and action instances, and segment pairs that are outside of the actor and action vocabulary. We propose a fully-convolutional model for pixel-level actor and action segmentation using an encoder-decoder architecture optimized for video. To show the potential of actor and action video segmentation from a sentence, we extend two popular actor and action datasets with more than 7,500 natural language descriptions. Experiments demonstrate the quality of the sentence-guided segmentations, the generalization ability of our model, and its advantage for traditional actor and action segmentation compared to the state-of-the-art.

PDF Abstract CVPR 2018 PDF CVPR 2018 Abstract

Code

Add Remove Mark official

JerryX1110/awesome-rvos

Tasks

Add Remove

Action Segmentation

Referring Expression Segmentation

Segmentation

Sentence

Video Segmentation

Video Semantic Segmentation

Datasets

Introduced in the Paper:

A2D Sentences

Used in the Paper:

JHMDB

A2D

Results from the Paper

Edit

Ranked #13 on Referring Expression Segmentation on J-HMDB

Get a GitHub badge

Task	Dataset	Model	Metric Name	Metric Value	Global Rank	Benchmark
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al. (Optical flow)	Precision@0.5	0.5	# 22	Compare
			Precision@0.9	0.004	# 24	Compare
			IoU overall	0.551	# 24	Compare
			IoU mean	0.426	# 24	Compare
			Precision@0.6	0.376	# 23	Compare
			Precision@0.7	0.231	# 23	Compare
			Precision@0.8	0.094	# 23	Compare
			AP	0.215	# 19	Compare
Referring Expression Segmentation	A2D Sentences	Gavriluyk el al.	Precision@0.5	0.475	# 25	Compare
			Precision@0.9	0.002	# 25	Compare
			IoU overall	0.536	# 25	Compare
			IoU mean	0.421	# 25	Compare
			Precision@0.6	0.347	# 24	Compare
			Precision@0.7	0.211	# 24	Compare
			Precision@0.8	0.08	# 24	Compare
			AP	0.198	# 20	Compare
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al. (Optical flow)	Precision@0.5	0.712	# 16	Compare
			Precision@0.6	0.518	# 17	Compare
			Precision@0.7	0.264	# 18	Compare
			Precision@0.8	0.030	# 19	Compare
			Precision@0.9	0.000	# 11	Compare
			AP	0.267	# 13	Compare
			IoU overall	0.555	# 15	Compare
			IoU mean	0.570	# 15	Compare
Referring Expression Segmentation	J-HMDB	Gavrilyuk et al.	Precision@0.5	0.699	# 17	Compare
			Precision@0.6	0.460	# 19	Compare
			Precision@0.7	0.173	# 19	Compare
			Precision@0.8	0.014	# 20	Compare
			Precision@0.9	0.000	# 11	Compare
			AP	0.233	# 15	Compare
			IoU overall	0.541	# 18	Compare
			IoU mean	0.542	# 18	Compare

Methods

Add Remove

No methods listed for this paper. Add relevant methods here

Edit Social Preview

Actor and Action Video Segmentation from a Sentence

Code Edit Add Remove Mark official

Tasks Edit Add Remove

Datasets Edit

Results from the Paper Edit

Methods Edit Add Remove

Code

Add Remove Mark official

Tasks

Add Remove

Datasets

Results from the Paper

Edit

Methods

Add Remove