Search Results for author:
Found 203 papers, 98 papers with code
Automatic Animation of Hair Blowing in Still Portrait Photos
Considering the complexity of hair structure, we innovatively treat hair wisp extraction as an instance segmentation problem, where a hair wisp is referred to as an instance.
SoccerNet 2023 Challenges Results
More information on the tasks, challenges, and leaderboards are available on https://www. soccer-net. org.
Learning Semantic Segmentation with Query Points Supervision on Aerial Images
In this work, we present a weakly supervised learning algorithm to train semantic segmentation algorithms that only rely on query point annotations instead of full mask labels.
Learning to Read Analog Gauges from Synthetic Data
We propose a two-stage CNN pipeline that identifies the key structural components of an analog gauge and outputs an angular reading.
ShadowNet for Data-Centric Quantum System Learning
Understanding the dynamics of large quantum systems is hindered by the curse of dimensionality.
Learning to Identify Critical States for Reinforcement Learning from Videos
Recent work on deep reinforcement learning (DRL) has pointed out that algorithmic information about good policies can be extracted from offline data which lack explicit information about executed actions.
Deformable Mixer Transformer with Gating for Multi-Task Learning of Dense Prediction
In this work, we present a novel MTL model by combining both merits of deformable CNN and query-based Transformer with shared gating for multi-task learning of dense prediction.
Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants
Beyond the normal case, long-tail class incremental learning and few-shot class incremental learning are also proposed to consider the data imbalance and data scarcity, respectively, which are common in real-world implementations and further exacerbate the well-known problem of catastrophic forgetting.
class-incremental learning
Few-Shot Class-Incremental Learning
+1
Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors
We present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors.
Towards Open Vocabulary Learning: A Survey
To our knowledge, this is the first comprehensive literature review of open vocabulary learning.
Enhancing Neural Rendering Methods with Image Augmentations
We found that introducing image augmentations during training presents challenges such as geometric and photometric inconsistencies for learning NRMs from images.
Dynamically Masked Discriminator for Generative Adversarial Networks
By treating the generated data in training as a stream, we propose to detect whether the discriminator slows down the learning of new knowledge in generated data.
Exploring Open-Vocabulary Semantic Segmentation without Human Labels
Although acquired extensive knowledge of visual concepts, it is non-trivial to exploit knowledge from these VL models to the task of semantic segmentation, as they are usually trained at an image level.
Open Vocabulary Semantic Segmentation
Semantic Segmentation
+2
Just a Glimpse: Rethinking Temporal Information for Video Continual Learning
Studying continual learning in the video domain poses even more challenges, as video data contains a large number of frames, which places a higher burden on the replay memory.
How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers
Our method propagates semantics on the scene graphs based on language priors and scene statistics to introduce semantic knowledge to the geometric frontiers.
Mindstorms in Natural Language-Based Societies of Mind
What should be the social structure of an NLSOM?
Rapid Adaptation in Online Continual Learning: Are We Evaluating It Right?
We revisit the common practice of evaluating adaptation of Online Continual Learning (OCL) algorithms through the metric of online accuracy, which measures the accuracy of the model on the immediate next few samples.
Large-capacity and Flexible Video Steganography via Invertible Neural Network
For large-capacity, we present a reversible pipeline to perform multiple videos hiding and recovering through a single invertible neural network (INN).
LLM as A Robotic Brain: Unifying Egocentric Memory and Control
In this paper, we propose a novel and generalizable framework called LLM-Brain: using Large-scale Language Model as a robotic brain to unify egocentric memory and control.
SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries
By providing broadcasters with a tool to summarize the content of their video with the same level of engagement as a live game, our method could help satisfy the needs of the numerous fans who follow their team but cannot necessarily watch the live game.
VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views
The Video Assistant Referee (VAR) has revolutionized association football, enabling referees to review incidents on the pitch, make informed decisions, and ensure fairness.
Revisiting Test Time Adaptation under Online Evaluation
To address this issue, we propose a more realistic evaluation protocol for TTA methods, where data is received in an online fashion from a constant-speed data stream, thereby accounting for the method's adaptation speed.
Towards Active Learning for Action Spotting in Association Football Videos
In this paper, we propose an active learning framework that selects the most informative video samples to be annotated next, thus drastically reducing the annotation effort and accelerating the training of action spotting models to reach the highest accuracy at a faster pace.
Improving Visual Question Answering Models through Robustness Analysis and In-Context Learning with a Chain of Basic Questions
This work proposes a new method that utilizes semantically related questions, referred to as basic questions, acting as noise to evaluate the robustness of VQA models.
Boundary-Denoising for Video Activity Localization
To alleviate the boundary ambiguity, we propose to study the video activity localization problem from a denoising perspective.
Online Distillation with Continual Learning for Cyclic Domain Shifts
In this paper, we propose a solution to this issue by leveraging the power of continual learning methods to reduce the impact of domain shifts.
CAMEL: Communicative Agents for "Mind" Exploration of Large Scale Language Model Society
To address the challenges of achieving autonomous cooperation, we propose a novel communicative agent framework named role-playing.
Don't FREAK Out: A Frequency-Inspired Approach to Detecting Backdoor Poisoned Samples in DNNs
In this paper we investigate the frequency sensitivity of Deep Neural Networks (DNNs) when presented with clean samples versus poisoned samples.
Computationally Budgeted Continual Learning: What Does Matter?
Our conclusions are consistent in a different number of stream time steps, e. g., 20 to 200, and under several computational budgets.
A Unified Continual Learning Framework with General Parameter-Efficient Tuning
1) Learning: the pre-trained model adapts to the new task by tuning an online PET module, along with our adaptation speed calibration to align different PET modules, 2) Accumulation: the task-specific knowledge learned by the online PET module is accumulated into an offline PET module through momentum update, 3) Ensemble: During inference, we respectively construct two experts with online/offline PET modules (which are favored by the novel/historical tasks) for prediction ensemble.
FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model
In this work, we propose a training-Free conditional Diffusion Model (FreeDoM) used for various conditions.
Re-ReND: Real-time Rendering of NeRFs across Devices
Re-ReND is designed to achieve real-time performance by converting the NeRF into a representation that can be efficiently processed by standard graphics pipelines.
Improving GAN Training via Feature Space Shrinkage
The evaluation results demonstrate that our AdaptiveMix can facilitate the training of GANs and effectively improve the image quality of generated samples.
Localizing Moments in Long Video Via Multimodal Guidance
The recent introduction of the large-scale long-form MAD dataset for language grounding in videos has enabled researchers to investigate the performance of current state-of-the-art methods in the long-form setup, with unexpected findings.
Ranked #1 on
Natural Language Moment Retrieval
on MAD
Natural Language Moment Retrieval
Natural Language Visual Grounding
+2
Real-Time Evaluation in Online Continual Learning: A New Hope
We show that a simple baseline outperforms state-of-the-art CL methods under this evaluation, questioning the applicability of existing methods in realistic settings.
Look, Listen, and Attack: Backdoor Attacks Against Video Action Recognition
Although backdoor attacks have been extensively studied in the image domain, there are very few works that explore such attacks in the video domain, and they tend to conclude that image backdoor attacks are less effective in the video domain.
NewsNet: A Novel Dataset for Hierarchical Temporal Segmentation
Temporal video segmentation is the get-to-go automatic video analysis, which decomposes a long-form video into smaller components for the following-up understanding tasks.
Re2TAL: Rewiring Pretrained Video Backbones for Reversible Temporal Action Localization
Temporal action localization (TAL) requires long-form reasoning to predict actions of various durations and complex content.
Exploring Open-Vocabulary Semantic Segmentation from CLIP Vision Encoder Distillation Only
Semantic segmentation is a crucial task in computer vision that involves segmenting images into semantically meaningful regions at the pixel level.
Open Vocabulary Semantic Segmentation
Semantic Segmentation
+2
AdaptiveMix: Improving GAN Training via Feature Space Shrinkage
The evaluation results demonstrate that our AdaptiveMix can facilitate the training of GANs and effectively improve the image quality of generated samples.
MVTN: Learning Multi-View Transformations for 3D Understanding
Multi-view projection techniques have shown themselves to be highly effective in achieving top-performing results in the recognition of 3D shapes.
SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Images
SuRFNet employs partial SRFs from few/one images and a specialized SRF loss to learn to generate high-quality sparse voxel radiance fields that can be rendered from novel views.
EgoLoc: Revisiting 3D Object Localization from Egocentric Videos with Visual Queries
Yet, we point out that the low number of camera poses caused by camera re-localization from previous VQ3D methods severally hinders their overall success rate.
PIVOT: Prompting for Video Continual Learning
In this paper, we address the problem of continual learning for video data.
On Robust Learning from Noisy Labels: A Permutation Layer Approach
We provide two variants of PermLL in this paper: one applies the permutation layer to the model's prediction, while the other applies it directly to the given noisy label.
SimCS: Simulation for Online Domain-Incremental Continual Segmentation
In ODICS, the model is continually presented with batches of densely labeled images from different domains; computation is limited and no information about the task boundaries is available.
Multi-Modal Few-Shot Temporal Action Detection
In this work, we introduce a new multi-modality few-shot (MMFS) TAD problem, which can be considered as a marriage of FS-TAD and ZS-TAD by leveraging few-shot support videos and new class names jointly.
Re^2TAL: Rewiring Pretrained Video Backbones for Reversible Temporal Action Localization
Temporal action localization (TAL) requires long-form reasoning to predict actions of various durations and complex content.
Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training
Generating images from graph-structured inputs, such as scene graphs, is uniquely challenging due to the difficulty of aligning nodes and connections in graphs with objects and their relations in images.
SegNeRF: 3D Part Segmentation with Neural Radiance Fields
The predicted semantic fields allow SegNeRF to achieve an average mIoU of $\textbf{30. 30%}$ for 2D novel view segmentation, and $\textbf{37. 46%}$ for 3D part segmentation, boasting competitive performance against point-based methods by using only a few posed images.
Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localization
Our experiments show the proposed adaptations improve egocentric query detection, leading to a better visual query localization system in both 2D and 3D configurations.
Estimating more camera poses for ego-centric videos is essential for VQ3D
Visual queries 3D localization (VQ3D) is a task in the Ego4D Episodic Memory Benchmark.
Decoupled Mixup for Generalized Visual Recognition
Since the observation is that noise-prone regions such as textural and clutter backgrounds are adverse to the generalization ability of CNN models during training, we enhance features from discriminative regions and suppress noise-prone ones when combining an image pair.
SoccerNet 2022 Challenges Results
The SoccerNet 2022 challenges were the second annual video understanding challenges organized by the SoccerNet team.
Generalizability of Adversarial Robustness Under Distribution Shifts
Recent progress in empirical and certified robustness promises to deliver reliable and deployable Deep Neural Networks (DNNs).
Improving Standard Transformer Models for 3D Point Cloud Understanding with Image Pretraining
This is mainly due to the limitation of Transformers: a demanding need for large training data.
Combating Mode Collapse in GANs via Manifold Entropy Estimation
In this paper, we propose a novel training pipeline to address the mode collapse issue of GANs.
Negative Frames Matter in Egocentric Visual Query 2D Localization
The repeated gradient computation of the same object lead to an inefficient training; (2) The false positive rate is high on background frames.
Egocentric Video-Language Pretraining @ Ego4D Challenge 2022
In this report, we propose a video-language pretraining (VLP) based solution \cite{kevin2022egovlp} for four Ego4D challenge tasks, including Natural Language Query (NLQ), Moment Query (MQ), Object State Change Classification (OSCC), and PNR Localization (PNR).
PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies
In this work, we revisit the classical PointNet++ through a systematic study of model training and scaling strategies, and offer two major contributions.
Ranked #3 on
3D Part Segmentation
on ShapeNet-Part
Certified Robustness in Federated Learning
However, as in the single-node supervised learning setup, models trained in federated learning suffer from vulnerability to imperceptible input transformations known as adversarial attacks, questioning their deployment in security-related applications.
Egocentric Video-Language Pretraining
Video-Language Pretraining (VLP), which aims to learn transferable representation to advance a wide range of video-text downstream tasks, has recently received increasing attention.
Ranked #2 on
Object State Change Classification
on Ego4D
ETAD: Training Action Detection End to End on a Laptop
We propose to sequentially forward the snippet frame through the video encoder, and backward only a small necessary portion of gradients to update the encoder.
UnrealNAS: Can We Search Neural Architectures with Unreal Data?
Neural architecture search (NAS) has shown great success in the automatic design of deep neural networks (DNNs).
Contrastive Language-Action Pre-training for Temporal Localization
Long-form video understanding requires designing approaches that are able to temporally localize activities or language.
Contrastive Learning
Few Shot Temporal Action Localization
+3
SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos
Tracking objects in soccer videos is extremely important to gather both player and team statistics, whether it is to estimate the total distance run, the ball possession or the team formation.
3DeformRS: Certifying Spatial Deformations on Point Clouds
In this work, we propose 3DeformRS, a method to certify the robustness of point cloud Deep Neural Networks (DNNs) against real-world deformations.
When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search
TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture.
Real-time Hyperspectral Imaging in Hardware via Trained Metasurface Encoders
Hyperspectral imaging has attracted significant attention to identify spectral signatures for image classification and automated pattern recognition in computer vision.
End-to-End Active Speaker Detection
Recent advances in the Active Speaker Detection (ASD) problem build upon a two-stage process: feature extraction and spatio-temporal context aggregation.
Ranked #4 on
Audio-Visual Active Speaker Detection
on AVA-ActiveSpeaker
(using extra training data)
R-DFCIL: Relation-Guided Representation Learning for Data-Free Class Incremental Learning
After RRL, the classification head is refined with global class-balanced classification loss to address the data imbalance issue as well as learn the decision boundaries between new and previous classes.
Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels
To provide an alternative, we propose a novel approach that utilizes monocular RGB images and point clouds to generate pseudo scene flow labels for training scene flow networks.
SegTAD: Precise Temporal Action Detection via Semantic Segmentation
To address these issues and precisely model temporal action detection, we formulate the task of temporal action detection in a novel perspective of semantic segmentation.
OWL (Observe, Watch, Listen): Audiovisual Temporal Context for Localizing Actions in Egocentric Videos
We validate our approach in two large-scale datasets, EPIC-Kitchens, and HOMAGE.
Towards Assessing and Characterizing the Semantic Robustness of Face Recognition
We propose a methodology for assessing and characterizing the robustness of FRMs against semantic perturbations to their input.
On the Robustness of Quality Measures for GANs
Finally, we show that the FID can be robustified by simply replacing the standard Inception with a robust Inception.
vCLIMB: A Novel Video Class Incremental Learning Benchmark
We perform in-depth evaluations of existing CL methods in vCLIMB, and observe two unique challenges in video data.
Spatio-temporal Relation Modeling for Few-shot Action Recognition
Experiments are performed on four few-shot action recognition benchmarks: Kinetics, SSv2, HMDB51 and UCF101.
Ranked #1 on
Few Shot Action Recognition
on HMDB51
Low-Fidelity Video Encoder Optimization for Temporal Action Localization
This results in a task discrepancy problem for the video encoder – trained for action classification, but used for TAL.
Ranked #6 on
Temporal Action Localization
on HACS
MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions
The recent and increasing interest in video-language research has driven the development of large-scale datasets that enable data-intensive machine learning techniques.
Ranked #2 on
Natural Language Moment Retrieval
on MAD
Voint Cloud: Multi-View Point Cloud Representation for 3D Understanding
To this end, we introduce the concept of the multi-view point cloud (Voint cloud), representing each 3D point as a set of features extracted from several view-points.
ASSANet: An Anisotropic Separable Set Abstraction for Efficient Point Cloud Representation Learning
We then introduce a new Anisotropic Reduction function into our Separable SA module and propose an Anisotropic Separable SA (ASSA) module that substantially increases the network's accuracy.
Ranked #27 on
Semantic Segmentation
on S3DIS Area5
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
Relation-aware Video Reading Comprehension for Temporal Language Grounding
Temporal language grounding in videos aims to localize the temporal span relevant to the given query sentence.
Check Your Other Door! Creating Backdoor Attacks in the Frequency Domain
Deep Neural Networks (DNNs) are ubiquitous and span a variety of applications ranging from image classification to real-time object detection.
MovieCuts: A New Dataset and Benchmark for Cut Type Recognition
Advances in automatic Cut-type recognition can unleash new experiences in the video editing industry, such as movie analysis for education, video re-editing, virtual cinematography, machine-assisted trailer generation, machine-assisted video editing, among others.
Learning to Cut by Watching Movies
Video content creation keeps growing at an incredible pace; yet, creating engaging stories remains challenging and requires non-trivial video editing expertise.
Enhancing Adversarial Robustness via Test-time Transformation Ensembling
Deep learning models are prone to being fooled by imperceptible perturbations known as adversarial attacks.
ANCER: Anisotropic Certification via Sample-wise Volume Maximization
Randomized smoothing has recently emerged as an effective tool that enables certification of deep neural network classifiers at scale.
DeformRS: Certifying Input Deformations with Randomized Smoothing
Deep neural networks are vulnerable to input deformations in the form of vector fields of pixel displacements and to other parameterized geometric deformations e. g. translations, rotations, etc.
Training Graph Neural Networks with 1000 Layers
Deep graph neural networks (GNNs) have achieved excellent results on various tasks on increasingly large graph datasets with millions of nodes and edges.
Ranked #1 on
Node Property Prediction
on ogbn-proteins
APES: Audiovisual Person Search in Untrimmed Video
To showcase the potential of our new dataset, we propose an audiovisual baseline and benchmark for person retrieval.
SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation
We propose a novel scene flow estimation approach to capture and infer 3D motions from point clouds.
Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spotting
Specifically, we distill a powerful commercial calibration tool in a recent neural network architecture on the large-scale SoccerNet dataset, composed of untrimmed broadcast videos of 500 soccer games.
Temporally-Aware Feature Pooling for Action Spotting in Soccer Broadcasts
In this paper, we focus our analysis on action spotting in soccer broadcast, which consists in temporally localizing the main actions in a soccer game.
Ranked #7 on
Action Spotting
on SoccerNet-v2
(Average-mAP metric)
Low-Fidelity End-to-End Video Encoder Pre-training for Temporal Action Localization
This results in a task discrepancy problem for the video encoder -- trained for action classification, but used for TAL.
Combating Adversaries with Anti-Adversaries
Deep neural networks are vulnerable to small input perturbations known as adversarial attacks.
AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised Anime Face Generation
Specifically, a new generator architecture is proposed to simultaneously transfer color/texture styles and transform local facial shapes into anime-like counterparts based on the style of a reference anime-face, while preserving the global structure of the source photo-face.
MAAS: Multi-modal Assignation for Active Speaker Detection
Active speaker detection requires a solid integration of multi-modal cues.
DeeperGCN: Training Deeper GCNs with Generalized Aggregation Functions
We add our generalized aggregation into a deep GCN framework and show it achieves state-of-the-art results in six benchmarks from OGB.
On the Decision Boundaries of Neural Networks. A Tropical Geometry Perspective
This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piecewise linear non-linearity activations.
High Quality Disparity Remapping With Two-Stage Warping
In the second stage, we derive another warping model to refine warping results in less important regions by eliminating serious distortions in shape, disparity and 3D structure.
Vocal Bursts Intensity Prediction
Vocal Bursts Valence Prediction
SALA: Soft Assignment Local Aggregation for Parameter Efficient 3D Semantic Segmentation
Since it is learnable, this mapping is allowed to be different per layer instead of being applied uniformly throughout the depth of the network.
Data-Dependent Randomized Smoothing
In this work, we revisit Gaussian randomized smoothing and show that the variance of the Gaussian distribution can be optimized at each input so as to maximize the certification radius for the construction of the smooth classifier.
Video Self-Stitching Graph Network for Temporal Action Localization
In VSS, we focus on a short period of a video and magnify it along the temporal dimension to obtain a larger scale.
Ranked #11 on
Temporal Action Localization
on ActivityNet-1.3
MVTN: Multi-View Transformation Network for 3D Shape Recognition
MVTN exhibits clear performance gains in the tasks of 3D shape classification and 3D shape retrieval without the need for extra training supervision.
Ranked #1 on
3D Object Retrieval
on ShapeNetCore 55
SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos
In this work, we propose SoccerNet-v2, a novel large-scale corpus of manual annotations for the SoccerNet video dataset, along with open challenges to encourage more research in soccer understanding and broadcast production.
Ranked #1 on
Camera shot segmentation
on SoccerNet-v2
TSP: Temporally-Sensitive Pretraining of Video Encoders for Localization Tasks
Extensive experiments show that using features trained with our novel pretraining strategy significantly improves the performance of recent state-of-the-art methods on three tasks: Temporal Action Localization, Action Proposal Generation, and Dense Video Captioning.
Boundary-sensitive Pre-training for Temporal Localization in Videos
However, most existing models developed for these tasks are pre-trained on general video action classification tasks.
Ranked #17 on
Temporal Action Localization
on ActivityNet-1.3
VLG-Net: Video-Language Graph Matching Network for Video Grounding
Grounding language queries in videos aims at identifying the time interval (or moment) semantically relevant to a language query.
Ranked #1 on
Natural Language Moment Retrieval
on TACoS
Robust Optimization as Data Augmentation for Large-scale Graphs
Data augmentation helps neural networks generalize better by enlarging the training set, but it remains an open question how to effectively augment graph data to enhance the performance of GNNs (Graph Neural Networks).
Ranked #1 on
Graph Property Prediction
on ogbg-ppa
LC-NAS: Latency Constrained Neural Architecture Search for Point Cloud Networks
In this paper, we introduce a new NAS framework, dubbed LC-NAS, where we search for point cloud architectures that are constrained to a target latency.
The End-of-End-to-End: A Video Understanding Pentathlon Challenge (2020)
This report summarizes the results of the first edition of the challenge together with the findings of the participants.
Learning Heat Diffusion for Network Alignment
EDNA uses the diffusion signal as a proxy for computing node similarities between networks.
Network Moments: Extensions and Sparse-Smooth Attacks
This has incited research on the reaction of DNNs to noisy input, namely developing adversarial input attacks and strategies that lead to robust DNNs to these attacks.
DeeperGCN: All You Need to Train Deeper GCNs
Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs.
Ranked #1 on
Node Property Prediction
on ogbn-proteins
Rethinking Clustering for Robustness
This paper studies how encouraging semantically-aligned features during deep neural network training can increase network robustness.
Active Speakers in Context
Current methods for active speak er detection focus on modeling short-term audiovisual information from a single speaker.
Adaptive Learning of the Optimal Batch Size of SGD
Recent advances in the theoretical understanding of SGD led to a formula for the optimal batch size minimizing the number of effective data passes, i. e., the number of iterations times the batch size.
On the Decision Boundaries of Neural Networks: A Tropical Geometry Perspective
Our main finding is that the decision boundaries are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes.
RGB-based Semantic Segmentation Using Self-Supervised Depth Pre-Training
These labels, denoted by HN-labels, represent different height and normal patches, which allow mining of local semantic information that is useful in the task of semantic RGB segmentation.
Ranked #92 on
Semantic Segmentation
on NYU Depth v2
Analytical Moment Regularizer for Training Robust Networks
In this work, we propose a new training regularizer that aims to minimize the probabilistic expected training loss of a DNN subject to a generic Gaussian input.
Gabor Layers Enhance Network Robustness
We revisit the benefits of merging classical vision concepts with deep learning models.
A Context-Aware Loss Function for Action Spotting in Soccer Videos
We benchmark our loss on a large dataset of soccer videos, SoccerNet, and achieve an improvement of 12. 8% over the baseline.
Ranked #3 on
Action Spotting
on SoccerNet
AdvPC: Transferable Adversarial Perturbations on 3D Point Clouds
Our proposed attack increases the attack success rate by up to 40% for those transferred to unseen networks (transferability), while maintaining a high success rate on the attacked network.
SGAS: Sequential Greedy Architecture Search
Architecture design has become a crucial component of successful deep learning.
Ranked #4 on
Node Classification
on PPI
PU-GCN: Point Cloud Upsampling using Graph Convolutional Networks
We combine Inception DenseGCN with NodeShuffle into a new point upsampling pipeline called PU-GCN.
Assessing the Robustness of Visual Question Answering Models
In this work, we propose a new method that uses semantically related questions, dubbed basic questions, acting as noise to evaluate the robustness of VQA models.
Self-Supervised Learning by Cross-Modal Audio-Video Clustering
To the best of our knowledge, XDC is the first self-supervised learning method that outperforms large-scale fully-supervised pretraining for action recognition on the same architecture.
PointRGCN: Graph Convolution Networks for 3D Vehicles Detection Refinement
We integrate residual GCNs in a two-stage 3D object detection pipeline, where 3D object proposals are refined using a novel graph representation.
Ranked #14 on
3D Object Detection
on KITTI Cars Hard
G-TAD: Sub-Graph Localization for Temporal Action Detection
In this work, we propose a graph convolutional network (GCN) model to adaptively incorporate multi-level semantic context into video features and cast temporal action detection as a sub-graph localization problem.
Ranked #4 on
Temporal Action Localization
on FineAction
DeepGCNs: Making GCNs Go as Deep as CNNs
This work transfers concepts such as residual/dense connections and dilated convolutions from CNNs to GCNs in order to successfully train very deep GCNs.
Ranked #5 on
3D Semantic Segmentation
on PartNet
Expected Tight Bounds for Robust Deep Neural Network Training
al. that bounded input intervals can be inexpensively propagated from layer to layer through deep networks.
On the Decision Boundaries of Deep Neural Networks: A Tropical Geometry Perspective
We use tropical geometry, a new development in the area of algebraic geometry, to provide a characterization of the decision boundaries of a simple neural network of the form (Affine, ReLU, Affine).
Finding Moments in Video Collections Using Natural Language
We evaluate our approach on two recently proposed datasets for temporal localization of moments in video with natural language (DiDeMo and Charades-STA) extended to our video corpus moment retrieval setting.
Constrained Clustering: General Pairwise and Cardinality Constraints
Extensive experiments on both synthetic and real data demonstrate when: (1) utilizing a single category of constraint, the proposed model is superior to or competitive with SOTA constrained clustering models, and (2) utilizing both categories of constraints jointly, the proposed model shows better performance than the case of the single category.
3D Instance Segmentation via Multi-Task Metric Learning
The second goal is to learn instance information by densely estimating directional information of the instance's center of mass for each voxel.
Ranked #2 on
3D Semantic Instance Segmentation
on ScanNetV2
Expected Tight Bounds for Robust Training
In this paper, we closely examine the bounds of a block of layers composed in the form of Affine-ReLU-Affine.
MAP Inference via L2-Sphere Linear Program Reformulation
Thus, LS-LP is equivalent to the original MAP inference problem.
Rethinking Learning-based Demosaicing, Denoising, and Super-Resolution Pipeline
In this work, we comprehensively study the effects of pipelines on the mixture problem of learning-based DN, DM, and SR, in both sequential and joint solutions.
Deep Layers as Stochastic Solvers
In particular, we show that a forward pass through a standard dropout layer followed by a linear layer and a non-linear activation is equivalent to optimizing a convex optimization objective with a single iteration of a $\tau$-nice Proximal Stochastic Gradient method.
Analytical Moment Regularizer for Gaussian Robust Networks
Despite the impressive performance of deep neural networks (DNNs) on numerous vision tasks, they still exhibit yet-to-understand uncouth behaviours.
Learning a Controller Fusion Network by Online Trajectory Filtering for Vision-based UAV Racing
A common approach is to learn an end-to-end policy that directly predicts controls from raw images by imitating an expert.
IAN: Combining Generative Adversarial Networks for Imaginative Face Generation
Generative Adversarial Networks (GANs) have gained momentum for their ability to model image distributions.
MAIN: Multi-Attention Instance Network for Video Segmentation
Instance-level video segmentation requires a solid integration of spatial and temporal information.
ThumbNet: One Thumbnail Image Contains All You Need for Recognition
Although deep convolutional neural networks (CNNs) have achieved great success in computer vision tasks, its real-world application is still impeded by its voracious demand of computational resources.
BAOD: Budget-Aware Object Detection
We adopt a hybrid supervised learning framework to train the object detector from both these types of annotation.
Towards Analyzing Semantic Robustness of Deep Neural Networks
Despite the impressive performance of Deep Neural Networks (DNNs) on various vision tasks, they still exhibit erroneous high sensitivity toward semantic primitives (e. g. object pose).
DeepGCNs: Can GCNs Go as Deep as CNNs?
Finally, we use these new concepts to build a very deep 56-layer GCN, and show how it significantly boosts performance (+3. 7% mIoU over state-of-the-art) in the task of point cloud semantic segmentation.
MortonNet: Self-Supervised Learning of Local Features in 3D Point Clouds
In fact, we show how Morton features can be used to significantly improve performance (+3% for 2 popular se