Search Results for author:
Found 24 papers, 17 papers with code
EdgeTAM: On-Device Track Anything Model
Given that video segmentation is a dense prediction task, we find preserving the spatial structure of the memories is essential so that the queries are split into global-level and patch-level groups.
MetaMorph: Multimodal Understanding and Generation via Instruction Tuning
Our results suggest that LLMs may have strong "prior" vision capabilities that can be efficiently adapted to both visual understanding and generation with a relatively simple instruction tuning process.
Efficient Track Anything
The high computation complexity of multistage image encoder and memory module has limited its applications in real-world tasks, e. g., video object segmentation on mobile devices.
LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding
Given a light-weight LLM, our LongVU also scales effectively into a smaller size with state-of-the-art video understanding performance.
Agent-as-a-Judge: Evaluate Agents with Agents
To address this, we introduce the Agent-as-a-Judge framework, wherein agentic systems are used to evaluate agentic systems.
An Introduction to Vision-Language Modeling
Then, we present and discuss approaches to evaluate VLMs.
MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
The resultant models, denoted as MobileLLM-LS, demonstrate a further accuracy enhancement of 0. 7%/0. 8% than MobileLLM 125M/350M.
SqueezeSAM: User friendly mobile interactive segmentation
A common user expectation is that a click on a specific part of an object will result in the segmentation of the entire object.
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything
On segment anything task such as zero-shot instance segmentation, our EfficientSAMs with SAMI-pretrained lightweight image encoders perform favorably with a significant gain (e. g., ~4 AP on COCO/LVIS) over other fast SAM models.
Ranked #3 on
Zero-Shot Instance Segmentation
on LVIS v1.0 val
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning
Motivated by this, we target to build a unified interface for completing many vision-language tasks including image description, visual question answering, and visual grounding, among others.
Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts
In NLP tasks like machine translation and pre-trained language modeling, there is a significant performance gap between supernet and training from scratch for the same model architecture, necessitating retraining post optimal architecture identification.
Self-positioning Point-based Transformer for Point Cloud Understanding
In this paper, we present a Self-Positioning point-based Transformer (SPoTr), which is designed to capture both local and global shape contexts with reduced complexity.
Ranked #4 on
3D Part Segmentation
on ShapeNet-Part
PathFusion: Path-consistent Lidar-Camera Deep Feature Fusion
Fusing 3D LiDAR features with 2D camera features is a promising technique for enhancing the accuracy of 3D detection, thanks to their complementary physical properties.
Fast Point Cloud Generation with Straight Flows
We perform evaluations on multiple 3D tasks and find that our PSF performs comparably to the standard diffusion model, outperforming other efficient 3D point cloud generation methods.
Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference
Vision Transformers (ViTs) have shown impressive performance but still require a high computation cost as compared to convolutional neural networks (CNNs), one reason is that ViTs' attention measures global similarities and thus has a quadratic complexity with the number of input tokens.
SageMix: Saliency-Guided Mixup for Point Clouds
Mixup is a simple and widely-used data augmentation technique that has proven effective in alleviating the problems of overfitting and data scarcity.
Ranked #42 on
3D Part Segmentation
on ShapeNet-Part
You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling
In this paper, we show that a Bernoulli sampling attention mechanism based on Locality Sensitive Hashing (LSH), decreases the quadratic complexity of such models to linear.
Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention
The scalability of Nystr\"{o}mformer enables application to longer sequences with thousands of tokens.
Ranked #13 on
Semantic Textual Similarity
on MRPC
(F1 metric)
MobileDets: Searching for Object Detection Architectures for Mobile Accelerators
By incorporating regular convolutions in the search space and directly optimizing the network architectures for object detection, we obtain a family of object detection models, MobileDets, that achieve state-of-the-art results across mobile accelerators.
Mixed Effects Neural Networks (MeNets) With Applications to Gaze Estimation
nature of this data suggests better estimation may be possible if the model explicitly made use of such "repeated measurements" from each user as is commonly done in classical statistical analysis using so-called mixed effects models.
Resource Constrained Neural Network Architecture Search: Will a Submodularity Assumption Help?
In the latter case, the optimization is often non-differentiable and also not very amenable to derivative-free optimization methods.
ANTNets: Mobile Convolutional Neural Networks for Resource Efficient Image Classification
It boosts the representational power by modeling, in a high dimensional space, interdependency of channels between a depthwise convolution layer and a projection layer in the ANTBlocks.
Building Bayesian Neural Networks with Blocks: On Structure, Interpretability and Uncertainty
We provide simple schemes to build Bayesian Neural Networks (BNNs), block by block, inspired by a recent idea of computation skeletons.
Filter Flow Made Practical: Massively Parallel and Lock-Free
This paper is inspired by a relatively recent work of Seitz and Baker which introduced the so-called Filter Flow model.
