Search Results for author:
Found 26 papers, 12 papers with code
PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models
To this end, we extend each image into a "static" video and introduce a unified token compression strategy called Progressive Visual Token Compression (PVC), where the tokens of each frame are progressively encoded and adaptively compressed to supplement the information not extracted from previous frames.
Fine-Tuning Attention Modules Only: Enhancing Weight Disentanglement in Task Arithmetic
To further understand how our method improves the weight disentanglement of task arithmetic, we present a comprehensive study of task arithmetic by differentiating the role of the representation module and task-specific module.
Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression Learning
Recently, vision model pre-training has evolved from relying on manually annotated datasets to leveraging large-scale, web-crawled image-text data.
Shifted Interpolation for Differential Privacy
Notably, this leads to the first exact privacy analysis in the foundational setting of strongly convex optimization.
WildfireGPT: Tailored Large Language Model for Wildfire Analysis
Recent advancement of large language models (LLMs) represents a transformational capability at the frontier of artificial intelligence.
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs.
Ranked #1 on
Zero-Shot Video Retrieval
on MSR-VTT-full
(using extra training data)
Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory
These agents, equipped with the logic and common sense capabilities of LLMs, can skillfully navigate complex, sparse-reward environments with text-based interactions.
Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information
It has been proved that combining multiple pre-training strategies and data from various modalities/sources can greatly boost the training of large-scale models.
Ranked #2 on
Object Detection
on COCO test-dev
The alignment property of SGD noise and how it helps select flat minima: A stability analysis
In this paper, we provide an explanation of this striking phenomenon by relating the particular noise structure of SGD to its \emph{linear stability} (Wu et al., 2018).
You Are the Best Reviewer of Your Own Papers: The Isotonic Mechanism
Subsequently, the raw review scores are calibrated based on this ranking to produce adjusted scores.
Siamese Image Modeling for Self-Supervised Vision Representation Learning
Driven by these analysis, we propose Siamese Image Modeling (SiameseIM), which predicts the dense representations of an augmented view, based on another masked view from the same image but with different augmentations.
You Are the Best Reviewer of Your Own Papers: An Owner-Assisted Scoring Mechanism
To address this withholding of information, in this paper, I introduce the Isotonic Mechanism, a simple and efficient approach to improving on the imprecise raw scores by leveraging certain information that the owner is incentivized to provide.
Deformable DETR: Deformable Transformers for End-to-End Object Detection
DETR has been recently proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance.
Ranked #7 on
2D Object Detection
on SARDet-100K
Benign Overfitting and Noisy Features
Modern machine learning often operates in the regime where the number of parameters is much higher than the number of data points, with zero training loss and yet good generalization, thereby contradicting the classical bias-variance trade-off.
Robust Learning Rate Selection for Stochastic Optimization via Splitting Diagnostic
This paper proposes SplitSGD, a new dynamic learning rate schedule for stochastic optimization.
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short).
Ranked #1 on
Visual Question Answering (VQA)
on VCR (Q-A) dev
Algorithmic Analysis and Statistical Estimation of SLOPE via Approximate Message Passing
SLOPE is a relatively new convex optimization procedure for high-dimensional linear regression via the sorted l1 penalty: the larger the rank of the fitted coefficient, the larger the penalty.
Statistical Inference for the Population Landscape via Moment Adjusted Stochastic Gradients
Modern statistical inference tasks often require iterative optimization methods to compute the solution.
Group SLOPE - adaptive selection of groups of predictors
Sorted L-One Penalized Estimation (SLOPE) is a relatively new convex optimization procedure which allows for adaptive selection of regressors under sparse high dimensional designs.
Methodology 46N10 G.1.6
False Discoveries Occur Early on the Lasso Path
In regression settings where explanatory variables have very low correlations and there are relatively few effects, each of large magnitude, we expect the Lasso to find the important variables with few errors, if any.
Communication-Efficient False Discovery Rate Control via Knockoff Aggregation
The false discovery rate (FDR)---the expected fraction of spurious discoveries among all the discoveries---provides a popular statistical assessment of the reproducibility of scientific studies in various disciplines.
SLOPE is Adaptive to Unknown Sparsity and Asymptotically Minimax
We consider high-dimensional sparse regression problems in which we observe $y = X \beta + z$, where $X$ is an $n \times p$ design matrix and $z$ is an $n$-dimensional vector of independent Gaussian errors, each with variance $\sigma^2$.
Statistics Theory Information Theory Information Theory Statistics Theory
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
We derive a second-order ordinary differential equation (ODE) which is the limit of Nesterov's accelerated gradient method.
A Differential Equation for Modeling Nesterov’s Accelerated Gradient Method: Theory and Insights
We derive a second-order ordinary differential equation (ODE), which is the limit of Nesterov’s accelerated gradient method.
SLOPE - Adaptive variable selection via convex optimization
SLOPE, short for Sorted L-One Penalized Estimation, is the solution to \[\min_{b\in\mathbb{R}^p}\frac{1}{2}\Vert y-Xb\Vert _{\ell_2}^2+\lambda_1\vert b\vert _{(1)}+\lambda_2\vert b\vert_{(2)}+\cdots+\lambda_p\vert b\vert_{(p)},\] where $\lambda_1\ge\lambda_2\ge\cdots\ge\lambda_p\ge0$ and $\vert b\vert_{(1)}\ge\vert b\vert_{(2)}\ge\cdots\ge\vert b\vert_{(p)}$ are the decreasing absolute values of the entries of $b$.
Methodology
