Search Results for author:
Found 51 papers, 26 papers with code
Camera-Based Remote Physiology Sensing for Hundreds of Subjects Across Skin Tones
Remote photoplethysmography (rPPG) emerges as a promising method for non-invasive, convenient measurement of vital signs, utilizing the widespread presence of cameras.
HeadEvolver: Text to Head Avatars via Locally Learnable Mesh Deformation
We present HeadEvolver, a novel framework to generate stylized head avatars from text guidance.
SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting
We present SplattingAvatar, a hybrid 3D representation of photorealistic human avatars with Gaussian Splatting embedded on a triangle mesh, which renders over 300 FPS on a modern GPU and 30 FPS on a mobile device.
MemoNav: Working Memory Model for Visual Navigation
Subsequently, a graph attention module encodes the retained STM and the LTM to generate working memory (WM) which contains the scene features essential for efficient navigation.
Multi-Human Mesh Recovery with Transformers
Conventional approaches to human mesh recovery predominantly employ a region-based strategy.
HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification
The best-performing models aim to learn a static representation by combining document and hierarchical label information.
BoNuS: Boundary Mining for Nuclei Segmentation with Partial Point Labels
To alleviate this problem, in this paper, we propose a weakly-supervised nuclei segmentation method that only requires partial point labels of nuclei.
Revisiting Adversarial Training at Scale
For example, by training on DataComp-1B dataset, our AdvXL empowers a vanilla ViT-g model to substantially surpass the previous records of $l_{\infty}$-, $l_{2}$-, and $l_{1}$-robust accuracy by margins of 11. 4%, 14. 2% and 12. 9%, respectively.
Multi-Modal Representation Learning for Molecular Property Prediction: Sequence, Graph, Geometry
Molecular property prediction refers to the task of labeling molecules with some biochemical properties, playing a pivotal role in the drug discovery and design process.
A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In Distributional Reinforcement Learning
Distributional Reinforcement Learning (RL) estimates return distribution mainly by learning quantile values via minimizing the quantile Huber loss function, entailing a threshold parameter often selected heuristically or via hyperparameter search, which may not generalize well and can be suboptimal.
Voila-A: Aligning Vision-Language Models with User's Gaze Attention
In this paper, we introduce gaze information, feasibly collected by AR or VR devices, as a proxy for human attention to guide VLMs and propose a novel approach, Voila-A, for gaze alignment to enhance the interpretability and effectiveness of these models in real-world applications.
MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual Storytelling via Multi-Layered Semantic-Aware Denoising
Visual storytelling often uses nontypical aspect-ratio images like scroll paintings, comic strips, and panoramas to create an expressive and compelling narrative.
Dietary Assessment with Multimodal ChatGPT: A Systematic Analysis
Although artificial intelligence (AI)-based solutions have been devised to automate the dietary assessment process, these prior AI methodologies encounter challenges in their ability to generalize across a diverse range of food types, dietary behaviors, and cultural contexts.
Rejuvenating image-GPT as Strong Visual Representation Learners
This paper enhances image-GPT (iGPT), one of the pioneering works that introduce autoregressive pretraining to predict next pixels for visual representation learning.
Sculpting Holistic 3D Representation in Contrastive Language-Image-3D Pre-training
Contrastive learning has emerged as a promising paradigm for 3D open-world understanding, i. e., aligning point cloud representation to image and text embedding space individually.
Ranked #1 on
Zero-shot 3D classification
on Objaverse LVIS
(using extra training data)
FedConv: Enhancing Convolutional Neural Networks for Handling Data Heterogeneity in Federated Learning
Federated learning (FL) is an emerging paradigm in machine learning, where a shared model is collaboratively learned using data from multiple devices to mitigate the risk of data leakage.
Few-Shot Multi-Label Aspect Category Detection Utilizing Prototypical Network with Sentence-Level Weighting and Label Augmentation
Since aspect category detection often suffers from limited datasets and data sparsity, the prototypical network with attention mechanisms has been applied for few-shot aspect category detection.
DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillation
In this work, we propose to boost the representation learning of a multi-camera BEV based student detector by training it to imitate the features of a well-trained LiDAR based teacher detector.
CLIPA-v2: Scaling CLIP Training with 81.1% Zero-shot ImageNet Accuracy within a \$10,000 Budget; An Extra \$4,000 Unlocks 81.8% Accuracy
The recent work CLIPA presents an inverse scaling law for CLIP training -- whereby the larger the image/text encoders used, the shorter the sequence length of image/text tokens that can be applied in training.
Subgraph Networks Based Contrastive Learning
In addition, we also investigate the impact of the second-order subgraph augmentation on mining graph structure interactions, and further, propose a contrastive objective that fuses the first-order and second-order subgraph information.
ZeroAvatar: Zero-shot 3D Avatar Generation from a Single Image
Recent advancements in text-to-image generation have enabled significant progress in zero-shot 3D shape generation.
SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
Our evaluation shows that SpecInfer outperforms existing LLM serving systems by 1. 5-2. 8x for distributed LLM inference and by 2. 6-3. 5x for offloading-based LLM inference, while preserving the same generative performance.
An Inverse Scaling Law for CLIP Training
However, its associated training cost is prohibitively high, imposing a significant barrier to its widespread exploration.
Click-Feedback Retrieval
In this work, we study a setting where the feedback is provided through users clicking liked and disliked searching results.
A robust method for reliability updating with equality information using sequential adaptive importance sampling
Reliability updating refers to a problem that integrates Bayesian updating technique with structural reliability analysis and cannot be directly solved by structural reliability methods (SRMs) when it involves equality information.
On the Adversarial Robustness of Camera-based 3D Object Detection
In recent years, camera-based 3D object detection has gained widespread attention for its ability to achieve high performance with low computational cost.
Thermal Infrared Image Inpainting via Edge-Aware Guidance
In this paper, we propose a novel task -- Thermal Infrared Image Inpainting, which aims to reconstruct missing regions of TIR images.
Bag of Tricks for FGSM Adversarial Training
Adversarial training (AT) with samples generated by Fast Gradient Sign Method (FGSM), also known as FGSM-AT, is a computationally simple method to train robust networks.
Masked Autoencoders Enable Efficient Knowledge Distillers
For example, by distilling the knowledge from an MAE pre-trained ViT-L into a ViT-B, our method achieves 84. 0% ImageNet top-1 accuracy, outperforming the baseline of directly distilling a fine-tuned ViT-L by 1. 2%.
Predicting Word Learning in Children from the Performance of Computer Vision Systems
The performance of the computer vision systems is correlated with human judgments of the concreteness of words, which are in turn a predictor of children's word learning, suggesting that these models are capturing the relationship between words and visual phenomena.
InsMix: Towards Realistic Generative Data Augmentation for Nuclei Instance Segmentation
Nuclei Segmentation from histology images is a fundamental task in digital pathology analysis.
Learning Optimal Treatment Strategies for Sepsis Using Offline Reinforcement Learning in Continuous Space
Sepsis is a leading cause of death in the ICU.
Can CNNs Be More Robust Than Transformers?
The recent success of Vision Transformers is shaking the long dominance of Convolutional Neural Networks (CNNs) in image recognition for a decade.
Gamma and Vega Hedging Using Deep Distributional Reinforcement Learning
We show how D4PG can be used in conjunction with quantile regression to develop a hedging strategy for a trader responsible for derivatives that arrive stochastically and depend on a single underlying asset.
Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection
Language models increasingly rely on massive web dumps for diverse text data.
Multi-Query Video Retrieval
Retrieving target videos based on text descriptions is a task of great practical value and has received increasing attention over the past few years.
Human Activity Recognition Using 3D Orthogonally-projected EfficientNet on Radar Time-Range-Doppler Signature
In radar activity recognition, 2D signal representations such as spectrogram, cepstrum and cadence velocity diagram are often utilized, while range information is often neglected.
Chi-square Loss for Softmax: an Echo of Neural Network Structure
In addition, we studied the sample distribution of this loss function by visualization and found that the distribution is related to the neural network structure, which is distinct compared to cross-entropy.
Semi-Supervised Crowd Counting from Unlabeled Data
Our method achieved competitive performance in semi-supervised learning approaches on these crowd counting datasets.
maplet: An extensible R toolbox for modular and reproducible omics pipelines
This paper presents maplet, an open-source R package for the creation of highly customizable, fully reproducible statistical pipelines for omics data analysis, with a special focus on metabolomics-based methods.
Plot2API: Recommending Graphic API from Plot via Semantic Parsing Guided Neural Network
Considering the lack of technologies in Plot2API, we present a novel deep multi-task learning approach named Semantic Parsing Guided Neural Network (SPGNN) which translates the Plot2API issue as a multi-label image classification and an image semantic parsing tasks for the solution.
Towards Unique and Informative Captioning of Images
We find that modern captioning systems return higher likelihoods for incorrect distractor sentences compared to ground truth captions, and that evaluation metrics like SPICE can be 'topped' using simple captioning systems relying on object detectors.
Value of Information Analysis via Active Learning and Knowledge Sharing in Error-Controlled Adaptive Kriging
The proposed VoI analysis framework is applied for an optimal decision-making problem involving load testing of a truss bridge.
REAK: Reliability analysis through Error rate-based Adaptive Kriging
Reliability analysis for these systems when failure probabilities are small is significantly challenging, requiring a large number of costly simulations.
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation
We design a simple but surprisingly effective visual recognition benchmark for studying bias mitigation.
Ranked #1 on
Out-of-Distribution Generalization
on UrbanCars
SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines
Following these guidelines, we design our Fully Convolutional Siamese tracker++ (SiamFC++) by introducing both classification and target state estimation branch(G1), classification score without ambiguity(G2), tracking without prior knowledge(G3), and estimation quality score(G4).
Ranked #2 on
Visual Object Tracking
on VOT2017/18
(using extra training data)
Self-supervised Learning of Detailed 3D Face Reconstruction
The displacement map and the coarse model are used to render a final detailed face, which again can be compared with the original input image to serve as a photometric loss for the second stage.
SCANN: Synthesis of Compact and Accurate Neural Networks
To address these challenges, we propose a two-step neural network synthesis methodology, called DR+SCANN, that combines two complementary approaches to design compact and accurate DNNs.
Multi-band Weighted $l_p$ Norm Minimization for Image Denoising
Low rank matrix approximation (LRMA) has drawn increasing attention in recent years, due to its wide range of applications in computer vision and machine learning.
A Data-driven Adversarial Examples Recognition Framework via Adversarial Feature Genome
In this regard, we propose the Adversarial Feature Genome (AFG), a novel type of data that contains both the differences and features about classes.
AniCode: Authoring Coded Artifacts for Network-Free Personalized Animations
We introduce AniCode, a novel framework for authoring and consuming time-based media.
Graphics
