Search Results for author:
Found 462 papers, 136 papers with code
Building Goal-oriented Document-grounded Dialogue Systems
In this paper, we describe our systems for solving the two Doc2Dial shared task: knowledge identification and response generation.
Domain-specific knowledge distillation yields smaller and better models for conversational commerce
We demonstrate that knowledge distillation can be used not only to reduce model size, but to simultaneously adapt a contextual language model to a specific domain.
SemEval-2022 Task 8: Multilingual news article similarity
Thousands of new news articles appear daily in outlets in different languages.
A Three-step Method for Multi-Hop Inference Explanation Regeneration
Then, a selection module is employed to choose those most relative facts in an autoregressive manner, giving a preliminary order for the retrieved facts.
MedSAM-CA: A CNN-Augmented ViT with Attention-Enhanced Multi-Scale Fusion for Medical Image Segmentation
Medical image segmentation plays a crucial role in clinical diagnosis and treatment planning, where accurate boundary delineation is essential for precise lesion localization, organ identification, and quantitative assessment.
PhysUniBench: An Undergraduate-Level Physics Reasoning Benchmark for Multimodal Models
These results highlight that current MLLMs struggle with advanced physics reasoning, especially on multi-step problems and those requiring precise diagram interpretation.
TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
However, it is challenging to leverage such token-level reward as guidance for Direct Preference Optimization (DPO), since DPO is formulated as a sequence-level bandit problem.
FocalClick-XL: Towards Unified and High-quality Interactive Segmentation
Interactive segmentation enables users to extract binary masks of target objects through simple interactions such as clicks, scribbles, and boxes.
MultiFinBen: A Multilingual, Multimodal, and Difficulty-Aware Benchmark for Financial LLM Evaluation
We introduce MultiFinBen, the first multilingual and multimodal benchmark tailored to the global financial domain, evaluating LLMs across modalities (text, vision, audio) and linguistic settings (monolingual, bilingual, multilingual) on domain-specific tasks.
Zero-shot denoising via neural compression: Theoretical and algorithmic framework
Zero-shot denoising aims to denoise observations without access to training samples or clean reference images.
ChineseHarm-Bench: A Chinese Harmful Content Detection Benchmark
In addition, we propose a knowledge-augmented baseline that integrates both human-annotated knowledge rules and implicit knowledge from large language models, enabling smaller models to achieve performance comparable to state-of-the-art LLMs.
ConStyX: Content Style Augmentation for Generalizable Medical Image Segmentation
Medical images are usually collected from multiple domains, leading to domain shifts that impair the performance of medical image segmentation models.
PlayerOne: Egocentric World Simulator
Experimental results demonstrate its great generalization ability in precise control of varying human movements and worldconsistent modeling of diverse scenarios.
Step-Audio-AQAA: a Fully End-to-End Expressive Large Audio Language Model
This work contributes a promising solution for end-to-end LALMs and highlights the critical role of token-based vocoder in enhancing overall performance for AQAA tasks.
STELLA: Towards Protein Function Prediction with Multimodal LLMs Integrating Sequence-Structure Representations
Since protein sequences determine tertiary structures, which in turn govern functions, integrating sequence and structure information is essential for accurate prediction of protein functions.
Multilook Coherent Imaging: Theoretical Guarantees and Algorithms
In this paper, we study both the theoretical and algorithmic aspects of likelihood-based approaches for multilook coherent imaging, providing a rigorous framework for analysis and method development.
ScEdit: Script-based Assessment of Knowledge Editing
Knowledge Editing (KE) has gained increasing attention, yet current KE tasks remain relatively simple.
CityGo: Lightweight Urban Modeling and Rendering with Proxy Buildings and Residual Gaussians
We propose CityGo, a hybrid framework that combines textured proxy geometry with residual and surrounding 3D Gaussians for lightweight, photorealistic rendering of urban scenes from aerial perspectives.
Unleashing Diffusion Transformers for Visual Correspondence by Modulating Massive Activations
Experimental results demonstrate that our DiTF outperforms both DINO and SD-based models and establishes a new state-of-the-art performance for DiTs in different visual correspondence tasks (\eg, with +9. 4\% on Spair-71k and +4. 4\% on AP-10K-C. S.
LFTF: Locating First and Then Fine-Tuning for Mitigating Gender Bias in Large Language Models
To better explore and quantifying the degree of gender bias in LLMs, we propose a pair of datasets named GenBiasEval and GenHintEval, respectively.
Generalizable Multispectral Land Cover Classification via Frequency-Aware Mixture of Low-Rank Token Experts
Existing methods predominantly rely on domain adaptation and generalization strategies, often utilizing small-scale models that exhibit limited performance.
A 2D Semantic-Aware Position Encoding for Vision Transformers
Traditional approaches like absolute position encoding and relative position encoding primarily focus on 1D linear position relationship, often neglecting the semantic similarity between distant yet contextually related patches.
An Identifiable Cost-Aware Causal Decision-Making Framework Using Counterfactual Reasoning
Emphasis is placed on making counterfactual reasoning processes identifiable in the presence of a large amount of mixed anomaly data, as well as finding the optimal intervention state in a continuous decision space.
ComPO: Preference Alignment via Comparison Oracles
Direct alignment methods are increasingly used for aligning large language models (LLMs) with human preferences.
Supporting renewable energy planning and operation with data-driven high-resolution ensemble weather forecast
The planning and operation of renewable energy, especially wind power, depend crucially on accurate, timely, and high-resolution weather information.
Rethinking Time Encoding via Learnable Transformation Functions
While previous methods focus on capturing time patterns, many rely on specific inductive biases, such as using trigonometric functions to model periodicity.
Debate-Feedback: A Multi-Agent Framework for Efficient Legal Judgment Prediction
The use of AI in legal analysis and prediction (LegalAI) has gained widespread attention, with past research focusing on retrieval-based methods and fine-tuning large models.
TextCrafter: Accurately Rendering Multiple Texts in Complex Visual Scenes
This paper explores the task of Complex Visual Text Generation (CVTG), which centers on generating intricate textual content distributed across diverse regions within visual images.
DeCompress: Denoising via Neural Compression
Learning-based denoising algorithms achieve state-of-the-art performance across various denoising tasks.
Gemma 3 Technical Report
We introduce Gemma 3, a multimodal addition to the Gemma family of lightweight open models, ranging in scale from 1 to 27 billion parameters.
Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Models
Existing TGNNs embed texts statically and rely heavily on encoding mechanisms that biasedly prioritize structural information, overlooking the temporal evolution of text semantics and the essential interplay between semantics and structures for synergistic reinforcement.
SciHorizon: Benchmarking AI-for-Science Readiness from Scientific Data to Large Language Models
In recent years, the rapid advancement of Artificial Intelligence (AI) technologies, particularly Large Language Models (LLMs), has revolutionized the paradigm of scientific discovery, establishing AI-for-Science (AI4Science) as a dynamic and evolving field.
Modular Customization of Diffusion Models via Blockwise-Parameterized Low-Rank Adaptation
In contrast, instant merging methods often cause identity loss and interference of individual merged concepts and are usually limited to a small number of concepts.
ObjectMover: Generative Object Movement with Video Prior
Simple as it seems, moving an object to another location within an image is, in fact, a challenging image-editing task that requires re-harmonizing the lighting, adjusting the pose based on perspective, accurately filling occluded regions, and ensuring coherent synchronization of shadows and reflections while maintaining the object identity.
GenieBlue: Integrating both Linguistic and Multimodal Capabilities for Large Language Models on Mobile Devices
Recent advancements in Multimodal Large Language Models (MLLMs) have enabled their deployment on mobile devices.
NoT: Federated Unlearning via Weight Negation
Federated unlearning (FU) aims to remove a participant's data contributions from a trained federated learning (FL) model, ensuring privacy and regulatory compliance.
Effective LLM Knowledge Learning via Model Generalization
We found and verified that knowledge learning for LLMs can be deemed as an implicit supervised task hidden in the autoregressive pre-training objective.
Generative assimilation and prediction for weather and climate
Machine learning models have shown great success in predicting weather up to two weeks ahead, outperforming process-based benchmarks.
Metering Error Estimation of Fast-Charging Stations Using Charging Data Analytics
In response, this paper adopts a data-driven approach and proposes the measuring performance comparison (MPC) method.
LMHLD: A Large-scale Multi-source High-resolution Landslide Dataset for Landslide Detection based on Deep Learning
However, DL models rely heavily on high-quality labeled landslide data for strong feature extraction capabilities.
Composable Strategy Framework with Integrated Video-Text based Large Language Models for Heart Failure Assessment
By integrating these various data sources, our framework offers a more holistic evaluation and optimized treatment plan for patients.
Monotonicity Testing of High-Dimensional Distributions with Subcube Conditioning
We study monotonicity testing of high-dimensional distributions on $\{-1, 1\}^n$ in the model of subcube conditioning, suggested and studied by Canonne, Ron, and Servedio~\cite{CRS15} and Bhattacharyya and Chakraborty~\cite{BC18}.
HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) systems often struggle with imperfect retrieval, as traditional retrievers focus on lexical or semantic similarity rather than logical relevance.
Exploring the Impact of Personality Traits on LLM Bias and Toxicity
The findings demonstrate the sensitivity of all three models to HEXACO personality traits and, more importantly, a consistent variation in the biases, negative sentiment and toxicity of their output.
Connector-S: A Survey of Connectors in Multi-modal Large Language Models
With the rapid advancements in multi-modal large language models (MLLMs), connectors play a pivotal role in bridging diverse modalities and enhancing model performance.
VRoPE: Rotary Position Embedding for Video Large Language Models
Rotary Position Embedding (RoPE) has shown strong performance in text-based Large Language Models (LLMs), but extending it to video remains a challenge due to the intricate spatiotemporal structure of video frames.
Step-Audio: Unified Understanding and Generation in Intelligent Speech Interaction
Based on our new StepEval-Audio-360 evaluation benchmark, Step-Audio achieves state-of-the-art performance in human evaluations, especially in terms of instruction following.
Dynamic Pricing with Adversarially-Censored Demands
We study an online dynamic pricing problem where the potential demand at each time period $t=1, 2,\ldots, T$ is stochastic and dependent on the price.
PH-VAE: A Polynomial Hierarchical Variational Autoencoder Towards Disentangled Representation Learning
The variational autoencoder (VAE) is a simple and efficient generative artificial intelligence method for modeling complex probability distributions of various types of data, such as images and texts.
Stochastic Linear Bandits with Latent Heterogeneity
Notably, our theoretical analysis reveals two distinct types of regret measures: a ``strong regret'' against an oracle with perfect knowledge of customer memberships, which remains non-sub-linear due to inherent classification uncertainty, and a ``regular regret'' against an oracle aware only of deterministic components, for which our algorithm achieves a sub-linear rate that is minimax optimal in horizon length and dimension.
A Bias-Correction Decentralized Stochastic Gradient Algorithm with Momentum Acceleration
Distributed stochastic optimization algorithms can simultaneously process large-scale datasets, significantly accelerating model training.
Heterogeneity-aware Personalized Federated Learning via Adaptive Dual-Agent Reinforcement Learning
This uniform LiteModel plays a pivotal role in aggregating and sharing global knowledge, significantly enhancing the effectiveness of personalized local training.
DiffDoctor: Diagnosing Image Diffusion Models Before Treating
In spite of the recent progress, image diffusion models still produce artifacts.
Disentangled Modeling of Preferences and Social Influence for Group Recommendation
The group recommendation (GR) aims to suggest items for a group of users in social networks.
MangaNinja: Line Art Colorization with Precise Reference Following
Derived from diffusion models, MangaNinjia specializes in the task of reference-guided line art colorization.
Large language models streamline automated systematic review: A preliminary study
We assessed GPT-4, Claude-3, and Mistral 8x7B across four systematic review tasks: study design formulation, search strategy development, literature screening, and data extraction.
Comprehensive Examination of Unrolled Networks for Solving Linear Inverse Problems
Unrolled networks have become prevalent in various computer vision and imaging tasks.
Integrated Offline and Online Learning to Solve a Large Class of Scheduling Problems
To overcome this difficulty, we generate and utilize a set of special instances, for which optimal solutions can be found with little computational effort, to train the ML model offline.
DreamMask: Boosting Open-vocabulary Panoptic Segmentation with Synthetic Data
For instance, when trained on COCO and tested on ADE20K, the model equipped with DreamMask outperforms the previous state-of-the-art by a substantial margin of 2. 1% mIoU.
Evidential Calibrated Uncertainty-Guided Interactive Segmentation paradigm for Ultrasound Images
Accurate and robust ultrasound image segmentation is critical for computer-aided diagnostic systems.
VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Control
It warps the pixel details according to the trajectories and fuses the warped features with the diffusion U-Net, thus improving detail preservation and supporting users in manipulating the motion trajectories.
EIDT-V: Exploiting Intersections in Diffusion Trajectories for Model-Agnostic, Zero-Shot, Training-Free Text-to-Video Generation
We use intersections in diffusion trajectories, working only with the latent values.
Embodied VideoAgent: Persistent Memory from Egocentric Videos and Embodied Sensors Enables Dynamic Scene Understanding
This paper investigates the problem of understanding dynamic 3D scenes from egocentric observations, a key challenge in robotics and embodied AI.
Online Video Understanding: OVBench and VideoChat-Online
Despite the lower computational cost and higher efficiency, VideoChat-Online outperforms existing state-of-the-art offline and online models across popular offline video benchmarks and OVBench, demonstrating the effectiveness of our model architecture and training strategy.
Ranked #2 on
Video Question Answering
on OVBench
Predicting Preschoolers' Externalizing Problems with Mother-Child Interaction Dynamics and Deep Learning
The deep learning model is a useful tool to further improve prediction accuracy.
Asynchronous Federated Clustering with Unknown Number of Clusters
However, some tricky but common FC problems are still relatively unexplored, including the heterogeneity in terms of clients' communication capacity and the unknown number of proper clusters $k^*$.
Performance limits of spectro-temporal unitary transformations for coherent modulation
We analyse the performance limits of coherent modulation based on lossless unitary transformations, demonstrating that they can achieve high ($>$30~dB) SINAD and outperform conventional IQ modulators at equivalent transmitter laser powers.
EnvGS: Modeling View-Dependent Appearance with Environment Gaussian
Reconstructing complex reflections in real-world scenes from 2D images is essential for achieving photorealistic novel view synthesis.
Reverse Region-to-Entity Annotation for Pixel-Level Visual Entity Linking
Additionally, the semantic tokenization method achieved a 5-point accuracy improvement over the trained baseline.
FashionComposer: Compositional Fashion Image Generation
We present FashionComposer for compositional fashion image generation.
UniReal: Universal Image Generation and Editing via Learning Real-world Dynamics
We introduce UniReal, a unified framework designed to address various image generation and editing tasks.
MotionStone: Decoupled Motion Intensity Modulation with Diffusion Transformer for Image-to-Video Generation
We then present a new I2V model, named MotionStone, developed with the decoupled motion estimator.
SyncVIS: Synchronized Video Instance Segmentation
Recent DETR-based methods have advanced the development of Video Instance Segmentation (VIS) through transformers' efficiency and capability in modeling spatial and temporal information.
CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
Unlike previous works that directly repurpose VLM for action prediction by simple action quantization, we propose a omponentized VLA architecture that has a specialized action module conditioned on VLM output.
From Diffusion to Resolution: Leveraging 2D Diffusion Models for 3D Super-Resolution Task
The model is then applied slice-by-slice in the lateral direction of low-resolution volume, recovering slices while preserving inherent lateral continuity.
TRACE: Temporal Grounding Video LLM via Causal Event Modeling
To effectively handle various tasks simultaneously and enable zero-shot prediction, there is a growing trend in employing video LLMs for VTG tasks.
Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?
In this way, we can preserve the low-rank constraint in the optimizer while achieving full-rank training for better performance.
LLM+KG@VLDB'24 Workshop Summary
The unification of large language models (LLMs) and knowledge graphs (KGs) has emerged as a hot topic.
Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness
Recently, there has been significant interest in replacing the reward model in Reinforcement Learning with Human Feedback (RLHF) methods for Large Language Models (LLMs), such as Direct Preference Optimization (DPO) and its variants.
AI Delegates with a Dual Focus: Ensuring Privacy and Strategic Self-Disclosure
Large language model (LLM)-based AI delegates are increasingly utilized to act on behalf of users, assisting them with a wide range of tasks through conversational interfaces.
Enhancing Long Video Understanding via Hierarchical Event-Based Memory
While this method excels in short video understanding, it may result in a blend of multiple event information in long videos due to coarse compression, which causes information redundancy.
TC-LLaVA: Rethinking the Transfer from Image to Video Understanding with Temporal Considerations
Multimodal Large Language Models (MLLMs) have significantly improved performance across various image-language applications.
FrozenSeg: Harmonizing Frozen Foundation Models for Open-Vocabulary Segmentation
To fully exploit pre-trained knowledge while minimizing training overhead, we freeze both foundation models, focusing optimization efforts solely on a lightweight transformer decoder for mask proposal generation-the performance bottleneck.
CLIBE: Detecting Dynamic Backdoors in Transformer-based NLP Models
CLIBE injects a "few-shot perturbation" into the suspect Transformer model by crafting optimized weight perturbation in the attention layers to make the perturbed model classify a limited number of reference samples as a target label.
Conan-embedding: General Text Embedding with More and Better Negative Samples
Embedding models are primarily trained through contrastive loss learning, with negative examples being a key component.
A Low-dose CT Reconstruction Network Based on TV-regularized OSEM Algorithm
Low-dose computed tomography (LDCT) offers significant advantages in reducing the potential harm to human bodies.
Self-Refined Generative Foundation Models for Wireless Traffic Prediction
Following this comprehensive feedback, our proposed TrafficLLM introduces refinement demonstration prompts, enabling the same LLM to further refine its predictions and thereby enhance prediction performance.
Learning Robust Treatment Rules for Censored Data
In this paper, we propose two robust criteria for learning optimal treatment rules with censored survival outcomes; the former one targets at an optimal treatment rule maximizing the restricted mean survival time, where the restriction is specified by a given quantile such as median; the latter one targets at an optimal treatment rule maximizing buffered survival probabilities, where the predetermined threshold is adjusted to account the restricted mean survival time.
Conformal Trajectory Prediction with Multi-View Data Integration in Cooperative Driving
In this work, we introduce V2INet, a novel trajectory prediction framework designed to model multi-view data by extending existing single-view models.
MSMA: Multi-agent Trajectory Prediction in Connected and Autonomous Vehicle Environment with Multi-source Data Integration
Our trajectory prediction task is aimed at all the detected surrounding vehicles.
DTFormer: A Transformer-Based Method for Discrete-Time Dynamic Graph Representation Learning
Discrete-Time Dynamic Graphs (DTDGs), which are prevalent in real-world implementations and notable for their ease of data acquisition, have garnered considerable attention from both academic researchers and industry practitioners.
Self-Reasoning Assistant Learning for non-Abelian Gauge Fields Design
This framework utilizes the forward diffusion process to capture and reproduce the complex patterns and details inherent in the target distribution through continuous transformation.
LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Models
We introduces LLaST, a framework for building high-performance Large Language model based Speech-to-text Translation systems.
LogoSticker: Inserting Logos into Diffusion Models for Customized Generation
Recent advances in text-to-image model customization have underscored the importance of integrating new concepts with a few examples.
ViLLa: Video Reasoning Segmentation with Large Language Model
To bridge the gap between image and video, in this work, we propose a new video segmentation task - video reasoning segmentation.
Deep Bag-of-Words Model: An Efficient and Interpretable Relevance Architecture for Chinese E-Commerce
Text relevance or text matching of query and product is an essential technique for the e-commerce search system to ensure that the displayed products can match the intent of the query.
PaliGemma: A versatile 3B VLM for transfer
PaliGemma is an open Vision-Language Model (VLM) that is based on the SigLIP-So400m vision encoder and the Gemma-2B language model.
A Re-solving Heuristic for Dynamic Assortment Optimization with Knapsack Constraints
In this paper, we consider a multi-stage dynamic assortment optimization problem with multi-nomial choice modeling (MNL) under resource knapsack constraints.
To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Models
Large Language Models (LLMs) trained on extensive corpora inevitably retain sensitive data, such as personal privacy information and copyrighted material.
Evaluation of Bias Towards Medical Professionals in Large Language Models
GPT-4, Claude-3, and Mistral-Large showed significant gender and racial biases when evaluating medical professionals for residency selection.
Towards LLM-Powered Ambient Sensor Based Multi-Person Human Activity Recognition
Human Activity Recognition (HAR) is one of the central problems in fields such as healthcare, elderly care, and security at home.
Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging
While large language models (LLMs) excel in many domains, their complexity and scale challenge deployment in resource-limited environments.
Job-SDF: A Multi-Granularity Dataset for Job Skill Demand Forecasting and Benchmarking
We benchmark a range of models on this dataset, evaluating their performance in standard scenarios, in predictions focused on lower value ranges, and in the presence of structural breaks, providing new insights for further research.
Towards Adaptive Neighborhood for Advancing Temporal Interaction Graph Modeling
These works can generate temporal node representations by encoding the surrounding neighborhoods for the target node.
Continuous-Time Digital Twin with Analogue Memristive Neural Ordinary Differential Equation Solver
We experimentally validate our approach by developing a digital twin of the HP memristor, which accurately extrapolates its nonlinear dynamics, achieving a 4. 2-fold projected speedup and a 41. 4-fold projected decrease in energy consumption compared to state-of-the-art digital hardware, while maintaining an acceptable error margin.
Zero-shot Image Editing with Reference Imitation
Image editing serves as a practical yet challenging task considering the diverse demands from users, where one of the hardest parts is to precisely describe how the edited image should look like.
Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms
We also employ the recently proposed bagged-deep-image-prior (bagged-DIP) idea to develop SCI Bagged Deep Video Prior (SCI-BDVP) algorithms that address the common challenges faced by standard UNN solutions.
Phy-Diff: Physics-guided Hourglass Diffusion Model for Diffusion MRI Synthesis
Our model introduces the physical principles of dMRI in the noise evolution in the diffusion process and introduce a query-based conditional mapping within the difussion model.
EPIDetect: Video-based convulsive seizure detection in chronic epilepsy mouse model for anti-epilepsy drug screening
In the preclinical translational studies, drug candidates with remarkable anti-epileptic efficacy demonstrate long-term suppression of spontaneous recurrent seizures (SRSs), particularly convulsive seizures (CSs), in mouse models of chronic epilepsy.
A Multilingual Similarity Dataset for News Article Frame
Overall we introduce the most extensive cross-lingual news article similarity dataset available to date with 26, 555 labeled news article pairs across 10 languages.
VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Grounding
Video Temporal Grounding (VTG) strives to accurately pinpoint event timestamps in a specific video using linguistic queries, significantly impacting downstream tasks like video browsing and editing.
Global News Synchrony and Diversity During the Start of the COVID-19 Pandemic
News coverage profoundly affects how countries and individuals behave in international relations.
IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination
This paper aims to recover object materials from posed images captured under an unknown static lighting condition.
Efficient and accurate neural field reconstruction using resistive memory
The GE harnesses the intrinsic stochasticity of resistive memory for efficient input encoding, while the PE achieves precise weight mapping through a Hardware-Aware Quantization (HAQ) circuit.
Proof-of-Learning with Incentive Security
Most concurrent blockchain systems rely heavily on the Proof-of-Work (PoW) or Proof-of-Stake (PoS) mechanisms for decentralized consensus and security assurance.
Towards Efficient Resume Understanding: A Multi-Granularity Multi-Modal Pre-Training Approach
In the contemporary era of widespread online recruitment, resume understanding has been widely acknowledged as a fundamental and crucial task, which aims to extract structured information from resume documents automatically.
Generative Resident Separation and Multi-label Classification for Multi-person Activity Recognition
This paper presents two models to address the problem of multi-person activity recognition using ambient sensors in a home.
Resistive Memory-based Neural Differential Equation Solver for Score-based Diffusion Model
Demonstrating equivalent generative quality to the software baseline, our system achieved remarkable enhancements in generative speed for both unconditional and conditional generation tasks, by factors of 64. 8 and 156. 5, respectively.
Data-Driven Knowledge Transfer in Batch $Q^*$ Learning
In data-driven decision-making in marketing, healthcare, and education, it is desirable to utilize a large amount of data from existing ventures to navigate high-dimensional feature spaces and address data scarcity in new ventures.
FlashFace: Human Image Personalization with High-fidelity Identity Preservation
This work presents FlashFace, a practical tool with which users can easily personalize their own photos on the fly by providing one or a few reference face images and a text prompt.
Ensemble Adversarial Defense via Integration of Multiple Dispersed Low Curvature Models
In this work, we aim to enhance ensemble diversity by reducing attack transferability.
Wear-Any-Way: Manipulable Virtual Try-on via Sparse Correspondence Alignment
This paper introduces a novel framework for virtual try-on, termed Wear-Any-Way.
Uncertainty-Aware Pseudo-Label Filtering for Source-Free Unsupervised Domain Adaptation
Source-free unsupervised domain adaptation (SFUDA) aims to enable the utilization of a pre-trained source model in an unlabeled target domain without access to source data.
Rethinking Generative Large Language Model Evaluation for Semantic Comprehension
Through a comprehensive evaluation of 24 models across 11 benchmarks, we highlight several potential drawbacks of MCQA, for instance, the inconsistency between the MCQA evaluation and the generation of open-ended responses in practical scenarios.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
In this report, we introduce the Gemini 1. 5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.
Ranked #1 on
Zero-Shot Video Question Answer
on Zero-shot Video Question Answering on LongVideoBench
(using extra training data)
Closing the Visual Sim-to-Real Gap with Object-Composable NeRFs
Deep learning methods for perception are the cornerstone of many robotic systems.
InstructEdit: Instruction-based Knowledge Editing for Large Language Models
Knowledge editing for large language models can offer an efficient solution to alter a model's behavior without negatively impacting the overall performance.
Does Combining Parameter-efficient Modules Improve Few-shot Transfer Accuracy?
Parameter-efficient fine-tuning stands as the standard for efficiently fine-tuning large language and vision models on downstream tasks.
OpenSUN3D: 1st Workshop Challenge on Open-Vocabulary 3D Scene Understanding
This report provides an overview of the challenge hosted at the OpenSUN3D Workshop on Open-Vocabulary 3D Scene Understanding held in conjunction with ICCV 2023.
Bagged Deep Image Prior for Recovering Images in the Presence of Speckle Noise
We investigate both the theoretical and algorithmic aspects of likelihood-based methods for recovering a complex-valued signal from multiple sets of measurements, referred to as looks, affected by speckle (multiplicative) noise.
Deep learning for the design of non-Hermitian topolectrical circuits
Non-Hermitian topological phases can produce some remarkable properties, compared with their Hermitian counterpart, such as the breakdown of conventional bulk-boundary correspondence and the non-Hermitian topological edge mode.
Understanding the Training Speedup from Sampling with Approximate Losses
In this work, we focus on the greedy approach of selecting samples with large \textit{approximate losses} instead of exact losses in order to reduce the selection overhead.
Prompt Learning on Temporal Interaction Graphs
In detail, we propose a temporal prompt generator to offer temporally-aware prompts for different tasks.
Multi-Scale Semantic Segmentation with Modified MBConv Blocks
Recently, MBConv blocks, initially designed for efficiency in resource-limited settings and later adapted for cutting-edge image classification performances, have demonstrated significant potential in image classification tasks.
Parametric Feature Transfer: One-shot Federated Learning with Foundation Models
This paper introduces FedPFT (Federated Learning with Parametric Feature Transfer), a methodology that harnesses the transferability of foundation models to enhance both accuracy and communication efficiency in one-shot FL.
DFML: Decentralized Federated Mutual Learning
In the realm of real-world devices, centralized servers in Federated Learning (FL) present challenges including communication bottlenecks and susceptibility to a single point of failure.
EK-Net:Real-time Scene Text Detection with Expand Kernel Distance
Recently, scene text detection has received significant attention due to its wide application.
Evaluating and Enhancing Large Language Models Performance in Domain-specific Medicine: Osteoarthritis Management with DocOA
This study focused on evaluating and enhancing the clinical capabilities of LLMs in specific domains, using osteoarthritis (OA) management as a case study.
The Distributional Reward Critic Framework for Reinforcement Learning Under Perturbed Rewards
Our proposed methods are compatible with any RL algorithm.
Enhancing Blood Flow Assessment in Diffuse Correlation Spectroscopy: A Transfer Learning Approach with Noise Robustness Analysis
While machine learning has demonstrated significant potential for measuring blood flow index (BFi), an open question concerning the success of this approach pertains to its robustness in scenarios involving deviations between datasets with varying Signal-to-Noise Ratios (SNRs) originating from diverse clinical applications and various setups.
Fast Cerebral Blood Flow Analysis via Extreme Learning Machine
We introduce a rapid and precise analytical approach for analyzing cerebral blood flow (CBF) using Diffuse Correlation Spectroscopy (DCS) with the application of the Extreme Learning Machine (ELM).
Transfer the linguistic representations from TTS to accent conversion with non-parallel data
This paper introduces a novel non-autoregressive framework for accent conversion that learns accent-agnostic linguistic representations and employs them to convert the accent in the source speech.
Depth Map Denoising Network and Lightweight Fusion Network for Enhanced 3D Face Recognition
With the increasing availability of consumer depth sensors, 3D face recognition (FR) has attracted more and more attention.
On Scaling Up a Multilingual Vision and Language Model
We explore the boundaries of scaling up a multilingual vision and language model both in terms of size of the components and the breadth of its training task mixture.
PredToken: Predicting Unknown Tokens and Beyond with Coarse-to-Fine Iterative Decoding
Building on this we present a "coarse-to-fine iterative decoding" method.
LaFFi: Leveraging Hybrid Natural Language Feedback for Fine-tuning Language Models
Fine-tuning Large Language Models (LLMs) adapts a trained model to specific downstream tasks, significantly improving task-specific performance.
Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation
A simple neural network then learns the implicit mapping from the intra- and inter-sample relations to an adaptive, sample-wise knowledge fusion ratio in a bilevel-optimization manner.
GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoning
In this paper, we evaluate the reasoning capabilities of VLMs along various axes through the lens of geometry problems.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Visual Question Answering (VQA)
on InfographicVQA
Calibrated One Round Federated Learning with Bayesian Inference in the Predictive Space
To improve scalability for larger models, one common Bayesian approach is to approximate the global predictive posterior by multiplying local predictive posteriors.
Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training
Implementing online training of RNNs on the edge calls for optimized algorithms for an efficient deployment on hardware.
Audio Deepfake Detection with Self-Supervised WavLM and Multi-Fusion Attentive Classifier
In this paper, we report our efforts to combine the self-supervised WavLM model and Multi-Fusion Attentive classifier for audio deepfake detection.
TMT-VIS: Taxonomy-aware Multi-dataset Joint Training for Video Instance Segmentation
What we possess are numerous isolated filed-specific datasets, thus, it is appealing to jointly train models across the aggregation of datasets to enhance data volume and diversity.
Cross Domain Generative Augmentation: Domain Generalization with Latent Diffusion Models
The VRM reduces the estimation error in ERM by replacing the point-wise kernel estimates with a more precise estimation of true data distribution that reduces the gap between data points \textbf{within each domain}.
Mismatch Quest: Visual and Textual Feedback for Image-Text Misalignment
While existing image-text alignment models reach high quality binary assessments, they fall short of pinpointing the exact source of misalignment.
LivePhoto: Real Image Animation with Text-guided Motion Control
In particular, considering the facts that (1) text can only describe motions roughly (e. g., regardless of the moving speed) and (2) text may include both content and motion descriptions, we introduce a motion intensity estimation module as well as a text re-weighting module to reduce the ambiguity of text-to-motion mapping.
A Lightweight Clustering Framework for Unsupervised Semantic Segmentation
We thus propose a lightweight clustering framework for unsupervised semantic segmentation.
Leverage Staking with Liquid Staking Derivatives (LSDs): Opportunities and Risks
In the Proof of Stake (PoS) Ethereum ecosystem, users can stake ETH on Lido to receive stETH, a Liquid Staking Derivative (LSD) that represents staked ETH and accrues staking rewards.
Utility Fairness in Contextual Dynamic Pricing with Demand Learning
This paper introduces a novel contextual bandit algorithm for personalized pricing under utility fairness constraints in scenarios with uncertain demand, achieving an optimal regret upper bound.
Learning Disentangled Identifiers for Action-Customized Text-to-Image Generation
Experimental results show that existing subject-driven customization methods fail to learn the representative characteristics of actions and struggle in decoupling actions from context features, including appearance.
YUAN 2.0: A Large Language Model with Localized Filtering-based Attention
In this work, we develop and release Yuan 2. 0, a series of large language models with parameters ranging from 2. 1 billion to 102. 6 billion.
Pruning random resistive memory for optimizing analogue AI
Here, we report a universal solution, software-hardware co-design using structural plasticity-inspired edge pruning to optimize the topology of a randomly weighted analogue resistive memory neural network.
Evolutionary City: Towards a Flexible, Agile and Symbiotic System
This paper introduces a novel approach, aiming to enable cities to evolve and respond more effectively to such dynamic demand.
Map-assisted TDOA Localization Enhancement Based On CNN
For signal processing related to localization technologies, non line of sight (NLOS) multipaths have a significant impact on the localization error level.
GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Models
In this work, we propose Goal-conditioned Offline Planning (GOPlan), a novel model-based framework that contains two key phases: (1) pretraining a prior policy capable of capturing multi-modal action distribution within the multi-goal dataset; (2) employing the reanalysis method with planning to generate imagined trajectories for funetuning policies.
Triplet Attention Transformer for Spatiotemporal Predictive Learning
Spatiotemporal predictive learning offers a self-supervised learning paradigm that enables models to learn both spatial and temporal patterns by predicting future sequences based on historical sequences.
PaLI-3 Vision Language Models: Smaller, Faster, Stronger
This paper presents PaLI-3, a smaller, faster, and stronger vision language model (VLM) that compares favorably to similar models that are 10x larger.
Ranked #2 on
Temporal/Casual QA
on NExT-QA
(using extra training data)
Can We Edit Multimodal Large Language Models?
In this paper, we focus on editing Multimodal Large Language Models (MLLMs).
Uni3DETR: Unified 3D Detection Transformer
In this paper, we propose Uni3DETR, a unified 3D detector that addresses indoor and outdoor 3D detection within the same framework.
Ranked #1 on
3D Object Detection
on SUN-RGBD
Online Estimation and Inference for Robust Policy Evaluation in Reinforcement Learning
In this paper, we develop an online robust policy evaluation procedure, and establish the limiting distribution of our estimator, based on its Bahadur representation.
Unveiling the Pitfalls of Knowledge Editing for Large Language Models
This paper pioneers the investigation into the potential pitfalls associated with knowledge editing for LLMs.
Adaptive Liquidity Provision in Uniswap V3 with Deep Reinforcement Learning
The DRL policy aims to optimize trading fees earned by LPs against associated costs, such as gas fees and hedging expenses, which is referred to as loss-versus-rebalancing (LVR).
RDGSL: Dynamic Graph Representation Learning with Structure Learning
To address the noise dynamics issue, we introduce the Dynamic Graph Filter, where we innovatively propose a dynamic noise function that dynamically captures both current and historical noise, enabling us to assess the temporal aspect of noise and generate a denoised graph.
OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentation
In this work, we introduce OpenIns3D, a new 3D-input-only framework for 3D open-vocabulary scene understanding.
Ranked #1 on
Zero-shot 3D Point Cloud Classification
on ScanNetV2
3D Open-Vocabulary Instance Segmentation
3D Open-Vocabulary Object Detection
+6
SPEED: Streaming Partition and Parallel Acceleration for Temporal Interaction Graph Embedding
Simultaneously, resource consumption of a single-GPU can be diminished by up to 69%, thus enabling the multiple GPU-based training and acceleration encompassing millions of nodes and billions of edges.
SwinFace: A Multi-task Transformer for Face Recognition, Expression Recognition, Age Estimation and Attribute Estimation
To address the conflicts among multiple tasks and meet the different demands of tasks, a Multi-Level Channel Attention (MLCA) module is integrated into each task-specific analysis subnet, which can adaptively select the features from optimal levels and channels to perform the desired tasks.
Understanding Hessian Alignment for Domain Generalization
We validate the OOD generalization ability of proposed methods in different scenarios, including transferability, severe correlation shift, label shift and diversity shift.
Understanding the Role of Layer Normalization in Label-Skewed Federated Learning
In this work, we reveal the profound connection between layer normalization and the label shift problem in federated learning.
Teacher-Student Architecture for Knowledge Distillation: A Survey
Recently, Teacher-Student architectures have been effectively and widely embraced on various knowledge distillation (KD) objectives, including knowledge compression, knowledge expansion, knowledge adaptation, and knowledge enhancement.
Convolutional Occupancy Models for Dense Packing of Complex, Novel Objects
Dense packing in pick-and-place systems is an important feature in many warehouse and logistics applications.
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Our goal is to enable a single end-to-end trained model to both learn to map robot observations to actions and enjoy the benefits of large-scale pretraining on language and vision-language data from the web.
Ranked #4 on
Robot Manipulation
on SimplerEnv-Google Robot
(using extra training data)
AnyDoor: Zero-shot Object-level Image Customization
This work presents AnyDoor, a diffusion-based image generator with the power to teleport target objects to new scenes at user-specified locations in a harmonious way.
Is ChatGPT a Good Personality Recognizer? A Preliminary Study
Concretely, we employ a variety of prompting strategies to explore ChatGPT's ability in recognizing personality from given text, especially the level-oriented prompting strategy we designed for guiding ChatGPT in analyzing given text at a specified level.
Active Learning in Physics: From 101, to Progress, and Perspective
Active Learning (AL) is a family of machine learning (ML) algorithms that predates the current era of artificial intelligence.
A Comprehensive Survey of Artificial Intelligence Techniques for Talent Analytics
To this end, we present an up-to-date and comprehensive survey on AI technologies used for talent analytics in the field of human resource management.
Memory-Query Tradeoffs for Randomized Convex Optimization
We show that any randomized first-order algorithm which minimizes a $d$-dimensional, $1$-Lipschitz convex function over the unit ball must either use $\Omega(d^{2-\delta})$ bits of memory or make $\Omega(d^{1+\delta/6-o(1)})$ queries, for any constant $\delta\in (0, 1)$ and when the precision $\epsilon$ is quasipolynomially small in $d$.
Performance of data-driven inner speech decoding with same-task EEG-fMRI data fusion and bimodal models
Decoding inner speech from the brain signal via hybridisation of fMRI and EEG data is explored to investigate the performance benefits over unimodal models.
2D-Shapley: A Framework for Fragmented Data Valuation
Data valuation -- quantifying the contribution of individual data sources to certain predictive behaviors of a model -- is of great importance to enhancing the transparency of machine learning and designing incentive systems for data sharing.
Magnetic Resonance Spectroscopy Quantification Aided by Deep Estimations of Imperfection Factors and Macromolecular Signal
Methods: Linear Least Squares (LLS) is integrated with deep learning to reduce the complexity of solving this overall quantification.
PolyVoice: Language Models for Speech to Speech Translation
For the speech synthesis part, we adopt the existing VALL-E X approach and build a unit-based audio language model.
PaLI-X: On Scaling up a Multilingual Vision and Language Model
We present the training recipe and results of scaling up PaLI-X, a multilingual vision and language model, both in terms of size of the components and the breadth of its training task mixture.
Ranked #1 on
Fine-Grained Image Recognition
on OVEN
Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality
Stochastic gradient descent with momentum (SGDM) has been widely used in many machine learning and statistical applications.
CONA: A novel CONtext-Aware instruction paradigm for communication using large language model
We introduce CONA, a novel context-aware instruction paradigm for effective knowledge dissemination using generative pre-trained transformer (GPT) models.
Text Generation with Speech Synthesis for ASR Data Augmentation
In this work, we explore text augmentation for ASR using large-scale pre-trained neural networks, and systematically compare those to traditional text augmentation methods.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Schema-adaptable Knowledge Graph Construction
This necessitates a system that can handle evolving schema automatically to extract information for KGC.
