Search Results for author:
Found 195 papers, 80 papers with code
Accelerating Diffusion Sampling via Exploiting Local Transition Coherence
To address this issue, we discover a new statistical relationship in the transition operator between adjacent steps, focusing on the relationship of the outputs from the network.
Beam Selection in ISAC using Contextual Bandit with Multi-modal Transformer and Transfer Learning
ISAC unifies wireless communication and RADAR or other forms of sensing to optimize spectral and hardware resources.
MITracker: Multi-View Integration for Visual Object Tracking
Multi-view object tracking (MVOT) offers promising solutions to challenges such as occlusion and target loss, which are common in traditional single-view tracking.
DiffKAN-Inpainting: KAN-based Diffusion model for brain tumor inpainting
Brain tumors delay the standard preprocessing workflow for further examination.
Protein Large Language Models: A Comprehensive Survey
Protein-specific large language models (Protein LLMs) are revolutionizing protein science by enabling more efficient protein structure prediction, function annotation, and design.
EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models
Automated Essay Scoring (AES) plays a crucial role in educational assessment by providing scalable and consistent evaluations of writing tasks.
A Comprehensive Review of Protein Language Models
At the intersection of the rapidly growing biological data landscape and advancements in Natural Language Processing (NLP), protein language models (PLMs) have emerged as a transformative force in modern research.
Quasi-Conformal Convolution : A Learnable Convolution for Deep Learning on Riemann Surfaces
Each QCC operator is linked to a specific quasi-conformal mapping, enabling the adjustment of the convolution operation through manipulation of this mapping.
Addressing Bias in Generative AI: Challenges and Research Opportunities in Information Management
Generative AI technologies, particularly Large Language Models (LLMs), have transformed information management systems but introduced substantial biases that can compromise their effectiveness in informing business decision-making.
Generative AI-enabled Blockage Prediction for Robust Dual-Band mmWave Communication
In mmWave wireless networks, signal blockages present a significant challenge due to the susceptibility to environmental moving obstructions.
Rethinking Early-Fusion Strategies for Improved Multimodal Image Segmentation
To address this issue, we propose a novel multimodal fusion network (EFNet) based on an early fusion strategy and a simple but effective feature clustering for training efficient RGB-T semantic segmentation.
Correcting Large Language Model Behavior via Influence Function
Recent advancements in AI alignment techniques have significantly improved the alignment of large language models (LLMs) with static human preferences.
Efficient Speech Command Recognition Leveraging Spiking Neural Network and Curriculum Learning-based Knowledge Distillation
Furthermore, by executing KDCL, we reduce the number of time steps by 60% and decrease energy consumption by 54. 8% while maintaining comparable performance to recent SOTA results.
ChatDiT: A Training-Free Baseline for Task-Agnostic Free-Form Chatting with Diffusion Transformers
Building upon this foundation, we present ChatDiT, a zero-shot, general-purpose, and interactive visual generation framework that leverages pretrained diffusion transformers in their original form, requiring no additional tuning, adapters, or modifications.
The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control
We present The Matrix, the first foundational realistic world simulator capable of generating continuous 720p high-fidelity real-scene video streams with real-time, responsive control in both first- and third-person perspectives, enabling immersive exploration of richly dynamic environments.
Pan-protein Design Learning Enables Task-adaptive Generalization for Low-resource Enzyme Design
Computational protein design (CPD) offers transformative potential for bioengineering, but current deep CPD models, focused on universal domains, struggle with function-specific designs.
Compressive Spectrum Sensing with 1-bit ADCs
Efficient wideband spectrum sensing (WSS) is essential for managing spectrum scarcity in wireless communications.
Voltage Support Capability Analysis of Grid-Forming Inverters with Current-Limiting Control Under Asymmetrical Grid Faults
It is discovered that matching the phase angle of the virtual impedance, emulated by the CLC, with that of the composed impedance from the capacitor to the fault location can maximize the voltage support capability of GFM inverters under asymmetrical grid faults.
Cooperation and Personalization on a Seesaw: Choice-based FL for Safe Cooperation in Wireless Networks
In this work, we first overview benefits and concerns when applying FL to wireless networks.
Hierarchical Knowledge Graph Construction from Images for Scalable E-Commerce
For e-commerce, an efficient and low-cost automated knowledge graph construction method is the foundation of enabling various successful downstream applications.
pEBR: A Probabilistic Approach to Embedding Based Retrieval
Embedding retrieval aims to learn a shared semantic representation space for both queries and items, thus enabling efficient and effective item retrieval using approximate nearest neighbor (ANN) algorithms.
Multi-Modal Transformer and Reinforcement Learning-based Beam Management
In this work, we propose a two-step beam management method by combining MMT with RL for dynamic beam index prediction.
DICE: Discrete Inversion Enabling Controllable Editing for Multinomial Diffusion and Masked Generative Models
Discrete diffusion models have achieved success in tasks like image generation and masked language modeling but face limitations in controlled content editing.
Reconstruction of Particle Flow Energy Distribution Using Deep Learning Algorithms
In high-energy particle physics, extracting information from complex detector signals is crucial for energy reconstruction.
Epsilon-VAE: Denoising as Visual Decoding
Current visual tokenization methods rely on a traditional autoencoder framework, where the encoder compresses data into latent representations, and the decoder reconstructs the original input.
Understanding the Benefits of SimCLR Pre-Training in Two-Layer Convolutional Neural Networks
SimCLR is one of the most popular contrastive learning methods for vision tasks.
SEE: Semantically Aligned EEG-to-Text Translation
Decoding neurophysiological signals into language is of great research interest within brain-computer interface (BCI) applications.
Solving Integrated Process Planning and Scheduling Problem via Graph Neural Network Based Deep Reinforcement Learning
The Integrated Process Planning and Scheduling (IPPS) problem combines process route planning and shop scheduling to achieve high efficiency in manufacturing and maximize resource utilization, which is crucial for modern manufacturing systems.
MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinement
Existing works in single-image human reconstruction suffer from weak generalizability due to insufficient training data or 3D inconsistencies for a lack of comprehensive multi-view knowledge.
NUS-Emo at SemEval-2024 Task 3: Instruction-Tuning LLM for Multimodal Emotion-Cause Analysis in Conversations
Our project targets the challenges of subtask 2, dedicated to Multimodal Emotion-Cause Pair Extraction with Emotion Category (MECPE-Cat), and constructs a dual-component system tailored to the unique challenges of this task.
Emotion-Cause Pair Extraction
Emotion Recognition in Conversation
+1
MetaEnzyme: Meta Pan-Enzyme Learning for Task-Adaptive Redesign
Enzyme design plays a crucial role in both industrial production and biology.
A Reference-Based 3D Semantic-Aware Framework for Accurate Local Facial Attribute Editing
Facial attribute editing plays a crucial role in synthesizing realistic faces with specific characteristics while maintaining realistic appearances.
Evaluating the evolution and inter-individual variability of infant functional module development from 0 to 5 years old
The segregation and integration of infant brain networks undergo tremendous changes due to the rapid development of brain function and organization.
BACON: Supercharge Your VLM with Bag-of-Concept Graph to Mitigate Hallucinations
This paper presents Bag-of-Concept Graph (BACON) to gift models with limited linguistic abilities to taste the privilege of Vision Language Models (VLMs) and boost downstream tasks such as detection, visual question answering (VQA), and image generation.
ALTER: Augmentation for Large-Table-Based Reasoning
While extensive research has explored the use of large language models (LLMs) for table-based reasoning, most approaches struggle with scalability when applied to large tables.
SVFormer: A Direct Training Spiking Transformer for Efficient Video Action Recognition
To address this challenge, some researchers have turned to brain-inspired spiking neural networks (SNNs), such as recurrent SNNs and ANN-converted SNNs, leveraging their inherent temporal dynamics and energy efficiency.
Generative AI-in-the-loop: Integrating LLMs and GPTs into the Next Generation Networks
Our case study further demonstrates the advantages of our proposed idea.
VeriSplit: Secure and Practical Offloading of Machine Learning Inferences across IoT Devices
While this presents a compelling potential alternative to cloud offloading, concerns about the integrity of inferences, the confidentiality of model parameters, and the privacy of users' data mean that device vendors may be hesitant to offload their inferences to a platform managed by another manufacturer.
RTGen: Generating Region-Text Pairs for Open-Vocabulary Object Detection
Open-vocabulary object detection (OVD) requires solid modeling of the region-semantic relationship, which could be learned from massive region-text pairs.
Ranked #12 on
Open Vocabulary Object Detection
on LVIS v1.0
(using extra training data)
Generative AI Empowered LiDAR Point Cloud Generation with Multimodal Transformer
This will enable the base stations to achieve more precise environmental sensing.
LoCI-DiffCom: Longitudinal Consistency-Informed Diffusion Model for 3D Infant Brain Image Completion
The infant brain undergoes rapid development in the first few years after birth. Compared to cross-sectional studies, longitudinal studies can depict the trajectories of infants brain development with higher accuracy, statistical power and flexibility. However, the collection of infant longitudinal magnetic resonance (MR) data suffers a notorious dropout problem, resulting in incomplete datasets with missing time points.
Large Language Models in Wireless Application Design: In-Context Learning-enhanced Automatic Network Intrusion Detection
With experiments on a real network intrusion detection dataset, in-context learning proves to be highly beneficial in improving the task processing performance in a way that no further training or fine-tuning of LLMs is required.
Direct Training High-Performance Deep Spiking Neural Networks: A Review of Theories and Methods
In this paper, we provide a new perspective to summarize the theories and methods for training deep SNNs with high performance in a systematic and comprehensive way, including theory fundamentals, spiking neuron models, advanced SNN models and residual architectures, software frameworks and neuromorphic hardware, applications, and future trends.
4DBInfer: A 4D Benchmarking Toolbox for Graph-Centric Predictive Modeling on Relational DBs
Although RDBs store vast amounts of rich, informative data spread across interconnected tables, the progress of predictive machine learning models as applied to such tasks arguably falls well behind advances in other domains such as computer vision or natural language processing.
A Multi-Level Framework for Accelerating Training Transformer Models
Specifically, the framework is based on three basic operators, Coalescing, De-coalescing and Interpolation, which can be orchestrated to build a multi-level training framework.
Memory-based Cross-modal Semantic Alignment Network for Radiology Report Generation
To ensure the semantic consistency of the retrieved cross modal prior knowledge, a cross-modal semantic alignment module (SAM) is proposed.
Revolutionizing Disease Diagnosis with simultaneous functional PET/MR and Deeply Integrated Brain Metabolic, Hemodynamic, and Perfusion Networks
To fully exploit the inherent complex and nonlinear relation among modalities while producing fine-grained representations for uni-modal inference, we subsequently add a modal alignment module to line up a dominant modality (e. g., PET) with representations of auxiliary modalities (MR).
QKFormer: Hierarchical Spiking Transformer using Q-K Attention
ii) We incorporate the hierarchical structure, which significantly benefits the performance of both the brain and artificial neural networks, into spiking transformers to obtain multi-scale spiking representation.
FluoroSAM: A Language-aligned Foundation Model for X-ray Image Segmentation
Recently, foundation models (FMs) -- machine learning models trained on large amounts of highly variable data thus enabling broad applicability -- have emerged as promising tools for automated image analysis.
MUC: Mixture of Uncalibrated Cameras for Robust 3D Human Body Reconstruction
Rather than merely averaging the models across views, we develop a neural network trained to assign weights to individual views for all human body joints, based on the estimated distribution of joint distances from each camera.
QIS : Interactive Segmentation via Quasi-Conformal Mappings
In this paper, we propose the quasi-conformal interactive segmentation (QIS) model, which incorporates user input in the form of positive and negative clicks.
COPR: Continual Human Preference Learning via Optimal Policy Regularization
Reinforcement Learning from Human Feedback (RLHF) is commonly utilized to improve the alignment of Large Language Models (LLMs) with human preferences.
Cas-DiffCom: Cascaded diffusion model for infant longitudinal super-resolution 3D medical image completion
Early infancy is a rapid and dynamic neurodevelopmental period for behavior and neurocognition.
Federated Learning with Dual Attention for Robust Modulation Classification under Attacks
To this end, we leverage attention mechanisms as a defense against attacks in FL and propose a robust FL algorithm by integrating the attention mechanisms into the global model aggregation step.
CPPO: Continual Learning for Reinforcement Learning with Human Feedback
The approach of Reinforcement Learning from Human Feedback (RLHF) is widely used for enhancing pre-trained Language Models (LM), enabling them to better align with human preferences.
Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation
We use the novel multi-reward optimization algorithm to jointly optimize the T2I model and a prompt expansion network, resulting in significant improvement of image quality and also allow to control the trade-off of different rewards using a reward related prompt during inference.
Fundamental Limitation of Semantic Communications: Neural Estimation for Rate-Distortion
For the case with unknown semantic source distribution, while only a set of the source samples is available, we propose a neural-network-based method by leveraging the generative networks to learn the semantic source distribution.
Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensembles
Reinforcement learning from human feedback (RLHF) emerges as a promising paradigm for aligning large language models (LLMs).
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Visual Question Answering (VQA)
on InfographicVQA
VideoLCM: Video Latent Consistency Model
Consistency models have demonstrated powerful capability in efficient image generation and allowed synthesis within a few sampling steps, alleviating the high computational cost in diffusion models.
An Incentive Mechanism for Federated Learning Based on Multiple Resource Exchange
We formulate the interaction between MO and DOs as an optimization problem, and the objective is to effectively utilize the communication and computing resource of the MO and DOs to minimize the time to complete an FL task.
CCM: Adding Conditional Controls to Text-to-Image Consistency Models
Consistency Models (CMs) have showed a promise in creating visual content efficiently and with high quality.
GFS: Graph-based Feature Synthesis for Prediction over Relational Databases
Consequently, the prevalent approach for training machine learning models on data stored in relational databases involves performing feature engineering to merge the data from multiple tables into a single table and subsequently applying single table models.
K-BMPC: Derivative-based Koopman Bilinear Model Predictive Control for Tractor-Trailer Trajectory Tracking with Unknown Parameters
Koopman Bilinear Model Predictive control (K-BMPC) is proposed to solve the trajectory tracking problem.
COPR: Continual Learning Human Preference through Optimal Policy Regularization
The technique of Reinforcement Learning from Human Feedback (RLHF) is a commonly employed method to improve pre-trained Language Models (LM), enhancing their ability to conform to human preferences.
BatteryML:An Open-source platform for Machine Learning on Battery Degradation
Battery degradation remains a pivotal concern in the energy storage domain, with machine learning emerging as a potent tool to drive forward insights and solutions.
From Supervised to Generative: A Novel Paradigm for Tabular Deep Learning with Large Language Models
Through GTL, we not only foster a deeper integration of LLMs' sophisticated abilities into tabular data comprehension and application but also offer a new training resource and a test bed for LLMs to enhance their ability to comprehend tabular data.
Accurate battery lifetime prediction across diverse aging conditions with deep learning
Here we introduce a universal deep learning approach that is capable of accommodating various aging conditions and facilitating effective learning under low-resource conditions by leveraging data from rich conditions.
Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency
We demonstrate image generation and captioning performance on par with state-of-the-art text-to-image and image-to-text models with orders of magnitude fewer (only 3M) paired image-text data.
Deformation-Invariant Neural Network and Its Applications in Distorted Image Restoration and Analysis
The QCTN is a deep neural network that outputs a quasiconformal map, which can be used to transform a geometrically distorted image into an improved version that is closer to the distribution of natural or good images.
Chat2Brain: A Method for Mapping Open-Ended Semantic Queries to Brain Activation Maps
In this study, we propose a method called Chat2Brain that combines LLMs to basic text-2-image model, known as Text2Brain, to map open-ended semantic queries to brain activation maps in data-scarce and complex query environments.
Price-Discrimination Game for Distributed Resource Management in Federated Learning
In vanilla federated learning (FL) such as FedAvg, the parameter server (PS) and multiple distributed clients can form a typical buyer's market, where the number of PS/buyers of FL services is far less than the number of clients/sellers.
StoryBench: A Multifaceted Benchmark for Continuous Story Visualization
To fill this gap, we collect comprehensive human annotations on three existing datasets, and introduce StoryBench: a new, challenging multi-task benchmark to reliably evaluate forthcoming text-to-video models.
Differentiable Retrieval Augmentation via Generative Language Modeling for E-commerce Query Intent Classification
Retrieval augmentation, which enhances downstream models by a knowledge retriever and an external corpus instead of by merely increasing the number of model parameters, has been successfully applied to many natural language processing (NLP) tasks such as text classification, question answering and so on.
Errors Dynamics in Affine Group Systems
In this paper, we focus on the error dynamics analysis for an affine group system under external disturbances or random noises.
NEON: Living Needs Prediction System in Meituan
To address these two challenges, we design a system of living NEeds predictiON named NEON, consisting of three phases: feature mining, feature fusion, and multi-task prediction.
ArrayBot: Reinforcement Learning for Generalizable Distributed Manipulation through Touch
We present ArrayBot, a distributed manipulation system consisting of a $16 \times 16$ array of vertically sliding pillars integrated with tactile sensors, which can simultaneously support, perceive, and manipulate the tabletop objects.
Lipschitz Singularities in Diffusion Models
Diffusion models, which employ stochastic differential equations to sample images through integrals, have emerged as a dominant class of generative models.
Learning Disentangled Prompts for Compositional Image Synthesis
We study domain-adaptive image synthesis, the problem of teaching pretrained image generative models a new style or concept from as few as one image to synthesize novel images, to better understand the compositional image synthesis.
Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentation
As such, we explore how natural language descriptions of the domains seen in training data can be used with large vision models trained on diverse pretraining datasets to generate useful variations of the training data.
Enhancing the Performance of Transformer-based Spiking Neural Networks by SNN-optimized Downsampling with Precise Gradient Backpropagation
In this paper, we propose ConvBN-MaxPooling-LIF (CML), an SNN-optimized downsampling with precise gradient backpropagation.
Mining fMRI Dynamics with Parcellation Prior for Brain Disease Diagnosis
To characterize atypical brain dynamics under diseases, prevalent studies investigate functional magnetic resonance imaging (fMRI).
Spikingformer: Spike-driven Residual Learning for Transformer-based Spiking Neural Network
Based on this residual design, we develop Spikingformer, a pure transformer-based spiking neural network.
Pelphix: Surgical Phase Recognition from X-ray Images in Percutaneous Pelvic Fixation
Surgical phase recognition (SPR) is a crucial element in the digital transformation of the modern operating theater.
Identity Encoder for Personalized Diffusion
Our approach greatly reduces the overhead for personalized image generation and is more applicable in many potential applications.
Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models
This paper proposes a method for generating images of customized objects specified by users.
SVDiff: Compact Parameter Space for Diffusion Fine-Tuning
Diffusion models have achieved remarkable success in text-to-image generation, enabling the creation of high-quality images from text prompts or other modalities.
Delayed and Indirect Impacts of Link Recommendations
Moreover, we show that the effects of recommendations can persist in networks, in part due to their indirect impacts on natural dynamics even after recommendations are turned off.
Steering Prototypes with Prompt-tuning for Rehearsal-free Continual Learning
In the context of continual learning, prototypes-as representative class embeddings-offer advantages in memory conservation and the mitigation of catastrophic forgetting.
Streaming Kernel PCA Algorithm With Small Space
The kernel method, which is commonly used in learning algorithms such as Support Vector Machines (SVMs), has also been applied in PCA algorithms.
StraIT: Non-autoregressive Generation with Stratified Image Transformer
We propose Stratified Image Transformer(StraIT), a pure non-autoregressive(NAR) generative model that demonstrates superiority in high-quality image synthesis over existing autoregressive(AR) and diffusion models(DMs).
Deep learning reveals the common spectrum underlying multiple brain disorders in youth and elders from brain functional networks
Brain disorders in the early and late life of humans potentially share pathological alterations in brain functions.
Evolution of Filter Bubbles and Polarization in News Recommendation
However, most existing work focuses on a static setting or over a short-time window, leaving open questions about the long-term and dynamic impacts of news recommendations.
Muse: Text-To-Image Generation via Masked Generative Transformers
Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.
Ranked #1 on
Text-to-Image Generation
on MS-COCO
(FID metric)
Enhanced Training of Query-Based Object Detection via Selective Query Recollection
This paper investigates a phenomenon where query-based object detectors mispredict at the last decoding stage while predicting correctly at an intermediate stage.
Ranked #14 on
Object Detection
on COCO 2017 val
MAGVIT: Masked Generative Video Transformer
We introduce the MAsked Generative VIdeo Transformer, MAGVIT, to tackle various video synthesis tasks with a single model.
Ranked #1 on
Video Prediction
on Something-Something V2
FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation
To address these problems, we propose FineDance, which contains 14. 6 hours of music-dance paired data, with fine-grained hand motions, fine-grained genres (22 dance genres), and accurate posture.
Dimensionality-Varying Diffusion Process
Diffusion models, which learn to reverse a signal destruction process to generate new data, typically require the signal at each step to have the same dimension.
Cost Splitting for Multi-Objective Conflict-Based Search
Our theoretical results show that, when combined with either of these two new splitting strategies, MO-CBS maintains its completeness and optimality guarantees.
MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis
In this work, we propose MAsked Generative Encoder (MAGE), the first framework to unify SOTA image generation and self-supervised representation learning.
Ranked #3 on
Unconditional Image Generation
on ImageNet 256x256
GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization
We envision our multi-year datasets can support the ML community in developing generalizable longitudinal behavior modeling algorithms.
Using Language to Extend to Unseen Domains
It is expensive to collect training data for every possible domain that a vision model may encounter when deployed.
A Learning-based Framework for Topology-Preserving Segmentation using Quasiconformal Mappings
The proposed network can also be trained in an unsupervised manner, eliminating the need for labeled training data.
Towards Semi-automatic Detection and Localization of Indoor Accessibility Issues using Mobile Depth Scanning and Computer Vision
To help improve the safety and accessibility of indoor spaces, researchers and health professionals have created assessment instruments that enable homeowners and trained experts to audit and improve homes.
Phenaki: Variable Length Video Generation From Open Domain Textual Description
To the best of our knowledge, this is the first time a paper studies generating videos from time variable prompts.
Ranked #4 on
Video Prediction
on BAIR Robot Pushing
Visual Prompt Tuning for Generative Transfer Learning
We base our framework on state-of-the-art generative vision transformers that represent an image as a sequence of visual tokens to the autoregressive or non-autoregressive transformers.
Hierarchical Graph Convolutional Network Built by Multiscale Atlases for Brain Disorder Diagnosis Using Functional Connectivity
In this study, we propose a novel framework to perform multiscale FCN analysis for brain disorder diagnosis.
Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNets
Beyond classification, Conv-Adapter can generalize to detection and segmentation tasks with more than 50% reduction of parameters but comparable performance to the traditional full fine-tuning.
Pre-training Tasks for User Intent Detection and Embedding Retrieval in E-commerce Search
BERT-style models pre-trained on the general corpus (e. g., Wikipedia) and fine-tuned on specific task corpus, have recently emerged as breakthrough techniques in many NLP tasks: question answering, text classification, sequence labeling and so on.
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
We present the Pathways Autoregressive Text-to-Image (Parti) model, which generates high-fidelity photorealistic images and supports content-rich synthesis involving complex compositions and world knowledge.
Ranked #1 on
Text-to-Image Generation
on COCO
Faithful Explanations for Deep Graph Models
Second, our analytical and empirical results demonstrate that feature attribution methods cannot capture the nonlinear effect of edge features, while existing subgraph explanation methods are not faithful.
Dimension Reduction for Efficient Dense Retrieval via Conditional Autoencoder
These embeddings need to be high-dimensional to fit training signals and guarantee the retrieval effectiveness of dense retrievers.
Ranked #1 on
Information Retrieval
on MS MARCO
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning
Continual learning aims to enable a single model to learn a sequence of tasks without catastrophic forgetting.
Powering Finetuning in Few-Shot Learning: Domain-Agnostic Bias Reduction with Selected Sampling
Class-agnostic bias is defined as the distribution shifting introduced by domain difference, which we propose Distribution Calibration Module(DCM) to reduce.
MaxViT: Multi-Axis Vision Transformer
We also show that our proposed model expresses strong generative modeling capability on ImageNet, demonstrating the superior potential of MaxViT blocks as a universal vision module.
Ranked #1 on
Object Detection
on COCO 2017
Unitail: Detecting, Reading, and Matching in Retail Scene
To make full use of computer vision technology in stores, it is required to consider the actual needs that fit the characteristics of the retail scene.
Ranked #1 on
Dense Object Detection
on SKU-110K
Learning Instance-Specific Adaptation for Cross-Domain Segmentation
We propose a test-time adaptation method for cross-domain image segmentation.
PNM: Pixel Null Model for General Image Segmentation
A major challenge in image segmentation is classifying object boundaries.
Givens Coordinate Descent Methods for Rotation Matrix Learning in Trainable Embedding Indexes
Product quantization (PQ) coupled with a space rotation, is widely used in modern approximate nearest neighbor (ANN) search systems to significantly compress the disk storage for embeddings and speed up the inner product computation.
Topology-Preserving Segmentation Network: A Deep Learning Segmentation Framework for Connected Component
TPSN is a deformation-based model that yields a deformation map through a UNet, which takes the medical image and a template mask as inputs.
StyleBERT: Chinese pretraining by font style information
With the success of down streaming task using English pre-trained language model, the pre-trained Chinese language model is also necessary to get a better performance of Chinese NLP task.
MaskGIT: Masked Generative Image Transformer
At inference time, the model begins with generating all tokens of an image simultaneously, and then refines the image iteratively conditioned on the previous generation.
Ranked #2 on
Text-to-Image Generation
on LHQC
MAXIM: Multi-Axis MLP for Image Processing
In this work, we present a multi-axis MLP based architecture called MAXIM, that can serve as an efficient and flexible general-purpose vision backbone for image processing tasks.
Ranked #1 on
Deblurring
on HIDE (trained on GOPRO)
ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation
To explore the landscape of large-scale pre-training for bidirectional text-image generation, we train a 10-billion parameter ERNIE-ViLG model on a large-scale dataset of 145 million (Chinese) image-text pairs which achieves state-of-the-art performance for both text-to-image and image-to-text tasks, obtaining an FID of 7. 9 on MS-COCO for text-to-image synthesis and best results on COCO-CN and AIC-ICC for image captioning.
Ranked #41 on
Text-to-Image Generation
on MS COCO
Learning to Prompt for Continual Learning
The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge.
Temporal Transformer Networks with Self-Supervision for Action Recognition
The temporal sequence self-supervision module we employ unprecedentedly adopts the streamlined strategy of "random batch random channel" to reverse the sequence of video frames, allowing robust extractions of motion information representation from inversed temporal dimensions and improving the generalization capability of the model.
BLT: Bidirectional Layout Transformer for Controllable Layout Generation
During inference, BLT first generates a draft layout from the input and then iteratively refines it into a high-quality layout by masking out low-confident attributes.
threaTrace: Detecting and Tracing Host-based Threats in Node Level Through Provenance Graph Learning
Recent studies propose leveraging the rich contextual information in data provenance to detect threats in a host.
You Ought to Look Around: Precise, Large Span Action Detection
In this paper, we propose a method called YOLA (You Ought to Look Around) which includes three parts: 1) a robust backbone SPN-I3D for extracting spatio-temporal features.
Vector-quantized Image Modeling with Improved VQGAN
Motivated by this success, we explore a Vector-quantized Image Modeling (VIM) approach that involves pretraining a Transformer to predict rasterized image tokens autoregressively.
Ranked #6 on
Image Reconstruction
on ImageNet
DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications
Experimental results show that DeepAID can provide high-quality interpretations for unsupervised DL models while meeting the special requirements of security domains.
Deep Image Synthesis from Intuitive User Input: A Review and Perspectives
In many applications of computer graphics, art and design, it is desirable for a user to provide intuitive non-image input, such as text, sketch, stroke, graph or layout, and have a computer system automatically generate photo-realistic images that adhere to the input content.
ViTGAN: Training GANs with Vision Transformers
Recently, Vision Transformers (ViTs) have shown competitive performance on image recognition while requiring less vision-specific inductive biases.
Ranked #31 on
Image Generation
on CIFAR-10
Local semantic enhanced convnet for aerial scene recognition
Our LSE-Net consists of a context enhanced convolutional feature extractor, a local semantic perception module and a classification layer.
Ranked #2 on
Scene Recognition
on AID
SearchGCN: Powering Embedding Retrieval by Graph Convolution Networks for E-Commerce Search
Graph convolution networks (GCN), which recently becomes new state-of-the-art method for graph node classification, recommendation and other applications, has not been successfully applied to industrial-scale search engine yet.
Improved Transformer for High-Resolution GANs
Attention-based models, exemplified by the Transformer, can effectively model long range dependency, but suffer from the quadratic complexity of self-attention operation, making them difficult to be adopted for high-resolution image generation based on Generative Adversarial Networks (GANs).
Ranked #2 on
Image Generation
on CelebA 256x256
(FID metric)
Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding
Hierarchical structures are popular in recent vision transformers, however, they require sophisticated designs and massive datasets to work well.
Ranked #91 on
Image Classification
on CIFAR-10
Joint Learning of Deep Retrieval Model and Product Quantization based Embedding Index
Embedding index that enables fast approximate nearest neighbor(ANN) search, serves as an indispensable component for state-of-the-art deep retrieval systems.
PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
To enhance the generalization ability of PanGu-$\alpha$, we collect 1. 1TB high-quality Chinese data from a wide range of domains to pretrain the model.
Ranked #1 on
Reading Comprehension (One-Shot)
on DuReader
Cloze (multi-choices) (Few-Shot)
Cloze (multi-choices) (One-Shot)
+19
Learning Hamiltonian dynamics by reservoir computer
Reconstructing the KAM dynamics diagram of Hamiltonian system from the time series of a limited number of parameters is an outstanding question in nonlinear science, especially when the Hamiltonian governing the system dynamics are unknown.
Transfer training from smaller language model
We initialize a larger target model from a smaller source model by copy weight values from source model and padding with zeros or small initialization values on it to make the source and target model have approximate outputs, which is valid due to block matrix multiplication and residual connection in transformer structure.
Revisiting Hierarchical Approach for Persistent Long-Term Video Prediction
Despite the recent advances in the literature, existing approaches are limited to moderately short-term prediction (less than a few seconds), while extrapolating it to a longer future quickly leads to destruction in structure and content.
Query Rewriting via Cycle-Consistent Translation for E-Commerce Search
Online A/B experiments show that it improves core e-commerce business metrics significantly.
A Multiscale Graph Convolutional Network for Change Detection in Homogeneous and Heterogeneous Remote Sensing Images
In this paper, a novel CD method based on the graph convolutional network (GCN) and multiscale object-based technique is proposed for both homogeneous and heterogeneous images.
Cross-Modal Contrastive Learning for Text-to-Image Generation
The quality of XMC-GAN's output is a major step up from previous models, as we show on three challenging datasets.
Ranked #26 on
Text-to-Image Generation
on MS COCO
(using extra training data)
Modeling Heterogeneous Statistical Patterns in High-dimensional Data by Adversarial Distributions: An Unsupervised Generative Framework
Meanwhile, such applications usually require modeling the intrinsic clusters in high-dimensional data, which usually displays heterogeneous statistical patterns as the patterns of different clusters may appear in different dimensions.
Transfer learning of chaotic systems
Here we investigate transfer learning of chaotic systems from the perspective of synchronization-based state inference, in which a reservoir computer trained by chaotic system A is used to infer the unmeasured variables of chaotic system B, while A is different from B in either parameter or dynamics.
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding
We argue that such contiguously masking method neglects to model the intra-dependencies and inter-relation of coarse-grained linguistic information.
PseudoSeg: Designing Pseudo Labels for Semantic Segmentation
We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
Co-evolution of Functional Brain Network at Multiple Scales during Early Infancy
By applying our proposed methodological framework on the collected longitudinal infant dataset, we provided the first evidence that, in the first 2 years of life, the brain functional network is co-evolved at different scales, where each scale displays the unique reconfiguration pattern in terms of modular organization.
GloDyNE: Global Topology Preserving Dynamic Network Embedding
The main and common objective of Dynamic Network Embedding (DNE) is to efficiently update node embeddings while preserving network topology at each time step.
Improving NER's Performance with Massive financial corpus
Training large deep neural networks needs massive high quality annotation data, but the time and labor costs are too expensive for small business.
From Spectrum Wavelet to Vertex Propagation: Graph Convolutional Networks Based on Taylor Approximation
Graph convolutional networks (GCN) have been recently utilized to extract the underlying structures of datasets with some labeled data and high-dimensional features.
A Hybrid Evolutionary Algorithm for Reliable Facility Location Problem
The reliable facility location problem (RFLP) is an important research topic of operational research and plays a vital role in the decision-making and management of modern supply chain and logistics.
Image Augmentations for GAN Training
We provide new state-of-the-art results for conditional generation on CIFAR-10 with both consistency loss and contrastive loss as additional regularizations.
Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learning
Two critical challenges stay in today's e-commerce search: how to retrieve items that are semantically relevant but not exact matching to query terms, and how to retrieve items that are more personalized to different users for the same search query.
A Simple Semi-Supervised Learning Framework for Object Detection
Semi-supervised learning (SSL) has a potential to improve the predictive performance of machine learning models using unlabeled data.
Ranked #13 on
Semi-Supervised Object Detection
on COCO 100% labeled data
(using extra training data)
ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring
We improve the recently-proposed ``MixMatch semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring.
A multiple-instance densely-connected ConvNet for aerial scene classification
It regards aerial scene classification as a multiple-instance learning problem so that local semantics can be further investigated.
Ranked #3 on
Scene Recognition
on AID
Solving Missing-Annotation Object Detection with Background Recalibration Loss
In this paper, we introduce a superior solution called Background Recalibration Loss (BRL) that can automatically re-calibrate the loss signals according to the pre-defined IoU threshold and input image.
Improved Consistency Regularization for GANs
Recent work has increased the performance of Generative Adversarial Networks (GANs) by enforcing a consistency cost on the discriminator.
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation
Current pre-training works in natural language generation pay little attention to the problem of exposure bias on downstream tasks.
Ranked #1 on
Question Generation
on SQuAD1.1
(using extra training data)
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance.
MANELA: A Multi-Agent Algorithm for Learning Network Embeddings
On the other hand, learning network embeddings on distributively stored networks still remained understudied: To the best of our knowledge, all existing algorithms for learning network embeddings have hitherto been exclusively centralized and thus cannot be applied to these networks.
Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models
We introduce a new local sparse attention layer that preserves two-dimensional geometry and locality.
Ranked #20 on
Conditional Image Generation
on ImageNet 128x128
ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
Distribution alignment encourages the marginal distribution of predictions on unlabeled data to be close to the marginal distribution of ground-truth labels.
Multimodal, Multilingual Grapheme-to-Phoneme Conversion for Low-Resource Languages
Grapheme-to-phoneme conversion (g2p) is the task of predicting the pronunciation of words from their orthographic representation.
Small-GAN: Speeding Up GAN Training Using Core-sets
Recent work by Brock et al. (2018) suggests that Generative Adversarial Networks (GANs) benefit disproportionately from large mini-batch sizes.
Consistency Regularization for Generative Adversarial Networks
Generative Adversarial Networks (GANs) are known to be difficult to train, despite considerable research effort.
Ranked #5 on
Image Generation
on CelebA-HQ 128x128
Differentiable Combinatorial Losses through Generalized Gradients of Linear Programs
When samples have internal structure, we often see a mismatch between the objective optimized during training and the model's goal during inference.
Distilling Effective Supervision from Severe Label Noise
For instance, on CIFAR100 with a $40\%$ uniform noise ratio and only 10 trusted labeled data per class, our method achieves $80. 2{\pm}0. 3\%$ classification accuracy, where the error rate is only $1. 4\%$ higher than a neural network trained without label noise.
Distributed Equivalent Substitution Training for Large-Scale Recommender Systems
We present Distributed Equivalent Substitution (DES) training, a novel distributed training framework for large-scale recommender systems with dynamic sparse features.
Multiple instance dense connected convolution neural network for aerial image scene classification
While the current convolution neural network tends to extract global features and global semantic information in a scene, the geo-spatial objects can be located at anywhere in an aerial image scene and their spatial arrangement tends to be more complicated.
Building change detection based on multi-scale filtering and grid partition
Exhaustive experiments indicate that the proposed method can detect building change types directly and outperform the current multi-index learning method.
Approximation Capabilities of Neural ODEs and Invertible Residual Networks
Neural ODEs and i-ResNet are recently proposed methods for enforcing invertibility of residual neural models.
DynWalks: Global Topology and Recent Changes Awareness Dynamic Network Embedding
Dynamic network embedding aims to learn low dimensional embeddings for unseen and seen nodes by using any currently available snapshots of a dynamic network.
Brain Network Construction and Classification Toolbox (BrainNetClass)
The goal of this work is to introduce a toolbox namely "Brain Network Construction and Classification" (BrainNetClass) to the field to promote more advanced brain network construction methods.
ERNIE: Enhanced Representation through Knowledge Integration
We present a novel language representation model enhanced by knowledge called ERNIE (Enhanced Representation through kNowledge IntEgration).
Ranked #3 on
Natural Language Inference
on XNLI Chinese Dev
Chinese Named Entity Recognition
Chinese Sentence Pair Classification
+8
Signal Demodulation with Machine Learning Methods for Physical Layer Visible Light Communications: Prototype Platform, Open Dataset and Algorithms
The dataset is available online, which contains eight types of modulated signals.
Deep Learning for Signal Demodulation in Physical Layer Wireless Communications: Prototype Platform, Open Dataset, and Analytics
In this paper, we investigate deep learning (DL)-enabled signal demodulation methods and establish the first open dataset of real modulated signals for wireless communication systems.
Multi-Antenna Channel Interpolation via Tucker Decomposed Extreme Learning Machine
Channel interpolation is an essential technique for providing high-accuracy estimation of the channel state information (CSI) for wireless systems design where the frequency-space structural correlations of multi-antenna channel are typically hidden in matrix or tensor forms.
A Teacher-Student Framework for Maintainable Dialog Manager
Specifically, the {``}student{''} is an extended dialog manager based on a new ontology, and the {``}teacher{''} is existing resources used for guiding the learning process of the {``}student{''}.
A Unified Mammogram Analysis Method via Hybrid Deep Supervision
In this work, we present a unified mammogram analysis framework for both whole-mammogram classification and segmentation.
Deep Chronnectome Learning via Full Bidirectional Long Short-Term Memory Networks for MCI Diagnosis
Dynamic functional connectivity (dFC), consisting of time-varying spatiotemporal dynamics, may characterize "chronnectome" diagnostic information for improving MCI classification.
Self-Attention Generative Adversarial Networks
In this paper, we propose the Self-Attention Generative Adversarial Network (SAGAN) which allows attention-driven, long-range dependency modeling for image generation tasks.
Ranked #21 on
Conditional Image Generation
on ImageNet 128x128
(using extra training data)
Improving GANs Using Optimal Transport
We present Optimal Transport GAN (OT-GAN), a variant of generative adversarial nets minimizing a new metric measuring the distance between the generator distribution and the data distribution.
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks
In this paper, we propose an Attentional Generative Adversarial Network (AttnGAN) that allows attention-driven, multi-stage refinement for fine-grained text-to-image generation.
Ranked #1 on
Text-to-Image Generation
on MS-COCO
StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks
In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) aiming at generating high-resolution photo-realistic images.
Ranked #6 on
Text-to-Image Generation
on Oxford 102 Flowers
Link the head to the "beak": Zero Shot Learning from Noisy Text Description at Part Precision
We propose a learning framework that is able to connect text terms to its relevant parts and suppress connections to non-visual text terms without any part-text annotations.
SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation
Extensive experimental results demonstrate the effectiveness of the proposed SegAN with multi-scale loss: on BRATS 2013 SegAN gives performance comparable to the state-of-the-art for whole tumor and tumor core segmentation while achieves better precision and sensitivity for Gd-enhance tumor core segmentation; on BRATS 2015 SegAN achieves better performance than the state-of-the-art in both dice score and precision.
Ranked #1 on
Brain Tumor Segmentation
on BRATS-2013 leaderboard
StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks
Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications.
Ranked #4 on
Text-to-Image Generation
on Oxford 102 Flowers
(Inception score metric)
SPDA-CNN: Unifying Semantic Part Detection and Abstraction for Fine-Grained Recognition
In this paper, we propose a new CNN architecture that integrates semantic part detection and abstraction (SPDA-CNN) for fine-grained classification.
Multi-lingual Geoparsing based on Machine Translation
Our results for geoparsing Chinese and Arabic text using our multi-lingual geoparsing method are comparable to our results for geoparsing English text with our English tools.
