Search Results for author:
Found 69 papers, 37 papers with code
Speculative Contrastive Decoding
We proposed Speculative Contrastive Decoding (SCD), an accelerated decoding method leveraging the natural contrast between expert and amateur models in speculative decoding.
Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models
Zooter shows computation efficiency in inference as it introduces only a minor computation overhead of a routing function compared with reward model ranking methods.
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
Recently, instruction-following audio-language models have received broad attention for audio interaction with humans.
Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning
The key to our data sampling technique lies in the enhancement of diversity in the chosen subsets, as the model selects new data points most distinct from any existing ones according to its current embedding space.
Flexible uniform-sampling foveated Fourier single-pixel imaging
At a sampling ratio of 0. 0084 referring to HR FSI with 1024*768 pixels, experimentally, by UFFSI with 255*341 cells of 89% reduction in data redundancy, the ROI has a significantly better imaging quality to meet imaging needs.
OccuQuest: Mitigating Occupational Bias for Inclusive Large Language Models
Furthermore, we assemble three test sets for comprehensive evaluation, an occu-test set covering 25 occupational categories, an estate set focusing on real estate, and an occu-quora set containing real-world questions from Quora.
Query and Response Augmentation Cannot Help Out-of-domain Math Reasoning Generalization
In this paper, we conduct an investigation for such data augmentation in math reasoning and are intended to answer: (1) What strategies of data augmentation are more effective; (2) What is the scaling relationship between the amount of augmented data and model performance; and (3) Can data augmentation incentivize generalization to out-of-domain mathematical reasoning tasks?
Ranked #21 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
In this study, we specifically focus on the data composition between mathematical reasoning, code generation, and general human-aligning abilities during SFT.
LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT
In this paper, we propose LauraGPT, a unified GPT model for audio recognition, understanding, and generation.
Qwen Technical Report
Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to humans.
TouchStone: Evaluating Vision-Language Models by Language Models
Large vision-language models (LVLMs) have recently witnessed rapid advancements, exhibiting a remarkable capacity for perceiving, understanding, and processing visual information by connecting visual receptor with large language models (LLMs).
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images.
Ranked #2 on
Visual Question Answering (VQA)
on CORE-MM
#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models
Based on this observation, we propose a data selector based on InsTag to select 6K diverse and complex samples from open-source datasets and fine-tune models on InsTag-selected data.
Scaling Relationship on Learning Mathematical Reasoning with Large Language Models
We find with augmented samples containing more distinct reasoning paths, RFT improves mathematical reasoning performance more for LLMs.
Ranked #47 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
JOTR: 3D Joint Contrastive Learning with Transformers for Occluded Human Mesh Recovery
Our method includes an encoder-decoder transformer architecture to fuse 2D and 3D representations for achieving 2D$\&$3D aligned results in a coarse-to-fine manner and a novel 3D joint contrastive learning approach for adding explicitly global supervision for the 3D feature space.
TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering
However, such SPC-based representation i) optimizes under the volatile observation space which leads to the pose-misalignment between training and inference stages, and ii) lacks the global relationships among human parts that is critical for handling the incomplete painted SMPL.
ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
The answering quality of an aligned large language model (LLM) can be drastically improved if treated with proper crafting of prompts.
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
In this work, we explore a scalable way for building a general representation model toward unlimited modalities.
Ranked #1 on
Audio-Visual Question Answering (AVQA)
on AVQA
(using extra training data)
Video Frame Interpolation with Densely Queried Bilateral Correlation
To better model correlations and to produce more accurate motion fields, we propose the Densely Queried Bilateral Correlation (DQBC) that gets rid of the receptive field dependency problem and thus is more friendly to small and fast-moving objects.
Ranked #1 on
Video Frame Interpolation
on MSU Video Frame Interpolation
(VMAF metric)
Global-to-Local Modeling for Video-based 3D Human Pose and Shape Estimation
However, using a single kind of modeling structure is difficult to balance the learning of short-term and long-term temporal correlations, and may bias the network to one of them, leading to undesirable predictions like global location shift, temporal inconsistency, and insufficient local details.
Ranked #38 on
3D Human Pose Estimation
on 3DPW
Binary Embedding-based Retrieval at Tencent
To tackle the challenge, we propose a binary embedding-based retrieval (BEBR) engine equipped with a recurrent binarization algorithm that enables customized bits per dimension.
Transferring General Multimodal Pretrained Models to Text Recognition
This paper proposes a new method, OFA-OCR, to transfer multimodal pretrained models to text recognition.
OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist Models
As a starting point, we provide presets of 7 different modalities and 23 highly-diverse example tasks in OFASys, with which we also develop a first-in-kind, single model, OFA+, that can handle text, image, speech, video, and motion data.
Pretrained Diffusion Models for Unified Human Motion Synthesis
Generative modeling of human motion has broad applications in computer animation, virtual reality, and robotics.
MMSpeech: Multi-modal Multi-task Encoder-Decoder Pre-training for Speech Recognition
Therefore, we propose to introduce the phoneme modality into pre-training, which can help capture modality-invariant information between Mandarin speech and text.
Ranked #1 on
Speech Recognition
on AISHELL-1
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Contextual Expressive Text-to-Speech
To achieve this task, we construct a synthetic dataset and develop an effective framework.
Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese
The tremendous success of CLIP (Radford et al., 2021) has promoted the research and application of contrastive learning for vision-language pretraining.
Ranked #1 on
Zero-shot Image Retrieval
on MUGE Retrieval
Respecting Transfer Gap in Knowledge Distillation
The network response serves as additional supervision to formulate the machine domain, which uses the data collected from the human domain as a transfer set.
Prompt Tuning for Generative Multimodal Pretrained Models
Prompt tuning has become a new paradigm for model tuning and it has demonstrated success in natural language pretraining and even vision pretraining.
Ranked #2 on
Visual Entailment
on SNLI-VE test
Single Stage Virtual Try-on via Deformable Attention Flows
Virtual try-on aims to generate a photo-realistic fitting result given an in-shop garment and a reference person image.
Ranked #3 on
Virtual Try-on
on VITON
Instance-wise Prompt Tuning for Pretrained Language Models
Prompt Learning has recently gained great popularity in bridging the gap between pretraining tasks and various downstream tasks.
M6-Fashion: High-Fidelity Multi-modal Image Generation and Editing
The fashion industry has diverse applications in multi-modal image generation and editing.
M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems
Industrial recommender systems have been growing increasingly complex, may involve \emph{diverse domains} such as e-commerce products and user-generated contents, and can comprise \emph{a myriad of tasks} such as retrieval, ranking, explanation generation, and even AI-assisted content production.
In-N-Out Generative Learning for Dense Unsupervised Video Segmentation
By contrast, pixel-level optimization is more explicit, however, it is sensitive to the visual quality of training data and is not robust to object deformation.
Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)
Recently, it has been observed that the best uni-modal network outperforms the jointly trained multi-modal network, which is counter-intuitive since multiple signals generally bring more information.
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
In this work, we pursue a unified paradigm for multimodal pretraining to break the scaffolds of complex task/modality-specific customization.
Ranked #1 on
Visual Question Answering
on VQA v2 test-std
(yes/no metric)
Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understanding of their advancements.
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
Recent expeditious developments in deep learning algorithms, distributed training, and even hardware design for large models have enabled training extreme-scale models, say GPT-3 and Switch Transformer possessing hundreds of billions or even trillions of parameters.
Causal Attention for Unbiased Visual Recognition
Attention module does not always help deep models learn causal features that are robust in any confounding context, e. g., a foreground object feature is invariant to different backgrounds.
CSRNet: Cascaded Selective Resolution Network for Real-time Semantic Segmentation
Real-time semantic segmentation has received considerable attention due to growing demands in many practical applications, such as autonomous vehicles, robotics, etc.
Learning to Rehearse in Long Sequence Memorization
Further, we design a history sampler to select informative fragments for rehearsal training, making the memory focus on the crucial information.
Connecting Language and Vision for Natural Language-Based Vehicle Retrieval
In this paper, we apply one new modality, i. e., the language description, to search the vehicle of interest and explore the potential of this task in the real-world scenario.
Controllable Gradient Item Retrieval
A better way is to present a sequence of products with increasingly floral attributes based on the white dress, and allow the customer to select the most satisfactory one from the sequence.
Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation
Experimental results demonstrate that our method outperforms the previous state-of-the-art methods in both automatic and human evaluation, especially on coverage and faithfulness.
M6-T: Exploring Sparse Expert Models and Beyond
Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost, and thus it has become a trend in model scaling.
M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformers
Conditional image synthesis aims to create an image according to some multi-modal guidance in the forms of textual descriptions, reference images, and image blocks to preserve, as well as their combinations.
CogView: Mastering Text-to-Image Generation via Transformers
Text-to-Image generation in the general domain has long been an open problem, which requires both a powerful generative model and cross-modal understanding.
Ranked #55 on
Text-to-Image Generation
on COCO
(using extra training data)
UFC-BERT: Unifying Multi-Modal Controls for Conditional Image Synthesis
Conditional image synthesis aims to create an image according to some multi-modal guidance in the forms of textual descriptions, reference images, and image blocks to preserve, as well as their combinations.
M6: A Chinese Multimodal Pretrainer
In this work, we construct the largest dataset for multimodal pretraining in Chinese, which consists of over 1. 9TB images and 292GB texts that cover a wide range of domains.
CogDL: A Comprehensive Library for Graph Deep Learning
In CogDL, we propose a unified design for the training and evaluation of GNN models for various graph tasks, making it unique among existing graph learning libraries.
Continual Memory: Can We Reason After Long-Term Memorization?
Existing reasoning tasks often follow the setting of "reasoning while experiencing", which has an important assumption that the raw contents can be always accessed while reasoning.
VideoFlow: A Framework for Building Visual Analysis Pipelines
The past years have witnessed an explosion of deep learning frameworks like PyTorch and TensorFlow since the success of deep neural networks.
Local Clustering Graph Neural Networks
Based on the theoretical analysis, we propose Local Clustering Graph Neural Networks (LCGNN), a GNN learning paradigm that utilizes local clustering to efficiently search for small but compact subgraphs for GNN training and inference.
CogLTX: Applying BERT to Long Texts
BERTs are incapable of processing long texts due to its quadratically increasing memory and time consumption.
Disentangled Self-Supervision in Sequential Recommenders
There exist two challenges: i) reconstructing a future sequence containing many behaviors is exponentially harder than reconstructing a single next behavior, which can lead to difficulty in convergence, and ii) the sequence of all future behaviors can involve many intentions, not all of which may be predictable from the sequence of earlier behaviors.
Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems
Deep candidate generation (DCG) that narrows down the collection of relevant items from billions to hundreds via representation learning has become prevalent in industrial recommender systems.
Understanding Negative Sampling in Graph Representation Learning
To the best of our knowledge, we are the first to derive the theory and quantify that the negative sampling distribution should be positively but sub-linearly correlated to their positive sampling distribution.
Controllable Multi-Interest Framework for Recommendation
Recent works usually give an overall embedding from a user's behavior sequence.
Variational Autoencoders for Highly Multivariate Spatial Point Processes Intensities
To bridge this gap, we introduce a declustering based hidden variable model that leads to an efficient inference procedure via a variational autoencoder (VAE).
ExperienceThinking: Constrained Hyperparameter Optimization based on Knowledge and Pruning
Motivated by this, we propose ExperienceThinking algorithm to quickly find the best possible hyperparameter configuration of machine learning algorithms within a few configuration evaluations.
Learning Disentangled Representations for Recommendation
Our approach achieves macro disentanglement by inferring the high-level concepts associated with user intentions (e. g., to buy a shirt or a cellphone), while capturing the preference of a user regarding the different concepts separately.
Dimensional Reweighting Graph Convolution Networks
In this paper, we propose a method named Dimensional reweighting Graph Convolutional Networks (DrGCNs), to tackle the problem of variance between dimensional information in the node representations of GCNs.
Dimensional Reweighting Graph Convolutional Networks
Graph Convolution Networks (GCNs) are becoming more and more popular for learning node representations on graphs.
Cognitive Knowledge Graph Reasoning for One-shot Relational Learning
Inferring new facts from existing knowledge graphs (KG) with explainable reasoning processes is a significant problem and has received much attention recently.
Cognitive Graph for Multi-Hop Reading Comprehension at Scale
We propose a new CogQA framework for multi-hop question answering in web-scale documents.
Ranked #54 on
Question Answering
on HotpotQA
Multi-hop Question Answering
Multi-Hop Reading Comprehension
+1
Personalized Bundle List Recommendation
In particular, BGN improves the precision of the best competitors by 16\% on average while maintaining the highest diversity on four datasets, and yields a 3. 85x improvement of response time over the best competitors in the bundle list recommendation problem.
AliGraph: A Comprehensive Graph Neural Network Platform
An increasing number of machine learning tasks require dealing with large graph datasets, which capture rich and complex relationship among potentially billions of elements.
Distributed, Parallel, and Cluster Computing
Deep Interest Evolution Network for Click-Through Rate Prediction
Easy-to-use, Modular and Extendible package of deep-learning based CTR models. DeepFM, DeepInterestNetwork(DIN), DeepInterestEvolutionNetwork(DIEN), DeepCrossNetwork(DCN), AttentionalFactorizationMachine(AFM), Neural Factorization Machine(NFM), AutoInt
Ranked #1 on
Click-Through Rate Prediction
on Amazon Dataset
ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation
Downstream applications then can use the user behavior vectors via vanilla attention.
