Search Results for author:
Found 100 papers, 56 papers with code
GCoT: Chain-of-Thought Prompt Learning for Graphs
Specifically, we decompose the adaptation process for each downstream task into a series of inference steps, with each step consisting of prompt-based inference, ``thought'' generation, and thought-conditioned prompt learning.
SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation
This raises an important question: How can we train a graph foundational model on multiple source domains and adapt to an unseen target domain?
Dynamic Adaptation in Data Storage: Real-Time Machine Learning for Enhanced Prefetching
The exponential growth of data storage demands has necessitated the evolution of hierarchical storage management strategies [1].
Dynamic Optimization of Storage Systems Using Reinforcement Learning Techniques
Traditional heuristics employed for storage performance optimization often fail to adapt to the variability and complexity of contemporary workloads, leading to significant performance bottlenecks and resource inefficiencies.
Optimizing SSD Caches for Cloud Block Storage Systems Using Machine Learning Approaches
The management of data writes to SSD caches plays a crucial role in improving overall system performance, reducing latency, and extending the lifespan of storage devices.
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
We present the Qwen2-VL Series, an advanced upgrade of the previous Qwen-VL models that redefines the conventional predetermined-resolution approach in visual processing.
Ranked #3 on
Video Question Answering
on TVBench
Natural Language Visual Grounding
Temporal Relation Extraction
+2
IW-Bench: Evaluating Large Multimodal Models for Converting Image-to-Web
We have conducted extensive experiments on existing large multimodal models, offering insights into their performance and areas for improvement in image-to-web domain.
Predicting Rewards Alongside Tokens: Non-disruptive Parameter Insertion for Efficient Inference Intervention in Large Language Model
Transformer-based large language models (LLMs) exhibit limitations such as generating unsafe responses, unreliable reasoning, etc.
Synthesizing Text-to-SQL Data from Weak and Strong LLMs
The capability gap between open-source and closed-source large language models (LLMs) remains a challenge in text-to-SQL tasks.
Qwen2-Audio Technical Report
We introduce the latest progress of Qwen-Audio, a large-scale audio-language model called Qwen2-Audio, which is capable of accepting various audio signal inputs and performing audio analysis or direct textual responses with regard to speech instructions.
Qwen2 Technical Report
This report introduces the Qwen2 series, the latest addition to our large language models and large multimodal models.
Ranked #3 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Multi-Scenario Combination Based on Multi-Agent Reinforcement Learning to Optimize the Advertising Recommendation System
This paper explores multi-scenario optimization on large platforms using multi-agent reinforcement learning (MARL).
LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback
In recent progress, mathematical verifiers have achieved success in mathematical reasoning tasks by validating the correctness of solutions generated by policy models.
Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models
AutoIF transforms the validation of instruction-following data quality into code verification, requiring LLMs to generate instructions, the corresponding code to check the correctness of the instruction responses, and unit test samples to verify the code's correctness.
Ranked #1 on
Instruction Following
on IFEval
Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones?
Furthermore, we introduce the concept of consistency score to quantitatively measure this inconsistency and analyze the potential for improvement in consistency by relative consistency score.
Predict Click-Through Rates with Deep Interest Network Model in E-commerce Advertising
This paper proposes new methods to enhance click-through rate (CTR) prediction models using the Deep Interest Network (DIN) model, specifically applied to the advertising system of Alibaba's Taobao platform.
Research on Driver Facial Fatigue Detection Based on Yolov8 Model
In a society where traffic accidents frequently occur, fatigue driving has emerged as a grave issue.
Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment
Effectively aligning Large Language Models (LLMs) with human-centric values while preventing the degradation of abilities acquired through Pre-training and Supervised Fine-tuning (SFT) poses a central challenge in Reinforcement Learning from Human Feedback (RLHF).
Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models
To address these issues, we propose MDGPT, a text free Multi-Domain Graph Pre-Training and adaptation framework designed to exploit multi-domain knowledge for graph learning.
Optimizing Search Advertising Strategies: Integrating Reinforcement Learning with Generalized Second-Price Auctions for Enhanced Ad Ranking and Bidding
This paper explores the integration of strategic optimization methods in search advertising, focusing on ad ranking and bidding mechanisms within E-commerce platforms.
Language Models can Evaluate Themselves via Probability Discrepancy
In this paper, we initiate our discussion by demonstrating how Large Language Models (LLMs), when tasked with responding to queries, display a more even probability distribution in their answers if they are more adept, as opposed to their less skilled counterparts.
Improved AdaBoost for Virtual Reality Experience Prediction Based on Long Short-Term Memory Network
The confusion matrix for the training set shows a total of 177 correct predictions and 52 incorrect predictions, with an accuracy of 77%, precision of 88%, recall of 77% and f1 score of 82%.
Research on Image Recognition Technology Based on Multimodal Deep Learning
This project investigates the human multi-modal behavior identification algorithm utilizing deep neural networks.
An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models
To this end, we introduce FastV, a versatile plug-and-play method designed to optimize computational efficiency by learning adaptive attention patterns in early layers and pruning visual tokens in subsequent ones.
Training-Free Long-Context Scaling of Large Language Models
The ability of Large Language Models (LLMs) to process and generate coherent text is markedly weakened when the number of input tokens exceeds their pretraining length.
Traffic Control via Connected and Automated Vehicles: An Open-Road Field Experiment with 100 CAVs
The upper layer is called Speed Planner, and is a centralized optimal control algorithm.
AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension
By revealing the limitations of existing LALMs through evaluation results, AIR-Bench can provide insights into the direction of future research.
Sketch and Refine: Towards Fast and Accurate Lane Detection
At the "Sketch" stage, local directions of keypoints can be easily estimated by fast convolutional layers.
Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment
Nevertheless, we posit that LLMs inherently harbor role-play capabilities, owing to the extensive knowledge of characters and potential dialogues ingrained in their vast training corpora.
Controllable 3D Face Generation with Conditional Style Code Diffusion
For 3D GAN inversion, we introduce two methods which aim to enhance the representation of style codes and alleviate 3D inconsistencies.
MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs
Hence, in this paper, we propose MultiGPrompt, a novel multi-task pre-training and prompting framework to exploit multiple pretext tasks for more comprehensive pre-trained knowledge.
Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models
Zooter shows computation efficiency in inference as it introduces only a minor computation overhead of a routing function compared with reward model ranking methods.
Speculative Contrastive Decoding
Large language models~(LLMs) exhibit exceptional performance in language tasks, yet their auto-regressive inference is limited due to high computational requirements and is sub-optimal due to the exposure bias.
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
Recently, instruction-following audio-language models have received broad attention for audio interaction with humans.
Ranked #1 on
Acoustic Scene Classification
on TUT Acoustic Scenes 2017
(using extra training data)
Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning
The key to our data sampling technique lies in the enhancement of diversity in the chosen subsets, as the model selects new data points most distinct from any existing ones according to its current embedding space.
Flexible uniform-sampling foveated Fourier single-pixel imaging
At a sampling ratio of 0. 0084 referring to HR FSI with 1024*768 pixels, experimentally, by UFFSI with 255*341 cells of 89% reduction in data redundancy, the ROI has a significantly better imaging quality to meet imaging needs.
OccuQuest: Mitigating Occupational Bias for Inclusive Large Language Models
Furthermore, we assemble three test sets for comprehensive evaluation, an occu-test set covering 25 occupational categories, an estate set focusing on real estate, and an occu-quora set containing real-world questions from Quora.
How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
We propose four intriguing research questions to explore the association between model performance and various factors including data amount, composition ratio, model size and SFT strategies.
MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning
In this paper, we conduct an investigation for such data augmentation in math reasoning and are intended to answer: (1) What strategies of data augmentation are more effective; (2) What is the scaling relationship between the amount of augmented data and model performance; and (3) Can data augmentation incentivize generalization to out-of-domain mathematical reasoning tasks?
Ranked #60 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT
Previous mainstream audio-and-text LLMs use discrete audio tokens to represent both input and output audio; however, they suffer from performance degradation on tasks such as automatic speech recognition, speech-to-text translation, and speech enhancement over models using continuous speech features.
Qwen Technical Report
Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to humans.
Ranked #3 on
Multi-Label Text Classification
on CC3M-TagMask
TouchStone: Evaluating Vision-Language Models by Language Models
Large vision-language models (LVLMs) have recently witnessed rapid advancements, exhibiting a remarkable capacity for perceiving, understanding, and processing visual information by connecting visual receptor with large language models (LLMs).
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images.
Ranked #2 on
Spatial Reasoning
on EmbSpatial-Bench
#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models
Based on this observation, we propose a data selector based on InsTag to select 6K diverse and complex samples from open-source datasets and fine-tune models on InsTag-selected data.
Scaling Relationship on Learning Mathematical Reasoning with Large Language Models
We find with augmented samples containing more distinct reasoning paths, RFT improves mathematical reasoning performance more for LLMs.
Ranked #111 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
JOTR: 3D Joint Contrastive Learning with Transformers for Occluded Human Mesh Recovery
Our method includes an encoder-decoder transformer architecture to fuse 2D and 3D representations for achieving 2D$\&$3D aligned results in a coarse-to-fine manner and a novel 3D joint contrastive learning approach for adding explicitly global supervision for the 3D feature space.
TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering
However, such SPC-based representation i) optimizes under the volatile observation space which leads to the pose-misalignment between training and inference stages, and ii) lacks the global relationships among human parts that is critical for handling the incomplete painted SMPL.
ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
The answering quality of an aligned large language model (LLM) can be drastically improved if treated with proper crafting of prompts.
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
In this work, we explore a scalable way for building a general representation model toward unlimited modalities.
Ranked #1 on
Semantic Segmentation
on ADE20K
(using extra training data)
Video Frame Interpolation with Densely Queried Bilateral Correlation
To better model correlations and to produce more accurate motion fields, we propose the Densely Queried Bilateral Correlation (DQBC) that gets rid of the receptive field dependency problem and thus is more friendly to small and fast-moving objects.
Ranked #1 on
Video Frame Interpolation
on MSU Video Frame Interpolation
(VMAF metric)
Global-to-Local Modeling for Video-based 3D Human Pose and Shape Estimation
However, using a single kind of modeling structure is difficult to balance the learning of short-term and long-term temporal correlations, and may bias the network to one of them, leading to undesirable predictions like global location shift, temporal inconsistency, and insufficient local details.
Ranked #56 on
3D Human Pose Estimation
on 3DPW
Binary Embedding-based Retrieval at Tencent
To tackle the challenge, we propose a binary embedding-based retrieval (BEBR) engine equipped with a recurrent binarization algorithm that enables customized bits per dimension.
Transferring General Multimodal Pretrained Models to Text Recognition
This paper proposes a new method, OFA-OCR, to transfer multimodal pretrained models to text recognition.
OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist Models
As a starting point, we provide presets of 7 different modalities and 23 highly-diverse example tasks in OFASys, with which we also develop a first-in-kind, single model, OFA+, that can handle text, image, speech, video, and motion data.
Pretrained Diffusion Models for Unified Human Motion Synthesis
Generative modeling of human motion has broad applications in computer animation, virtual reality, and robotics.
MMSpeech: Multi-modal Multi-task Encoder-Decoder Pre-training for Speech Recognition
Therefore, we propose to introduce the phoneme modality into pre-training, which can help capture modality-invariant information between Mandarin speech and text.
Ranked #4 on
Speech Recognition
on AISHELL-1
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Contextual Expressive Text-to-Speech
To achieve this task, we construct a synthetic dataset and develop an effective framework.
Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese
The tremendous success of CLIP (Radford et al., 2021) has promoted the research and application of contrastive learning for vision-language pretraining.
Ranked #1 on
Zero-shot Image Retrieval
on MUGE Retrieval
Respecting Transfer Gap in Knowledge Distillation
The network response serves as additional supervision to formulate the machine domain, which uses the data collected from the human domain as a transfer set.
Prompt Tuning for Generative Multimodal Pretrained Models
Prompt tuning has become a new paradigm for model tuning and it has demonstrated success in natural language pretraining and even vision pretraining.
Ranked #2 on
Visual Entailment
on SNLI-VE test
Single Stage Virtual Try-on via Deformable Attention Flows
Virtual try-on aims to generate a photo-realistic fitting result given an in-shop garment and a reference person image.
Ranked #3 on
Virtual Try-on
on VITON
Instance-wise Prompt Tuning for Pretrained Language Models
Prompt Learning has recently gained great popularity in bridging the gap between pretraining tasks and various downstream tasks.
M6-Fashion: High-Fidelity Multi-modal Image Generation and Editing
The fashion industry has diverse applications in multi-modal image generation and editing.
M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems
Industrial recommender systems have been growing increasingly complex, may involve \emph{diverse domains} such as e-commerce products and user-generated contents, and can comprise \emph{a myriad of tasks} such as retrieval, ranking, explanation generation, and even AI-assisted content production.
In-N-Out Generative Learning for Dense Unsupervised Video Segmentation
By contrast, pixel-level optimization is more explicit, however, it is sensitive to the visual quality of training data and is not robust to object deformation.
Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)
Recently, it has been observed that the best uni-modal network outperforms the jointly trained multi-modal network, which is counter-intuitive since multiple signals generally bring more information.
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
In this work, we pursue a unified paradigm for multimodal pretraining to break the scaffolds of complex task/modality-specific customization.
Ranked #1 on
Visual Question Answering
on VQA v2 test-std
(yes/no metric)
Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understanding of their advancements.
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
Recent expeditious developments in deep learning algorithms, distributed training, and even hardware design for large models have enabled training extreme-scale models, say GPT-3 and Switch Transformer possessing hundreds of billions or even trillions of parameters.
Causal Attention for Unbiased Visual Recognition
Attention module does not always help deep models learn causal features that are robust in any confounding context, e. g., a foreground object feature is invariant to different backgrounds.
CSRNet: Cascaded Selective Resolution Network for Real-time Semantic Segmentation
Real-time semantic segmentation has received considerable attention due to growing demands in many practical applications, such as autonomous vehicles, robotics, etc.
Learning to Rehearse in Long Sequence Memorization
Further, we design a history sampler to select informative fragments for rehearsal training, making the memory focus on the crucial information.
Controllable Gradient Item Retrieval
A better way is to present a sequence of products with increasingly floral attributes based on the white dress, and allow the customer to select the most satisfactory one from the sequence.
Connecting Language and Vision for Natural Language-Based Vehicle Retrieval
In this paper, we apply one new modality, i. e., the language description, to search the vehicle of interest and explore the potential of this task in the real-world scenario.
M6-T: Exploring Sparse Expert Models and Beyond
Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost, and thus it has become a trend in model scaling.
Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation
Experimental results demonstrate that our method outperforms the previous state-of-the-art methods in both automatic and human evaluation, especially on coverage and faithfulness.
M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformers
Conditional image synthesis aims to create an image according to some multi-modal guidance in the forms of textual descriptions, reference images, and image blocks to preserve, as well as their combinations.
CogView: Mastering Text-to-Image Generation via Transformers
Text-to-Image generation in the general domain has long been an open problem, which requires both a powerful generative model and cross-modal understanding.
Ranked #53 on
Text-to-Image Generation
on MS COCO
(using extra training data)
UFC-BERT: Unifying Multi-Modal Controls for Conditional Image Synthesis
Conditional image synthesis aims to create an image according to some multi-modal guidance in the forms of textual descriptions, reference images, and image blocks to preserve, as well as their combinations.
M6: A Chinese Multimodal Pretrainer
In this work, we construct the largest dataset for multimodal pretraining in Chinese, which consists of over 1. 9TB images and 292GB texts that cover a wide range of domains.
CogDL: A Comprehensive Library for Graph Deep Learning
In CogDL, we propose a unified design for the training and evaluation of GNN models for various graph tasks, making it unique among existing graph learning libraries.
VideoFlow: A Framework for Building Visual Analysis Pipelines
The past years have witnessed an explosion of deep learning frameworks like PyTorch and TensorFlow since the success of deep neural networks.
Local Clustering Graph Neural Networks
Based on the theoretical analysis, we propose Local Clustering Graph Neural Networks (LCGNN), a GNN learning paradigm that utilizes local clustering to efficiently search for small but compact subgraphs for GNN training and inference.
Continual Memory: Can We Reason After Long-Term Memorization?
Existing reasoning tasks often follow the setting of "reasoning while experiencing", which has an important assumption that the raw contents can be always accessed while reasoning.
CogLTX: Applying BERT to Long Texts
BERTs are incapable of processing long texts due to its quadratically increasing memory and time consumption.
Disentangled Self-Supervision in Sequential Recommenders
There exist two challenges: i) reconstructing a future sequence containing many behaviors is exponentially harder than reconstructing a single next behavior, which can lead to difficulty in convergence, and ii) the sequence of all future behaviors can involve many intentions, not all of which may be predictable from the sequence of earlier behaviors.
Understanding Negative Sampling in Graph Representation Learning
To the best of our knowledge, we are the first to derive the theory and quantify that the negative sampling distribution should be positively but sub-linearly correlated to their positive sampling distribution.
Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems
Deep candidate generation (DCG) that narrows down the collection of relevant items from billions to hundreds via representation learning has become prevalent in industrial recommender systems.
Controllable Multi-Interest Framework for Recommendation
Recent works usually give an overall embedding from a user's behavior sequence.
Variational Autoencoders for Highly Multivariate Spatial Point Processes Intensities
To bridge this gap, we introduce a declustering based hidden variable model that leads to an efficient inference procedure via a variational autoencoder (VAE).
ExperienceThinking: Constrained Hyperparameter Optimization based on Knowledge and Pruning
Motivated by this, we propose ExperienceThinking algorithm to quickly find the best possible hyperparameter configuration of machine learning algorithms within a few configuration evaluations.
Learning Disentangled Representations for Recommendation
Our approach achieves macro disentanglement by inferring the high-level concepts associated with user intentions (e. g., to buy a shirt or a cellphone), while capturing the preference of a user regarding the different concepts separately.
Dimensional Reweighting Graph Convolution Networks
In this paper, we propose a method named Dimensional reweighting Graph Convolutional Networks (DrGCNs), to tackle the problem of variance between dimensional information in the node representations of GCNs.
Dimensional Reweighting Graph Convolutional Networks
Graph Convolution Networks (GCNs) are becoming more and more popular for learning node representations on graphs.
Cognitive Knowledge Graph Reasoning for One-shot Relational Learning
Inferring new facts from existing knowledge graphs (KG) with explainable reasoning processes is a significant problem and has received much attention recently.
Cognitive Graph for Multi-Hop Reading Comprehension at Scale
We propose a new CogQA framework for multi-hop question answering in web-scale documents.
Ranked #50 on
Question Answering
on HotpotQA
Personalized Bundle List Recommendation
In particular, BGN improves the precision of the best competitors by 16\% on average while maintaining the highest diversity on four datasets, and yields a 3. 85x improvement of response time over the best competitors in the bundle list recommendation problem.
AliGraph: A Comprehensive Graph Neural Network Platform
An increasing number of machine learning tasks require dealing with large graph datasets, which capture rich and complex relationship among potentially billions of elements.
Distributed, Parallel, and Cluster Computing
Deep Interest Evolution Network for Click-Through Rate Prediction
Easy-to-use, Modular and Extendible package of deep-learning based CTR models. DeepFM, DeepInterestNetwork(DIN), DeepInterestEvolutionNetwork(DIEN), DeepCrossNetwork(DCN), AttentionalFactorizationMachine(AFM), Neural Factorization Machine(NFM), AutoInt
Ranked #1 on
Click-Through Rate Prediction
on Amazon Dataset
ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation
Downstream applications then can use the user behavior vectors via vanilla attention.
