Search Results for author:
Found 238 papers, 152 papers with code
Finding the Dominant Winning Ticket in Pre-Trained Language Models
Empirically, we show that (a) the dominant winning ticket can achieve performance that is comparable with that of the full-parameter model, (b) the dominant winning ticket is transferable across different tasks, (c) and the dominant winning ticket has a natural structure within each parameter matrix.
Great~Truths~are ~Always ~Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models
Commonsense reasoning in natural language is a desired ability of artificial intelligent systems.
There Are a Thousand Hamlets in a Thousand People’s Eyes: Enhancing Knowledge-grounded Dialogue with Personal Memory
Knowledge-grounded conversation (KGC) shows great potential in building an engaging and knowledgeable chatbot, and knowledge selection is a key ingredient in it.
Optimal Partial Transport Based Sentence Selection for Long-form Document Matching
Enjoying the partial transport properties of OPT, the selected key sentences can not only effectively enhance the matching accuracy, but also be explained as the rationales for the matching results.
Semantic Sentence Matching via Interacting Syntax Graphs
In this paper, we formalize the task of semantic sentence matching as a problem of graph matching in which each sentence is represented as a directed graph according to its syntactic structures.
Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network
In this paper, we study how to continually pre-train language models for improving the understanding of math problems.
Enhancing Graph Contrastive Learning with Reliable and Informative Augmentation for Recommendation
Based on these discrete codes, we enhance the collaborative information of contrastive views by considering neighborhood structure and semantic relevance respectively.
Modeling Domain and Feedback Transitions for Cross-Domain Sequential Recommendation
In this paper, we propose $\text{Transition}^2$, a novel method to model transitions across both domains and types of user feedback.
Towards Effective and Efficient Continual Pre-training of Large Language Models
To make the CPT approach more traceable, this paper presents a technical report for continually pre-training Llama-3 (8B), which significantly enhances the Chinese language ability and scientific reasoning ability of the backbone model.
Very Large-Scale Multi-Agent Simulation in AgentScope
Recent advances in large language models (LLMs) have opened new avenues for applying multi-agent systems in very large-scale simulations.
Mix-CPT: A Domain Adaptation Framework via Decoupling Knowledge Learning and Format Alignment
However, this method may result in inefficient knowledge memorization due to a lack of awareness of knowledge utilization and imposes substantial demands on LLMs to simultaneously learn knowledge utilization and format alignment with limited training samples.
LLMBox: A Comprehensive Library for Large Language Models
To facilitate the research on large language models (LLMs), this paper presents a comprehensive and unified library, LLMBox, to ease the development, use, and evaluation of LLMs.
Discovering symbolic expressions with parallelized tree search
Symbolic regression plays a crucial role in modern scientific research thanks to its capability of discovering concise and interpretable mathematical expressions from data.
Query-oriented Data Augmentation for Session Search
However, this paradigm neglects the symmetric nature of the relevance between the session context and document, i. e., the clicked documents can also be paired with different search contexts when training.
YuLan: An Open-source Large Language Model
This paper presents the development of YuLan, a series of open-source LLMs with $12$ billion parameters.
Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation
Based on preference data, DPA-RAG accomplishes both external and internal preference alignment: 1) It jointly integrate pair-wise, point-wise, and contrastive preference alignment abilities into the reranker, achieving external preference alignment among RAG components.
Ranked #2 on
Knowledge Base Question Answering
on WebQuestionsSP
ICLEval: Evaluating In-Context Learning Ability of Large Language Models
In-Context Learning (ICL) is a critical capability of Large Language Models (LLMs) as it empowers them to comprehend and reason across interconnected inputs.
Towards Event-oriented Long Video Understanding
Leveraging an effective instruction synthesis method and an adaptive model architecture, VIM surpasses both state-of-the-art open-source models and GPT-4V on the Event-Bench.
Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning
The emergence of in-context learning (ICL) is potentially attributed to two major abilities: task recognition (TR) for recognizing the task from demonstrations and utilizing pre-trained priors, and task learning (TL) for learning from demonstrations.
Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models
Synthetic data has been proposed as a solution to address the issue of high-quality data scarcity in the training of large language models (LLMs).
Low-Redundant Optimization for Large Language Model Alignment
Concretely, we first identify the neurons that are related to the human preference data by a gradient-based strategy, then identify the alignment-related key tokens by reward models for computing loss.
Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector
Hallucination detection is a challenging task for large language models (LLMs), and existing studies heavily rely on powerful closed-source LLMs such as GPT-4.
Counteracting Duration Bias in Video Recommendation via Counterfactual Watch Time
In video recommendation, an ongoing effort is to satisfy users' personalized information needs by leveraging their logged watch time.
QAGCF: Graph Collaborative Filtering for Q&A Recommendation
Traditional recommendation methods treat the question-answer pair as a whole or only consider the answer as a single item, which overlooks the two challenges and cannot effectively model user interests.
Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training
In this paper, we introduce the Med-ST framework for fine-grained spatial and temporal modeling to exploit information from multiple spatial views of chest radiographs and temporal historical records.
One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models
To address this, we propose a novel method that involves learning scalable and pluggable virtual tokens for RAG.
Exploring Context Window of Large Language Models via Decomposed Positional Vectors
Based on our findings, we design two training-free context window extension methods, positional vector replacement and attention window extension.
Tool Learning with Large Language Models: A Survey
In this survey, we focus on reviewing existing literature from the two primary aspects (1) why tool learning is beneficial and (2) how tool learning is implemented, enabling a comprehensive understanding of tool learning with LLMs.
Source Echo Chamber: Exploring the Escalation of Source Bias in User, Data, and Recommender System Feedback Loop
How and to what extent the source bias affects the neural recommendation models within feedback loop remains unknown.
Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integration
The proliferation of Large Language Models (LLMs) has led to an influx of AI-generated content (AIGC) on the internet, transforming the corpus of Information Retrieval (IR) systems from solely human-written to a coexistence with LLM-generated content.
Towards Completeness-Oriented Tool Retrieval for Large Language Models
Existing tool retrieval methods primarily focus on semantic matching between user queries and tool descriptions, frequently leading to the retrieval of redundant, similar tools.
Ranked #1 on
Retrieval
on ToolLens
JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Models
We leverage it to synthesize 6 million math problems for pre-training our JiuZhang3. 0 model, which only needs to invoke GPT-4 API 9. 3k times and pre-train on 4. 6B data.
Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression
In this paper, we introduce \textbf{DecoQuant}, a novel data-free low-bit quantization technique based on tensor decomposition methods, to effectively compress KV cache.
A Survey on the Memory Mechanism of Large Language Model based Agents
Compared with original LLMs, LLM-based agents are featured in their self-evolving capability, which is the basis for solving real-world problems that need long-term and complex agent-environment interactions.
Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models
In real world, large language models (LLMs) can serve as the assistant to help users accomplish their jobs, and also support the development of advanced applications.
Dynamic Prompt Optimizing for Text-to-Image Generation
Users assign weights or alter the injection time steps of certain words in the text prompts to improve the quality of generated images.
Investigating the Robustness of Counterfactual Learning to Rank Models: A Reproducibility Study
Counterfactual learning to rank (CLTR) has attracted extensive attention in the IR community for its ability to leverage massive logged user interaction data to train ranking models.
Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models
In this paper, we delve into several mechanisms employed by Transformer-based language models (LLMs) for factual recall tasks.
EulerFormer: Sequential User Behavior Modeling with Complex Vector Attention
The core of transformer architecture lies in the self-attention mechanism, which computes the pairwise attention scores in a sequence.
Selecting Query-bag as Pseudo Relevance Feedback for Information-seeking Conversations
Given the user query, the information-seeking dialogue systems first retrieve a subset of response candidates, then further select the best response from the candidate set through re-ranking.
ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
In response to this challenge, we present an empirical investigation of CoT prompting and introduce CoTGenius, a novel framework designed for the automatic generation of superior CoT prompts.
A Large Language Model Enhanced Sequential Recommender for Joint Video and Comment Recommendation
Meanwhile, we leverage the LLM recommender as a supplemental component (discarded in deployment) to better capture underlying user preferences from heterogeneous interaction behaviors.
An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models
Based on these findings, we then propose several simple document pruning methods to reduce the storage overhead and compare the effectiveness of different pruning methods on different late-interaction models.
Less is More: Data Value Estimation for Visual Instruction Tuning
To investigate this issue, we conduct a series of empirical studies, which reveal a significant redundancy within the visual instruction datasets, and show that greatly reducing the amount of several instruction dataset even do not affect the performance.
Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models
In this paper, we study the harmlessness alignment problem of multimodal large language models (MLLMs).
StreamingDialogue: Prolonged Dialogue Learning via Long Context Compression with Minimal Losses
Accordingly, we introduce StreamingDialogue, which compresses long dialogue history into conv-attn sinks with minimal losses, and thus reduces computational complexity quadratically with the number of sinks (i. e., the number of utterances).
Masked Thought: Simply Masking Partial Reasoning Steps Can Improve Mathematical Reasoning Learning of Language Models
Furthermore, it is complementary to existing methods.
REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering
By combining the improvements in both architecture and training, our proposed REAR can better utilize external knowledge by effectively perceiving the relevance of retrieved documents.
BASES: Large-scale Web Search User Simulation with Large Language Model based Agents
Due to the excellent capacities of large language models (LLMs), it becomes feasible to develop LLM-based agents for reliable user simulation.
Unleashing the Potential of Large Language Models as Prompt Optimizers: An Analogical Analysis with Gradient-based Model Optimizers
Focused on the two aspects, we borrow the theoretical framework and learning methods from gradient-based optimization to design improved strategies for LLM-based prompt optimizers.
Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models
Large language models (LLMs) demonstrate remarkable multilingual capabilities without being pre-trained on specially curated multilingual parallel corpora.
UFO: a Unified and Flexible Framework for Evaluating Factuality of Large Language Models
To address these challenges, we categorize four available fact sources: human-written evidence, reference documents, search engine results, and LLM knowledge, along with five text generation tasks containing six representative datasets.
Large Language Model-based Human-Agent Collaboration for Complex Task Solving
We construct a human-agent collaboration dataset to train this policy model in an offline reinforcement learning environment.
Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs
The integration of large language models (LLMs) and search engines represents a significant evolution in knowledge acquisition methodologies.
KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graph
To guarantee the effectiveness, we leverage program language to formulate the multi-hop reasoning process over the KG, and synthesize a code-based instruction dataset to fine-tune the base LLM.
INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning
Despite this, their application to information retrieval (IR) tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language.
Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint
To address it, we propose a new RL method named RLMEC that incorporates a generative model as the reward model, which is trained by the erroneous solution rewriting task under the minimum editing constraint, and can produce token-level rewards for RL training.
Prompting Large Language Models for Recommender Systems: A Comprehensive Framework and Empirical Analysis
As for the use of LLMs as recommenders, we analyze the impact of public availability, tuning strategies, model architecture, parameter scale, and context length on recommendation results based on the classification of LLMs.
The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models
To tackle the LLM hallucination, three key questions should be well studied: how to detect hallucinations (detection), why do LLMs hallucinate (source), and what can be done to mitigate them (mitigation).
Distillation is All You Need for Practically Using Different Pre-trained Recommendation Models
Pre-trained recommendation models (PRMs) have attracted widespread attention recently.
ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph
To better perform reasoning on KG, recent work typically adopts a pre-trained language model~(PLM) to model the question, and a graph neural network~(GNN) based module to perform multi-hop reasoning on the KG.
A Survey of Reasoning with Foundation Models
Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation.
UFIN: Universal Feature Interaction Network for Multi-Domain Click-Through Rate Prediction
To address the above issue, we propose the Universal Feature Interaction Network (UFIN) approach for CTR prediction.
Scaling Law of Large Sequential Recommendation Models
We find that scaling up the model size can greatly boost the performance on these challenging tasks, which again verifies the benefits of large recommendation models.
Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation
To address this challenge, in this paper, we propose a new LLM-based recommendation model called LC-Rec, which can better integrate language and collaborative semantics for recommender systems.
Are We Falling in a Middle-Intelligence Trap? An Analysis and Mitigation of the Reversal Curse
Recent studies have highlighted a phenomenon in large language models (LLMs) known as "the reversal curse," in which the order of knowledge entities in the training data biases the models' comprehension.
AI-accelerated Discovery of Altermagnetic Materials
Since each altermagnetic material has a unique crystal structure, we propose an automated discovery approach empowered by an AI search engine that employs a pre-trained graph neural network to learn the intrinsic features of the material crystal structure, followed by fine-tuning a classifier with limited positive samples to predict the altermagnetism probability of a given material candidate.
Beyond Imitation: Leveraging Fine-grained Quality Signals for Alignment
Alignment with human preference is a desired property of large language models (LLMs).
Don't Make Your LLM an Evaluation Benchmark Cheater
Large language models~(LLMs) have greatly advanced the frontiers of artificial intelligence, attaining remarkable improvement in model capacity.
What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning
By conducting a comprehensive empirical study, we find that instructions focused on complex visual reasoning tasks are particularly effective in improving the performance of MLLMs on evaluation benchmarks.
DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy
Recent advances in large language models (LLMs) have revolutionized the landscape of reasoning tasks.
AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems
The optimized agents can also propagate their preferences to other agents in subsequent interactions, implicitly capturing the collaborative filtering idea.
BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models
Recently, multiple studies have committed to extending the context length and enhancing the long text modeling capabilities of LLMs.
Optimizing Factual Accuracy in Text Generation through Dynamic Knowledge Selection
Previous methods use external knowledge as references for text generation to enhance factuality but often struggle with the knowledge mix-up(e. g., entity mismatch) of irrelevant references.
A Survey on Large Language Model based Autonomous Agents
In this paper, we present a comprehensive survey of these studies, delivering a systematic review of the field of LLM-based autonomous agents from a holistic perspective.
Uncovering User Interest from Biased and Noised Watch Time in Video Recommendation
In the video recommendation, watch time is commonly adopted as an indicator of user interest.
Large Language Models for Information Retrieval: A Survey
This evolution requires a combination of both traditional methods (such as term-based sparse retrieval methods with rapid response) and modern neural architectures (such as language models with powerful language understanding capacity).
LTP-MMF: Towards Long-term Provider Max-min Fairness Under Recommendation Feedback Loops
Multi-stakeholder recommender systems involve various roles, such as users, and providers.
Counterfactual Cross-modality Reasoning for Weakly Supervised Video Moment Localization
Finally, by suppressing the unimodal effect of masked query, we can rectify the reconstructions of video proposals to perform reasonable contrastive learning.
Synthesizing Long-Term Human Motions with Diffusion Models via Coherent Sampling
Firstly, they cannot directly generate coherent motions and require additional operations such as interpolation to process the generated actions.
Spatio-Temporal Branching for Motion Prediction using Motion Increments
Human motion prediction (HMP) has emerged as a popular research topic due to its diverse applications, but it remains a challenging task due to the stochastic and aperiodic nature of future poses.
Alleviating the Long-Tail Problem in Conversational Recommender Systems
Conversational recommender systems (CRS) aim to provide the recommendation service via natural language conversations.
Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation
In this study, we present an initial analysis of the factual knowledge boundaries of LLMs and how retrieval augmentation affects LLMs on open-domain QA.
Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community
The research field of Information Retrieval (IR) has evolved significantly, expanding beyond traditional search to meet diverse user information needs.
Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study
Different from previous studies focused on overall performance, this work aims to investigate the impact of quantization on \emph{emergent abilities}, which are important characteristics that distinguish LLMs from small language models.
SSP: Self-Supervised Post-training for Conversational Search
Conversational search has been regarded as the next-generation search paradigm.
JiuZhang 2.0: A Unified Chinese Pre-trained Language Model for Multi-task Mathematical Problem Solving
Specially, we construct a Mixture-of-Experts~(MoE) architecture for modeling mathematical text, so as to capture the common mathematical knowledge across tasks.
RETA-LLM: A Retrieval-Augmented Large Language Model Toolkit
To support research in this area and facilitate the development of retrieval-augmented LLM systems, we develop RETA-LLM, a {RET}reival-{A}ugmented LLM toolkit.
Improving Conversational Recommendation Systems via Counterfactual Data Simulation
To develop our approach, we characterize user preference and organize the conversation flow by the entities involved in the dialogue, and design a multi-stage recommendation dialogue simulator based on a conversation flow language model.
User Behavior Simulation with Large Language Model based Agents
Simulating high quality user behavior data has always been a fundamental problem in human-centered applications, where the major difficulty originates from the intricate mechanism of human decision process.
Evaluating and Improving Tool-Augmented Computation-Intensive Math Reasoning
Based on this finding, we propose a new approach that can deliberate the reasoning steps with tool interfaces, namely \textbf{DELI}.
Learning to Imagine: Visually-Augmented Natural Language Generation
People often imagine relevant scenes to aid in the writing process.
Zero-shot Visual Question Answering with Language Model Feedback
In this paper, we propose a novel language model guided captioning approach, LAMOC, for knowledge-based visual question answering (VQA).
ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models
Although large language models (LLMs) have achieved excellent performance in a variety of evaluation benchmarks, they still struggle in complex reasoning tasks which require specific knowledge and multi-hop reasoning.
Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models
The recent success of large language models (LLMs) has shown great potential to develop more powerful conversational recommender systems (CRSs), which rely on natural language conversations to satisfy user needs.
HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
Large language models (LLMs), such as ChatGPT, are prone to generate hallucinations, i. e., content that conflicts with the source or cannot be verified by the factual knowledge.
When Search Meets Recommendation: Learning Disentangled Search Representation for Recommendation
In our paper, we propose a Search-Enhanced framework for the Sequential Recommendation (SESRec) that leverages users' search interests for recommendation, by disentangling similar and dissimilar representations within S&R behaviors.
TOME: A Two-stage Approach for Model-based Retrieval
Recently, model-based retrieval has emerged as a new paradigm in text retrieval that discards the index in the traditional retrieval model and instead memorizes the candidate corpora using model parameters.
The Web Can Be Your Oyster for Improving Large Language Models
In order to further improve the capacity of LLMs for knowledge-intensive tasks, we consider augmenting LLMs with the large-scale web using search engine.
Evaluating Object Hallucination in Large Vision-Language Models
Despite the promising progress on LVLMs, we find that LVLMs suffer from the hallucination problem, i. e. they tend to generate objects that are inconsistent with the target images in the descriptions.
StructGPT: A General Framework for Large Language Model to Reason over Structured Data
Specially, we propose an \emph{invoking-linearization-generation} procedure to support LLMs in reasoning on the structured data with the help of the external interfaces.
Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach
Inspired by the recent progress on large language models (LLMs), we take a different approach to developing the recommendation models, considering recommendation as instruction following by LLMs.
Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation
To solve it, we propose Diffusion-NAT, which introduces discrete diffusion models~(DDM) into NAR text-to-text generation and integrates BART to improve the performance.
GlyphDiffusion: Text Generation as Image Generation
In this way, conditional text generation can be cast as a glyph image generation task, and it is then natural to apply continuous diffusion models to discrete texts.
EulerNet: Adaptive Feature Interaction Learning via Euler's Formula for CTR Prediction
EulerNet converts the exponential powers of feature interactions into simple linear combinations of the modulus and phase of the complex features, making it possible to adaptively learn the high-order feature interactions in an efficient way.
WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpus
In this paper, we introduce a new NLP task -- generating short factual articles with references for queries by mining supporting evidence from the Web.
A Survey of Large Language Models
To discriminate the difference in parameter scale, the research community has coined the term large language models (LLM) for the PLMs of significant size.
Scaling Pre-trained Language Models to Deeper via Parameter-efficient Architecture
Based on such a decomposition, our architecture shares the central tensor across all layers for reducing the model size and meanwhile keeps layer-specific auxiliary tensors (also using adapters) for enhancing the adaptation flexibility.
Dually Enhanced Propensity Score Estimation in Sequential Recommendation
According to the graph, two complementary propensity scores are estimated from the views of item and user, respectively, based on the same set of user feedback data.
Diffusion Models for Non-autoregressive Text Generation: A Survey
In this survey, we review the recent progress in diffusion models for NAR text generation.
REASONER: An Explainable Recommendation Dataset with Multi-aspect Real User Labeled Ground Truths Towards more Measurable Explainable Recommendation
To alleviate the above problems, we propose to build an explainable recommendation dataset with multi-aspect real user labeled ground truths.
TextBox 2.0: A Text Generation Library with Pre-trained Language Models
To facilitate research on text generation, this paper presents a comprehensive and unified library, TextBox 2. 0, focusing on the use of pre-trained language models (PLMs).
Ranked #1 on
Abstractive Text Summarization
on CNN/Daily Mail
MASTER: Multi-task Pre-trained Bottlenecked Masked Autoencoders are Better Dense Retrievers
Pre-trained Transformers (\eg BERT) have been commonly used in existing dense retrieval methods for parameter initialization, and recent studies are exploring more effective pre-training tasks for further improving the quality of dense vectors.
Visually-augmented pretrained language models for NLP tasks without images
Although pre-trained language models~(PLMs) have shown impressive performance by text-only self-supervised training, they are found lack of visual semantics or commonsense.
UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph
Multi-hop Question Answering over Knowledge Graph~(KGQA) aims to find the answer entities that are multiple hops away from the topic entities mentioned in a natural language question on a large-scale Knowledge Graph (KG).
CDSM: Cascaded Deep Semantic Matching on Textual Graphs Leveraging Ad-hoc Neighbor Selection
In the first stage, a lightweight CNN-based ad-hod neighbor selector is deployed to filter useful neighbors for the matching task with a small computation cost.
Recent Advances in RecBole: Extensions with more Practical Considerations
In order to show the recent update in RecBole, we write this technical report to introduce our latest improvements on RecBole.
Dense Text Retrieval based on Pretrained Language Models: A Survey
With powerful PLMs, we can effectively learn the representations of queries and texts in the latent representation space, and further construct the semantic matching function between the dense vectors for relevance modeling.
Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation
Some recent knowledge distillation based methods transfer knowledge from complex teacher models to shallow student models for accelerating the online model inference.
ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation
However, NAR models usually generate texts of lower quality due to the absence of token dependency in the output text.
SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval
Thus, we propose a simple ambiguous negatives sampling method, SimANS, which incorporates a new sampling probability distribution to sample more ambiguous negatives.
Privacy-Preserved Neural Graph Similarity Learning
To develop effective and efficient graph similarity learning (GSL) models, a series of data-driven neural algorithms have been proposed in recent years.
Law Article-Enhanced Legal Case Matching: a Causal Learning Approach
We show that the framework is model-agnostic, and a number of legal case matching models can be applied as the underlying models.
Modeling Multiple Views via Implicitly Preserving Global Consistency and Local Complementarity
To this end, we propose a methodology, specifically consistency and complementarity network (CoCoNet), which avails of strict global inter-view consistency and local cross-view complementarity preserving regularization to comprehensively learn representations from multiple views.
A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language
Although artificial intelligence (AI) has made significant progress in understanding molecules in a wide range of fields, existing models generally acquire the single cognitive ability from the single molecular modality.
Ranked #7 on
Molecule Captioning
on ChEBI-20
Enhancing User Behavior Sequence Modeling by Generative Tasks for Session Search
To help the encoding of the current user behavior sequence, we propose to use a decoder and the information of future sequences and a supplemental query.
Ultron: An Ultimate Retriever on Corpus with a Model-based Indexer
In order to unify these two stages, we explore a model-based indexer for document retrieval.
Modeling Two-Way Selection Preference for Person-Job Fit
To model the two-way selection preference from the dual-perspective of job seekers and employers, we incorporate two different nodes for each candidate (or job) and characterize both successful matching and failed matching via a unified dual-perspective interaction graph.
Multimodal foundation models are better simulators of the human brain
Further, from the perspective of neural encoding (based on our foundation model), we find that both visual and lingual encoders trained multimodally are more brain-like compared with unimodal ones.
STAR-GNN: Spatial-Temporal Video Representation for Content-based Retrieval
It models a video with a lattice feature graph in which the nodes represent regions of different granularity, with weighted edges that represent the spatial and temporal links.
Explainable Legal Case Matching via Inverse Optimal Transport-based Rationale Extraction
As an essential operation of legal retrieval, legal case matching plays a central role in intelligent legal systems.
MVP: Multi-task Supervised Pre-training for Natural Language Generation
Motivated by the success of supervised pre-training, we propose Multi-task superVised Pre-training (MVP) for natural language generation.
Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning
Our approach unifies the recommendation and conversation subtasks into the prompt learning paradigm, and utilizes knowledge-enhanced prompts based on a fixed pre-trained language model (PLM) to fulfill both subtasks in a unified approach.
Ranked #1 on
Text Generation
on ReDial
RecBole 2.0: Towards a More Up-to-Date Recommendation Library
In order to support the study of recent advances in recommender systems, this paper presents an extended recommendation library consisting of eight packages for up-to-date topics and architectures.
JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding
Considering the complex nature of mathematical texts, we design a novel curriculum pre-training approach for improving the learning of mathematical PLMs, consisting of both basic and advanced courses.
Towards Universal Sequence Representation Learning for Recommender Systems
In order to develop effective sequential recommenders, a series of sequence representation learning (SRL) methods are proposed to model historical user behaviors.
Feature-aware Diversified Re-ranking with Disentangled Representations for Relevant Recommendation
Relevant recommendation is a special recommendation scenario which provides relevant items when users express interests on one target item (e. g., click, like and purchase).
Negative Sampling for Contrastive Representation Learning: A Review
The learn-to-compare paradigm of contrastive representation learning (CRL), which compares positive samples with negative ones for representation learning, has achieved great success in a wide range of domains, including natural language processing, computer vision, information retrieval and graph learning.
Great Truths are Always Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models
Commonsense reasoning in natural language is a desired ability of artificial intelligent systems.
Learning to Transfer Prompts for Text Generation
First, PTG learns a set of source prompts for various source generation tasks and then transfers these prompts as target prompts to perform target generation tasks.
ElitePLM: An Empirical Study on General Language Ability Evaluation of Pretrained Language Models
In this paper, we present a large-scale empirical study on general language ability evaluation of PLMs (ElitePLM).
Debiased Contrastive Learning of Unsupervised Sentence Representations
In DCLR, we design an instance weighting method to punish false negatives and generate noise-based negatives to guarantee the uniformity of the representation space.
