Search Results for author:
Found 71 papers, 35 papers with code
Reciprocal Learning of Knowledge Retriever and Response Ranker for Knowledge-Grounded Conversations
In this paper, we propose a reciprocal learning approach to jointly optimize a knowledge retriever and a response ranker for knowledge-grounded response retrieval without ground-truth knowledge labels.
Social Norms-Grounded Machine Ethics in Complex Narrative Situation
Besides a norm-grounding knowledge model, we present a novel norm-supported ethical judgment model in line with neural module networks to alleviate dilemma situations and improve norm-level explainability.
Dual-Personalizing Adapter for Federated Foundation Models
To address challenges in this new setting, we explore a simple yet effective solution to learn a comprehensive foundation model.
Corpus-Steered Query Expansion with Large Language Models
Recent studies demonstrate that query expansions generated by large language models (LLMs) can considerably enhance information retrieval systems by generating hypothetical documents that answer the queries as expansions.
Meta-Task Prompting Elicits Embedding from Large Language Models
In this work, we introduce a new unsupervised embedding method, Meta-Task Prompting with Explicit One-Word Limitation (MetaEOL), for generating high-quality sentence embeddings from Large Language Models (LLMs) without the need for model fine-tuning or task-specific engineering.
A Survey on Knowledge Distillation of Large Language Models
In the era of Large Language Models (LLMs), Knowledge Distillation (KD) emerges as a pivotal methodology for transferring advanced capabilities from leading proprietary LLMs, such as GPT-4, to their open-source counterparts like LLaMA and Mistral.
Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models
Catastrophic forgetting emerges as a critical challenge when fine-tuning multi-modal large language models (MLLMs), where improving performance on unseen tasks often leads to a significant performance drop on the original tasks.
Rethinking Human-like Translation Strategy: Integrating Drift-Diffusion Model with Large Language Models for Machine Translation
Large language models (LLMs) have demonstrated promising potential in various downstream tasks, including machine translation.
Training-time Neuron Alignment through Permutation Subspace for Improving Linear Mode Connectivity and Model Fusion
In deep learning, stochastic gradient descent often yields functionally similar yet widely scattered solutions in the weight space even under the same initialization, causing barriers in the Linear Mode Connectivity (LMC) landscape.
Leveraging Large Language Models for NLG Evaluation: A Survey
In the rapidly evolving domain of Natural Language Generation (NLG) evaluation, introducing Large Language Models (LLMs) has opened new avenues for assessing generated content quality, e. g., coherence, creativity, and context relevance.
CrossBind: Collaborative Cross-Modal Identification of Protein Nucleic-Acid-Binding Residues
Accurate identification of protein nucleic-acid-binding residues poses a significant challenge with important implications for various biological processes and drug design.
Good Questions Help Zero-Shot Image Reasoning
QVix enables a wider exploration of visual scenes, improving the LVLMs' reasoning accuracy and depth in tasks such as visual question answering and visual entailment.
Thread of Thought Unraveling Chaotic Contexts
Large Language Models (LLMs) have ushered in a transformative era in the field of natural language processing, excelling in tasks related to text comprehension and generation.
Enhancing Secrecy Capacity in PLS Communication with NORAN based on Pilot Information Codebooks
At the same time, selecting a NORAN design with ideal security performance from a large number of design options is also a challenging problem.
Re-Reading Improves Reasoning in Large Language Models
To enhance the reasoning capabilities of off-the-shelf Large Language Models (LLMs), we introduce a simple, yet general and effective prompting method, Re2, i. e., \textbf{Re}-\textbf{Re}ading the question as input.
Accurate Prediction of Antibody Function and Structure Using Bio-Inspired Antibody Language Model
Remarkably, BALMFold outperforms those well-established methods like AlphaFold2, IgFold, ESMFold, and OmegaFold in the antibody benchmark, demonstrating significant potential to advance innovative engineering and streamline therapeutic antibody development by reducing the need for unnecessary trials.
Enhancing the Protein Tertiary Structure Prediction by Multiple Sequence Alignment Generation
The field of protein folding research has been greatly advanced by deep learning methods, with AlphaFold2 (AF2) demonstrating exceptional performance and atomic-level precision.
Spatial-temporal Prompt Learning for Federated Weather Forecasting
Federated weather forecasting is a promising collaborative learning framework for analyzing meteorological data across participants from different countries and regions, thus embodying a global-scale real-time weather data predictive analytics platform to tackle climate change.
AF2-Mutation: Adversarial Sequence Mutations against AlphaFold2 on Protein Tertiary Structure Prediction
Deep learning-based approaches, such as AlphaFold2 (AF2), have significantly advanced protein tertiary structure prediction, achieving results comparable to real biological experimental methods.
Synergistic Interplay between Search and Large Language Models for Information Retrieval
Information retrieval (IR) plays a crucial role in locating relevant resources from vast amounts of data, and its applications have evolved from traditional knowledge bases to modern retrieval models (RMs).
Large Language Models are Strong Zero-Shot Retriever
In this work, we propose a simple method that applies a large language model (LLM) to large-scale retrieval in zero-shot scenarios.
LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Retrieval
The conventional dense retrieval paradigm relies on encoding images and texts into dense representations using dual-stream encoders, however, it faces challenges with low retrieval speed in large-scale retrieval scenarios.
Prompt Federated Learning for Weather Forecasting: Toward Foundation Models on Meteorological Data
To relieve the data exposure concern across regions, a novel federated learning approach has been proposed to collaboratively learn a brand-new spatio-temporal Transformer-based foundation model across participants with heterogeneous meteorological data.
Iterative Proposal Refinement for Weakly-Supervised Video Grounding
Weakly-Supervised Video Grounding (WSVG) aims to localize events of interest in untrimmed videos with only video-level annotations.
LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Sparse Retrieval
To address this issue, we propose a novel sparse retrieval paradigm for ITR that exploits sparse representations in the vocabulary space for images and texts.
Adam: Dense Retrieval Distillation with Adaptive Dark Examples
Different from previous works that only rely on one positive and hard negatives as candidate passages, we create dark examples that all have moderate relevance to the query through mixing-up and masking in discrete space.
Fine-Grained Distillation for Long Document Retrieval
Long document retrieval aims to fetch query-relevant documents from a large-scale collection, where knowledge distillation has become de facto to improve a retriever by mimicking a heterogeneous yet powerful cross-encoder.
CCPrefix: Counterfactual Contrastive Prefix-Tuning for Many-Class Classification
Basically, an instance-dependent soft prefix, derived from fact-counterfactual pairs in the label space, is leveraged to complement the language verbalizers in many-class classification.
DeepNoise: Signal and Noise Disentanglement based on Classifying Fluorescent Microscopy Images via Deep Learning
The high-content image-based assay is commonly leveraged for identifying the phenotypic impact of genetic perturbations in biology field.
DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalization
DUET is deployed on a powerful cloud server that only requires the low cost of forwarding propagation and low time delay of data transmission between the device and the cloud.
LexMAE: Lexicon-Bottlenecked Pretraining for Large-Scale Retrieval
In large-scale retrieval, the lexicon-weighting paradigm, learning weighted sparse representations in vocabulary space, has shown promising results with high quality and low latency.
LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval
The alignment is achieved by weakened knowledge distillations to enlighten the retriever via two aspects -- 1) a lexicon-augmented contrastive objective to challenge the dense encoder and 2) a pair-wise rank-consistent regularization to make dense model's behavior incline to the other.
E2Efold-3D: End-to-End Deep Learning Method for accurate de novo RNA 3D Structure Prediction
When E2Efold-3D is coupled with the experimental techniques, the RNA structure prediction field can be greatly advanced.
KnowDA: All-in-One Knowledge Mixture Model for Data Augmentation in Low-Resource NLP
This paper focuses on the data augmentation for low-resource NLP tasks where the training set is limited.
Towards Robust Ranker for Text Retrieval
A ranker plays an indispensable role in the de facto 'retrieval & rerank' pipeline, but its training still lags behind -- learning from moderate negatives or/and serving as an auxiliary module for a retriever.
UnifieR: A Unified Retriever for Large-Scale Retrieval
Large-scale retrieval is to recall relevant documents from a huge collection given a query.
Interpretable RNA Foundation Model from Unannotated Data for Highly Accurate RNA Structure and Function Predictions
Non-coding RNA structure and function are essential to understanding various biological processes, such as cell signaling, gene expression, and post-transcriptional regulations.
ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification
Generating new events given context with correlated ones plays a crucial role in many event-centric reasoning tasks.
PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddings
A straightforward solution is resorting to more diverse positives from a multi-augmenting strategy, while an open question remains about how to unsupervisedly learn from the diverse positives but with uneven augmenting qualities in the text field.
FedDTG:Federated Data-Free Knowledge Distillation via Three-Player Generative Adversarial Networks
This technique mitigates the user heterogeneity problem and better protects user privacy.
Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI
However, edge computing, especially edge and cloud collaborative computing, are still in its infancy to announce their success due to the resource-constrained IoT scenarios with very limited algorithms deployed.
EventBERT: A Pre-Trained Model for Event Correlation Reasoning
Event correlation reasoning infers whether a natural language paragraph containing multiple events conforms to human common sense.
Hierarchical Relation-Guided Type-Sentence Alignment for Long-Tail Relation Extraction with Distant Supervision
It consists of (1) a pairwise type-enriched sentence encoding module injecting both context-free and -related backgrounds to alleviate sentence-level wrong labeling, and (2) a hierarchical type-sentence alignment module enriching a sentence with the triple fact's basic attributes to support long-tail relations.
Sequential Diagnosis Prediction with Transformer and Ontological Representation
Sequential diagnosis prediction on the Electronic Health Record (EHR) has been proven crucial for predictive analytics in the medical domain.
Eliminating Sentiment Bias for Aspect-Level Sentiment Classification with Unsupervised Opinion Extraction
Aspect-level sentiment classification (ALSC) aims at identifying the sentiment polarity of a specified aspect in a sentence.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+3
Federated Learning for Privacy-Preserving Open Innovation Future on Digital Health
Implementing an open innovation framework in the healthcare industry, namely open health, is to enhance innovation and creative capability of health-related organisations by building a next-generation collaborative framework with partner organisations and the research community.
Financing Entrepreneurship and Innovation in China
This study reports on the current state-of-affairs in the funding of entrepreneurship and innovations in China and provides a broad survey of academic findings on the subject.
Multi-Center Federated Learning: Clients Clustering for Better Personalization
By comparison, a mixture of multiple global models could capture the heterogeneity across various clients if assigning the client to different global models (i. e., centers) in FL.
Dynamic Prediction Model for NOx Emission of SCR System Based on Hybrid Data-driven Algorithms
Aiming at the problem that delay time is difficult to determine and prediction accuracy is low in building prediction model of SCR system, a dynamic modeling scheme based on a hybrid of multiple data-driven algorithms was proposed.
Improving Zero-Shot Cross-lingual Transfer for Multilingual Question Answering over Knowledge Graph
That is, we can only access training data in a high-resource language, while need to answer multilingual questions without any labeled data in target languages.
Federated Graph Learning -- A Position Paper
Federated learning (FL) is a an emerging technique that can collaboratively train a shared model while keeping the data decentralized, which is a rational solution for distributed GNN training.
EBM-Fold: Fully-Differentiable Protein Folding Powered by Energy-based Models
In this paper, we propose a fully-differentiable approach for protein structure optimization, guided by a data-driven generative network.
tFold-TR: Combining Deep Learning Enhanced Hybrid Potential Energy for Template-Based Modeling Structure Refinement
Protein structure prediction has been a grand challenge for over 50 years, owing to its broad scientific and application interests.
Federated Unsupervised Representation Learning
FURL poses two new challenges: (1) data distribution shift (Non-IID distribution) among clients would make local models focus on different categories, leading to the inconsistency of representation spaces.
RatE: Relation-Adaptive Translating Embedding for Knowledge Graph Completion
Many graph embedding approaches have been proposed for knowledge graph completion via link prediction.
Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention
Then, facilitated by the proposed base model, we introduce collaborating relation features shared among relations in the hierarchies to promote the relation-augmenting process and balance the training data for long-tail relations.
BiteNet: Bidirectional Temporal Encoder Network to Predict Medical Outcomes
Electronic health records (EHRs) are longitudinal records of a patient's interactions with healthcare systems.
Federated Mutual Learning
The experiments show that FML can achieve better performance than alternatives in typical FL setting, and clients can be benefited from FML with different models and tasks.
Self-Attention Enhanced Patient Journey Understanding in Healthcare System
The key challenge of patient journey understanding is to design an effective encoding mechanism which can properly tackle the aforementioned multi-level structured patient journey data with temporal sequential visits and a set of medical codes.
Multi-Center Federated Learning: Clients Clustering for Better Personalization
However, due to the diverse nature of user behaviors, assigning users' gradients to different global models (i. e., centers) can better capture the heterogeneity of data distributions across users.
Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion
In experiments, we achieve state-of-the-art performance on three benchmarks and a zero-shot dataset for link prediction, with highlights of inference costs reduced by 1-2 orders of magnitude compared to a textual encoding method.
Ranked #4 on
Link Prediction
on UMLS
Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning
In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training, to inject language models with structured knowledge via learning from raw text.
Self-Attention Enhanced Selective Gate with Entity-Aware Embedding for Distantly Supervised Relation Extraction
Distantly supervised relation extraction intrinsically suffers from noisy labels due to the strong assumption of distant supervision.
Multi-Task Learning for Conversational Question Answering over a Large-Scale Knowledge Base
We consider the problem of conversational question answering over a large-scale knowledge base.
Temporal Self-Attention Network for Medical Concept Embedding
In this paper, we propose a medical concept embedding method based on applying a self-attention mechanism to represent each medical concept.
Effective Search of Logical Forms for Weakly Supervised Knowledge-Based Question Answering
These two problems lead to a poorly-trained semantic parsing model.
Tensorized Self-Attention: Efficiently Modeling Pairwise and Global Dependencies Together
Neural networks equipped with self-attention have parallelizable computation, light-weight structure, and the ability to capture both long-range and local dependencies.
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
In this paper, we propose a model, called "bi-directional block self-attention network (Bi-BloSAN)", for RNN/CNN-free sequence encoding.
Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
In this paper, we integrate both soft and hard attention into one context fusion model, "reinforced self-attention (ReSA)", for the mutual benefit of each other.
Ranked #56 on
Natural Language Inference
on SNLI
DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
Recurrent neural nets (RNN) and convolutional neural nets (CNN) are widely used on NLP tasks to capture the long-term and local dependencies, respectively.
Ranked #68 on
Natural Language Inference
on SNLI
