Search Results for author:
Found 109 papers, 42 papers with code
Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-Supervision
Besides, the robustness of the previous neural methods is limited by the large-scale annotated data.
Turn-Level User Satisfaction Estimation in E-commerce Customer Service
To this end, we formalize the turn-level satisfaction estimation as a reinforcement learning problem, in which the model can be optimized with only session-level satisfaction labels.
Incorporating Casual Analysis into Diversified and Logical Response Generation
Although the Conditional Variational Auto-Encoder (CVAE) model can generate more diversified responses than the traditional Seq2Seq model, the responses often have low relevance with the input words or are illogical with the question.
Continual Few-shot Intent Detection
Intent detection is at the core of task-oriented dialogue systems.
ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy
In WebShop, the 1-shot performance of the A$^3$T agent matches human average, and 4 rounds of iterative refinement lead to the performance approaching human experts.
RoleInteract: Evaluating the Social Interaction of Role-Playing Agents
In this paper, we introduce RoleInteract, the first benchmark designed to systematically evaluate the sociality of role-playing conversational agents at both individual and group levels of social interactions.
mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding
In this work, we emphasize the importance of structure information in Visual Document Understanding and propose the Unified Structure Learning to boost the performance of MLLMs.
From Skepticism to Acceptance: Simulating the Attitude Dynamics Toward Fake News
Our simulation results uncover patterns in fake news propagation related to topic relevance, and individual traits, aligning with real-world observations.
OccFusion: Depth Estimation Free Multi-sensor Fusion for 3D Occupancy Prediction
We introduce OccFusion, a multi-modal fusion method free from depth estimation, and a corresponding point cloud sampling algorithm for dense integration of image features.
Improving Cross-lingual Representation for Semantic Retrieval with Code-switching
Semantic Retrieval (SR) has become an indispensable part of the FAQ system in the task-oriented question-answering (QA) dialogue scenario.
Semantics-enhanced Cross-modal Masked Image Modeling for Vision-Language Pre-training
In vision-language pre-training (VLP), masked image modeling (MIM) has recently been introduced for fine-grained cross-modal alignment.
Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval
In this work, we propose the UNIFY framework, which learns lexicon representations to capture fine-grained semantics and combines the strengths of latent and lexicon representations for video-text retrieval.
Budget-Constrained Tool Learning with Planning
Despite intensive efforts devoted to tool learning, the problem of budget-constrained tool learning, which focuses on resolving user queries within a specific budget constraint, has been widely overlooked.
Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models
Large Vision Language Models exhibit remarkable capabilities but struggle with hallucinations inconsistencies between images and their descriptions.
PANDA: Preference Adaptation for Enhancing Domain-Specific Abilities of LLMs
While Large language models (LLMs) have demonstrated considerable capabilities across various natural language tasks, they often fall short of the performance achieved by domain-specific state-of-the-art models.
Model Composition for Multimodal Large Language Models
In this paper, we propose a new paradigm through the model composition of existing MLLMs to create a new model that retains the modal understanding capabilities of each original model.
Meta Ranking: Less Capable Language Models are Capable for Single Response Judgement
Although Large Language Models (LLMs) have demonstrated strong performance on a wide range of tasks, they still face reliability challenges such as hallucination.
Browse and Concentrate: Comprehending Multimodal Content via prior-LLM Context Fusion
With the bloom of Large Language Models (LLMs), Multimodal Large Language Models (MLLMs) that incorporate LLMs with pre-trained vision models have recently demonstrated impressive performance across diverse vision-language tasks.
Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception
To assess the performance of Mobile-Agent, we introduced Mobile-Eval, a benchmark for evaluating mobile device operations.
Small LLMs Are Weak Tool Learners: A Multi-LLM Agent
Each component is implemented by a single LLM that focuses on a specific capability and collaborates with others to accomplish the task.
Knowledge Distillation for Closed-Source Language Models
The prior estimation aims to derive a prior distribution by utilizing the corpus generated by closed-source language models, while the posterior estimation employs a proxy model to update the prior distribution and derive a posterior distribution.
TiMix: Text-aware Image Mixing for Effective Vision-Language Pre-training
Self-supervised Multi-modal Contrastive Learning (SMCL) remarkably advances modern Vision-Language Pre-training (VLP) models by aligning visual and linguistic modalities.
Hallucination Augmented Contrastive Learning for Multimodal Large Language Model
We first analyzed the representation distribution of textual and visual tokens in MLLM, revealing two important findings: 1) there is a significant gap between textual and visual representations, indicating unsatisfactory cross-modal representation alignment; 2) representations of texts that contain and do not contain hallucinations are entangled, making it challenging to distinguish them.
Ranked #72 on
Visual Question Answering
on MM-Vet
mPLUG-PaperOwl: Scientific Diagram Analysis with the Multimodal Large Language Model
In this work, towards a more versatile copilot for academic paper writing, we mainly focus on strengthening the multi-modal diagram analysis ability of Multimodal LLMs.
Class Gradient Projection For Continual Learning
Catastrophic forgetting is one of the most critical challenges in Continual Learning (CL).
AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation
Despite making significant progress in multi-modal tasks, current Multi-modal Large Language Models (MLLMs) encounter the significant challenge of hallucinations, which may lead to harmful consequences.
mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration
Multi-modal Large Language Models (MLLMs) have demonstrated impressive instruction abilities across various open-ended tasks.
Ranked #11 on
Visual Question Answering (VQA)
on InfiMM-Eval
CycleAlign: Iterative Distillation from Black-box LLM to White-box Models for Better Human Alignment
With in-context learning (ICL) as the core of the cycle, the black-box models are able to rank the model-generated responses guided by human-craft instruction and demonstrations about their preferences.
MCC-KD: Multi-CoT Consistent Knowledge Distillation
Large language models (LLMs) have showcased remarkable capabilities in complex reasoning through chain of thought (CoT) prompting.
Improving Seq2Seq Grammatical Error Correction via Decoding Interventions
In this paper, we propose a unified decoding intervention framework that employs an external critic to assess the appropriateness of the token to be generated incrementally, and then dynamically influence the choice of the next token.
Ranked #1 on
Grammatical Error Correction
on MuCGEC
UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model
Text is ubiquitous in our visual world, conveying crucial information, such as in documents, websites, and everyday photographs.
DePT: Decoupled Prompt Tuning
Specifically, through an in-depth analysis of the learned features of the base and new tasks, we observe that the BNT stems from a channel bias issue, i. e., the vast majority of feature channels are occupied by base-specific knowledge, resulting in the collapse of taskshared knowledge important to new tasks.
ModelScope-Agent: Building Your Customizable Agent System with Open-source Large Language Models
Large language models (LLMs) have recently demonstrated remarkable capabilities to comprehend human intentions, engage in reasoning, and design planning-like behavior.
Evaluation and Analysis of Hallucination in Large Vision-Language Models
In this paper, we propose Hallucination Evaluation based on Large Language Models (HaELM), an LLM-based hallucination evaluation framework.
From Global to Local: Multi-scale Out-of-distribution Detection
Out-of-distribution (OOD) detection aims to detect "unknown" data whose labels have not been seen during the in-distribution (ID) training process.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
+1
Improving Anomaly Segmentation with Multi-Granularity Cross-Domain Alignment
Anomaly segmentation plays a pivotal role in identifying atypical objects in images, crucial for hazard detection in autonomous driving systems.
CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility
In this paper, we present CValues, the first Chinese human values evaluation benchmark to measure the alignment ability of LLMs in terms of both safety and responsibility criteria.
mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding
Nevertheless, without in-domain training, these models tend to ignore fine-grained OCR features, such as sophisticated tables or large blocks of text, which are essential for OCR-free document understanding.
DialoGPS: Dialogue Path Sampling in Continuous Semantic Space for Data Augmentation in Multi-Turn Conversations
In open-domain dialogue generation tasks, contexts and responses in most datasets are one-to-one mapped, violating an important many-to-many characteristic: a context leads to various responses, and a response answers multiple contexts.
Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Dataset for Pre-training and Benchmarks
In addition, to facilitate a comprehensive evaluation of video-language models, we carefully build the largest human-annotated Chinese benchmarks covering three popular video-language tasks of cross-modal retrieval, video captioning, and video category classification.
Distinguish Before Answer: Generating Contrastive Explanation as Knowledge for Commonsense Question Answering
Existing knowledge-enhanced methods have achieved remarkable results in certain QA tasks via obtaining diverse knowledge from different knowledge bases.
AMTSS: An Adaptive Multi-Teacher Single-Student Knowledge Distillation Framework For Multilingual Language Inference
To support cost-effective language inference in multilingual settings, we propose AMTSS, an adaptive multi-teacher single-student distillation framework, which allows distilling knowledge from multiple teachers to a single student.
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality
Our code, pre-trained model, instruction-tuned models, and evaluation set are available at https://github. com/X-PLUG/mPLUG-Owl.
Ranked #3 on
Visual Question Answering (VQA)
on HallusionBench
Visual Question Answering (VQA)
Zero-Shot Video Question Answer
ContrastMotion: Self-supervised Scene Motion Learning for Large-Scale LiDAR Point Clouds
In this paper, we propose a novel self-supervised motion estimator for LiDAR-based autonomous driving via BEV representation.
ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human
In this paper, we present ChatPLUG, a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format.
DETA: Denoised Task Adaptation for Few-Shot Learning
Test-time task adaptation in few-shot learning aims to adapt a pre-trained task-agnostic model for capturing taskspecific knowledge of the test task, rely only on few-labeled support samples.
Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariance
Category-level articulated object pose estimation aims to estimate a hierarchy of articulation-aware object poses of an unseen articulated object from a known category.
Active Velocity Estimation using Light Curtains via Self-Supervised Multi-Armed Bandits
To navigate in an environment safely and autonomously, robots must accurately estimate where obstacles are and how they move.
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
In contrast to predominant paradigms of solely relying on sequence-to-sequence generation or encoder-based instance discrimination, mPLUG-2 introduces a multi-module composition network by sharing common universal modules for modality collaboration and disentangling different modality modules to deal with modality entanglement.
Ranked #1 on
Video Captioning
on MSR-VTT
A Closer Look at Few-shot Classification Again
Few-shot classification consists of a training phase where a model is learned on a relatively large dataset and an adaptation phase where the learned model is adapted to previously-unseen tasks with limited labeled samples.
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
We achieve state-of-the-art results on 15 well-established video-language understanding and generation tasks, especially on temporal-oriented datasets (e. g., SSv2-Template and SSv2-Label) with 8. 6% and 11. 1% improvement respectively.
Ranked #1 on
Visual Question Answering (VQA)
on TGIF-QA
Intelligent Computing: The Latest Advances, Challenges and Future
In recent years, we have witnessed the emergence of intelligent computing, a new computing paradigm that is reshaping traditional computing and promoting digital revolution in the era of big data, artificial intelligence and internet-of-things with new computing theories, architectures, methods, systems, and applications.
Zero-shot Image Captioning by Anchor-augmented Vision-Language Space Alignment
We further propose Anchor Augment to guide the generative model's attention to the fine-grained information in the representation of CLIP.
MUI-TARE: Multi-Agent Cooperative Exploration with Unknown Initial Position
To intelligently balance the robustness of sub-map merging and exploration efficiency, we develop a new approach for lidar-based multi-agent exploration, which can direct one agent to repeat another agent's trajectory in an \emph{adaptive} manner based on the quality indicator of the sub-map merging process.
Incorporating Causal Analysis into Diversified and Logical Response Generation
Although the Conditional Variational AutoEncoder (CVAE) model can generate more diversified responses than the traditional Seq2Seq model, the responses often have low relevance with the input words or are illogical with the question.
Generating Persuasive Responses to Customer Reviews with Multi-Source Prior Knowledge in E-commerce
A multi-aspect attentive network is proposed to automatically attend to different aspects in a review and ensure most of the issues are tackled.
iSimLoc: Visual Global Localization for Previously Unseen Environments with Simulated Images
The visual camera is an attractive device in beyond visual line of sight (B-VLOS) drone operation, since they are low in size, weight, power, and cost, and can provide redundant modality to GPS failures.
Class-Level Logit Perturbation
However, limited studies have explicitly explored for the perturbation of logit vectors.
DictBERT: Dictionary Description Knowledge Enhanced Language Model Pre-training via Contrastive Learning
We evaluate our approach on a variety of knowledge driven and language understanding tasks, including NER, relation extraction, CommonsenseQA, OpenBookQA and GLUE.
RCA: Ride Comfort-Aware Visual Navigation via Self-Supervised Learning
Under shared autonomy, wheelchair users expect vehicles to provide safe and comfortable rides while following users high-level navigation plans.
Scene Recognition with Objectness, Attribute and Category Learning
Based on the complementarity of attribute and category labels, we propose a Multi-task Attribute-Scene Recognition (MASR) network which learns a category embedding and at the same time predicts scene attributes.
ALTO: A Large-Scale Dataset for UAV Visual Place Recognition and Localization
We present the ALTO dataset, a vision-focused dataset for the development and benchmarking of Visual Place Recognition and Localization methods for Unmanned Aerial Vehicles.
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval
However, cross-grained contrast, which is the contrast between coarse-grained representations and fine-grained representations, has rarely been explored in prior research.
Ranked #12 on
Video Retrieval
on MSVD
AutoMerge: A Framework for Map Assembling and Smoothing in City-scale Environments
We present AutoMerge, a LiDAR data processing framework for assembling a large number of map segments into a complete map.
SHREC'22 Track: Sketch-Based 3D Shape Retrieval in the Wild
We define two SBSR tasks and construct two benchmarks consisting of more than 46, 000 CAD models, 1, 700 realistic models, and 145, 000 sketches in total.
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections
Large-scale pretrained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks.
Ranked #1 on
Image Captioning
on COCO Captions
MGIMN: Multi-Grained Interactive Matching Network for Few-shot Text Classification
They also ignore the importance to capture the inter-dependency between query and the support set for few-shot text classification.
Auto-MLM: Improved Contrastive Learning for Self-supervised Multi-lingual Knowledge Retrieval
Unexpectedly, MLM ignores the sentence-level training, and CL also neglects extraction of the internal info from the query.
Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Grounding
Moreover, since the backbones are query-agnostic, it is difficult to completely avoid the inconsistency issue by training the visual backbone end-to-end in the visual grounding framework.
LQoCo: Learning to Optimize Cache Capacity Overloading in Storage Systems
Cache plays an important role to maintain high and stable performance (i. e. high throughput, low tail latency and throughput jitter) in storage systems.
Deep Multi-Branch Aggregation Network for Real-Time Semantic Segmentation in Street Scenes
This shows that DMA-Net provides a good tradeoff between segmentation quality and speed for semantic segmentation in street scenes.
Achieving Human Parity on Visual Question Answering
The Visual Question Answering (VQA) task utilizes both visual image and language analysis to answer a textual question with respect to an image.
Ranked #7 on
Visual Question Answering (VQA)
on VQA v2 test-dev
A Generic Knowledge Based Medical Diagnosis Expert System
The proposed disease diagnosis system also uses a graphical user interface (GUI) to facilitate users to interact with the expert system.
K-AID: Enhancing Pre-trained Language Models with Domain Knowledge for Question Answering
Knowledge enhanced pre-trained language models (K-PLMs) are shown to be effective for many public tasks in the literature but few of them have been successfully applied in practice.
AliMe MKG: A Multi-modal Knowledge Graph for Live-streaming E-commerce
Live streaming is becoming an increasingly popular trend of sales in E-commerce.
GGP: A Graph-based Grouping Planner for Explicit Control of Long Text Generation
With these two synergic representations, we then regroup these phrases into a fine-grained plan, based on which we generate the final long text.
SPMoE: Generate Multiple Pattern-Aware Outputs with Sparse Pattern Mixture of Experts
Each one-to-one mapping is associated with a conditional generation pattern and is modeled with an expert in SPMoE.
ROSITA: Enhancing Vision-and-Language Semantic Alignments via Cross- and Intra-modal Knowledge Integration
Nevertheless, most existing VLP approaches have not fully utilized the intrinsic knowledge within the image-text pairs, which limits the effectiveness of the learned alignments and further restricts the performance of their models.
KACE: Generating Knowledge Aware Contrastive Explanations for Natural Language Inference
In order to better understand the reason behind model behaviors (i. e., making predictions), most recent works have exploited generative models to provide complementary explanations.
Cognitive Visual Commonsense Reasoning Using Dynamic Working Memory
Moreover, the proposed model provides intuitive interpretation into visual commonsense reasoning.
Accurate Few-Shot Object Detection With Support-Query Mutual Guidance and Hybrid Loss
Most object detection methods require huge amounts of annotated data and can detect only the categories that appear in the training set.
i3dLoc: Image-to-range Cross-domain Localization Robust to Inconsistent Environmental Conditions
Based on such features, we further design a spherical convolution network to learn viewpoint-invariant symmetric place descriptors.
State-Promoted Investment for Industrial Reforms: an Information Design Approach
We analyze the optimal strategy for a government to promote large-scale investment projects under information frictions.
AdaVQA: Overcoming Language Priors with Adapted Margin Cosine Loss
Experimental results demonstrate that our adapted margin cosine loss can greatly enhance the baseline models with an absolute performance gain of 15\% on average, strongly verifying the potential of tackling the language prior problem in VQA from the angle of the answer feature space learning.
LSTM Based Sentiment Analysis for Cryptocurrency Prediction
Recent studies in big data analytics and natural language processing develop automatic techniques in analyzing sentiment in the social media information.
OneStop QAMaker: Extract Question-Answer Pairs from Text in a One-Stop Approach
This pipeline approach, however, is undesired in mining the most appropriate QA pairs from documents since it ignores the connection between question generation and answer extraction, which may lead to incompatible QA pair generation, i. e., the selected answer span is inappropriate for question generation.
Using Machine Learning to Automate Mammogram Images Analysis
Early detection of breast cancer in X-ray mammography is believed to have effectively reduced the mortality rate.
A Data-driven Human Responsibility Management System
An ideal safe workplace is described as a place where staffs fulfill responsibilities in a well-organized order, potential hazardous events are being monitored in real-time, as well as the number of accidents and relevant damages are minimized.
Distant Supervision for E-commerce Query Segmentation via Attention Network
The booming online e-commerce platforms demand highly accurate approaches to segment queries that carry the product requirements of consumers.
AI Marker-based Large-scale AI Literature Mining
Inspired by the idea of molecular markers tracing in the field of biochemistry, three named entities, namely, methods, datasets and metrics are used as AI markers for AI literature.
Method and Dataset Entity Mining in Scientific Literature: A CNN + Bi-LSTM Model with Self-attention
We finally apply our model on PAKDD papers published from 2009-2019 to mine insightful results from scientific papers published in a longer time span.
AliMe KG: Domain Knowledge Graph Construction and Application in E-commerce
Pre-sales customer service is of importance to E-commerce platforms as it contributes to optimizing customers' buying process.
Character Matters: Video Story Understanding with Character-Aware Relations
Without identifying the connection between appearing people and character names, a model is not able to obtain a genuine understanding of the plots.
Monocular Camera Localization in Prior LiDAR Maps with 2D-3D Line Correspondences
With the pose prediction from VIO, we can efficiently obtain coarse 2D-3D line correspondences.
Target-Guided Structured Attention Network for Target-Dependent Sentiment Analysis
Second, it shows a strong advantage in determining the sentiment of a target when the context sentence contains multiple semantic segments.
Method and Dataset Mining in Scientific Papers
In the field of machine learning, the involved methods (M) and datasets (D) are key information in papers.
Following Social Groups: Socially Compliant Autonomous Navigation in Dense Crowds
The underlying system incorporates a deep neural network to track social groups and join the flow of a social group in facilitating the navigation.
Keep it Consistent: Topic-Aware Storytelling from an Image Stream via Iterative Multi-agent Communication
Visual storytelling aims to generate a narrative paragraph from a sequence of images automatically.
Ranked #9 on
Visual Storytelling
on VIST
2nd Place Solution to the GQA Challenge 2019
We present a simple method that achieves unexpectedly superior performance for Complex Reasoning involved Visual Question Answering.
Graphical Contrastive Losses for Scene Graph Parsing
The first, Entity Instance Confusion, occurs when the model confuses multiple instances of the same type of entity (e. g. multiple cups).
A Deep Cascade Model for Multi-Document Reading Comprehension
To address this problem, we develop a novel deep cascade learning model, which progressively evolves from the document-level and paragraph-level ranking of candidate texts to more precise answer extraction with machine reading comprehension.
Ranked #2 on
Question Answering
on MS MARCO
An Interpretable Model for Scene Graph Generation
We propose an efficient and interpretable scene graph generator.
Introduction to the 1st Place Winning Model of OpenImages Relationship Detection Challenge
This article describes the model we built that achieved 1st place in the OpenImage Visual Relationship Detection Challenge on Kaggle.
Improving Multilingual Semantic Textual Similarity with Shared Sentence Encoder for Low-resource Languages
Our approach is extended from a basic monolingual STS framework to a shared multilingual encoder pretrained with translation task to incorporate rich-resource language data.
Semi-Autoregressive Neural Machine Translation
Existing approaches to neural machine translation are typically autoregressive models.
Exploiting Effective Representations for Chinese Sentiment Analysis Using a Multi-Channel Convolutional Neural Network
Effective representation of a text is critical for various natural language processing tasks.
Large-Scale Visual Relationship Understanding
Large scale visual understanding is challenging, as it requires a model to handle the widely-spread and imbalanced distribution of <subject, relation, object> triples.
Relationship Proposal Networks
We demonstrate the ability of our Rel-PN to localize relationships with only a few thousand proposals.
LOAM: Lidar Odometry and Mapping in Real-Time
We propose a real-time method for odometry and mapping using range measurements from a 2-axis lidar moving in 6-DOF.
