Search Results for author:
Found 32 papers, 16 papers with code
p-MoD: Building Mixture-of-Depths MLLMs via Progressive Ratio Decay
In this paper, we propose to build efficient MLLMs by leveraging the Mixture-of-Depths (MoD) mechanism, where each transformer decoder layer selects essential vision tokens to process while skipping redundant ones.
Weakly Supervised Framework Considering Multi-temporal Information for Large-scale Cropland Mapping with Satellite Imagery
On the one hand, to alleviate the overfitting problem caused by the model's over-trust of remaining errors in high-quality labels, we encode the similarity/aggregation of cropland in the visual/spatial domain to construct the unsupervised learning signal, and take it as the regularization term to constrain the supervised part.
Class-RAG: Real-Time Content Moderation with Retrieval Augmented Generation
Robust content moderation classifiers are essential for the safety of Generative AI systems.
ExpLLM: Towards Chain of Thought for Facial Expression Recognition
Specifically, we have designed the CoT mechanism from three key perspectives: key observations, overall emotional interpretation, and conclusion.
Facial Expression Recognition
Facial Expression Recognition (FER)
CogVLM2: Visual Language Models for Image and Video Understanding
Beginning with VisualGLM and CogVLM, we are continuously exploring VLMs in pursuit of enhanced vision-language fusion, efficient higher-resolution architecture, and broader modalities and applications.
Ranked #10 on
Visual Question Answering
on MM-Vet
Exploring The Neural Burden In Pruned Models: An Insight Inspired By Neuroscience
Vision Transformer and its variants have been adopted in many visual tasks due to their powerful capabilities, which also bring significant challenges in computation and storage.
LVBench: An Extreme Long Video Understanding Benchmark
To address this gap, we introduce LVBench, a benchmark specifically designed for long video understanding.
MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classification
This highlights the challenging nature of our benchmark for existing models and the significant gap between the multimodal reasoning capabilities of current models and humans.
An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning
Large language models (LLMs) are displaying emergent abilities for math reasoning tasks, and there is a growing attention on enhancing the ability of open-source LLMs through supervised fine-tuning (SFT). In this paper, we aim to explore a general data strategy for supervised data to help optimize and expand math reasoning ability. Firstly, we determine the ability boundary of reasoning paths augmentation by identifying these paths' minimal optimal set. Secondly, we validate that different abilities of the model can be cumulatively enhanced by Mix of Minimal Optimal Sets of corresponding types of data, while our models MMOS achieve SOTA performance on series base models under much lower construction costs. Besides, we point out GSM-HARD is not really hard and today's LLMs no longer lack numerical robustness. Also, we provide an Auto Problem Generator for robustness testing and educational applications. Our code and data are publicly available at https://github. com/cyzhh/MMOS.
Ranked #2 on
Math Word Problem Solving
on ASDiv-A
(using extra training data)
CogCoM: Train Large Vision-Language Models Diving into Details through Chain of Manipulations
Drawing inspiration from human cognition in solving visual problems (e. g., marking, zoom in), this paper introduces Chain of Manipulations, a mechanism that enables VLMs to solve problems step-by-step with evidence.
LongAlign: A Recipe for Long Context Alignment of Large Language Models
Extending large language models to effectively handle long contexts requires instruction fine-tuning on input sequences of similar length.
CogVLM: Visual Expert for Pretrained Language Models
We introduce CogVLM, a powerful open-source visual language foundation model.
Ranked #4 on
Visual Question Answering (VQA)
on InfiMM-Eval
Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment
Open Information Extraction (OIE) aims to extract objective structured knowledge from natural texts, which has attracted growing attention to build dedicated models with human experience.
VidCoM: Fast Video Comprehension through Large Language Models with Multimodal Tools
Building models that comprehends videos and responds specific user instructions is a practical and challenging topic, as it requires mastery of both vision understanding and knowledge reasoning.
Tree-of-Mixed-Thought: Combining Fast and Slow Thinking for Multi-hop Visual Reasoning
Our approach succeeds in performance while significantly saving inference steps.
KoLA: Carefully Benchmarking World Knowledge of Large Language Models
The unprecedented performance of large language models (LLMs) necessitates improvements in evaluations.
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction
In this paper, we present the first benchmark that simulates the evaluation of open information extraction models in the real world, where the syntactic and expressive distributions under the same knowledge meaning may drift variously.
GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation
Despite the recent emergence of video captioning models, how to generate vivid, fine-grained video descriptions based on the background knowledge (i. e., long and informative commentary about the domain-specific scenes with appropriate reasoning) is still far from being solved, which however has great applications such as automatic sports narrative.
Syntactically Robust Training on Partially-Observed Data for Open Information Extraction
In this paper, we propose a syntactically robust training framework that enables models to be trained on a syntactic-abundant distribution based on diverse paraphrase generation.
Privacy-Preserving Federated Deep Clustering based on GAN
Federated clustering (FC) is an essential extension of centralized clustering designed for the federated setting, wherein the challenge lies in constructing a global similarity measure without the need to share private data.
Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works
Deep learning has achieved great success in learning features from massive remote sensing images (RSIs).
Federated clustering with GAN-based data synthesis
Federated clustering (FC) is an extension of centralized clustering in federated settings.
ConstGCN: Constrained Transmission-based Graph Convolutional Networks for Document-level Relation Extraction
Document-level relation extraction with graph neural networks faces a fundamental graph construction gap between training and inference - the golden graph structure only available during training, which causes that most methods adopt heuristic or syntactic rules to construct a prior graph as a pseudo proxy.
Selective clustering ensemble based on kappa and F-score
Clustering ensemble has an impressive performance in improving the accuracy and robustness of partition results and has received much attention in recent years.
DiaKG: an Annotated Diabetes Dataset for Medical Knowledge Graph Construction
In order to accelerate the research for domain-specific knowledge graphs in the medical domain, we introduce DiaKG, a high-quality Chinese dataset for Diabetes knowledge graph, which contains 22, 050 entities and 6, 890 relations in total.
Ordered Semiautomatic Rings with Applications to Geometry
The present work looks at semiautomatic rings with automatic addition and comparisons which are dense subrings of the real numbers and asks how these can be used to represent geometric objects such that certain operations and transformations are automatic.
Formal Languages and Automata Theory Logic
Remote Sensing Image Scene Classification with Self-Supervised Paradigm under Limited Labeled Samples
With the development of deep learning, supervised learning methods perform well in remote sensing images (RSIs) scene classification.
Spatial Information Inference Net: Road Extraction Using Road-Specific Contextual Information
The validation experiments using three large datasets of very high-resolution (VHR) satellite imagery show that the proposed method can improve road extraction accuracy and provide an output that is more in line with human expectations.
A Self-Attentive Model with Gate Mechanism for Spoken Language Understanding
In this work, we propose a novel self-attentive model with gate mechanism to fully utilize the semantic correlation between slot and intent.
Chinese Grammatical Error Diagnosis Based on Policy Gradient LSTM Model
Chinese grammatical error diagnosis system is a very important tool, which can help Chinese learners automatically diagnose grammatical errors in many scenarios.
Endoscopic Depth Measurement and Super-Spectral-Resolution Imaging
Intra-operative measurements of tissue shape and multi/ hyperspectral information have the potential to provide surgical guidance and decision making support.
Probe-based Rapid Hybrid Hyperspectral and Tissue Surface Imaging Aided by Fully Convolutional Networks
In HSI mode standard endoscopic illumination is used, with the fibre probe collecting reflected light and encoding the spatial information into a linear format that can be imaged onto the slit of a spectrograph.
