Search Results for author:
Found 46 papers, 19 papers with code
Med-2E3: A 2D-Enhanced 3D Medical Multimodal Large Language Model
To the best of our knowledge, Med-2E3 is the first MLLM to integrate both 3D and 2D features for 3D medical image analysis.
Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE
To mitigate the tug-of-war problem of multi-modal multi-task optimization in MLLMs, recent advances primarily focus on improving the LLM components, while neglecting the connector that bridges the gap between modalities.
Enhancing elusive clues in knowledge learning by contrasting attention of language models
Causal language models acquire vast amount of knowledge from general text corpus during pretraining, but the efficiency of knowledge learning is known to be unsatisfactory, especially when learning from knowledge-dense and small-sized corpora.
Reliable and diverse evaluation of LLM medical knowledge mastery
Finally, these produced predicate variants are converted into textual language, resulting in a series of reliable and diverse test samples to evaluate whether LLMs fully master the given medical factual knowledge point.
Co-occurrence is not Factual Association in Language Models
Pretrained language models can encode a large amount of knowledge and utilize it for various reasoning tasks, yet they can still struggle to learn novel factual knowledge effectively from finetuning on limited textual demonstrations.
Spontaneous Speech-Based Suicide Risk Detection Using Whisper and Large Language Models
The early detection of suicide risk is important since it enables the intervention to prevent potential suicide attempts.
MultifacetEval: Multifaceted Evaluation to Probe LLMs in Mastering Medical Knowledge
In this paper, we aim to explore the causes of this gap by employing a multifaceted examination schema to systematically probe the actual mastery of medical knowledge by current LLMs.
Conditional Language Learning with Context
In this fashion, conditional finetuning achieves selective learning from a corpus, learning knowledge useful for downstream tasks while avoiding learning useless corpus statistics like topic biases.
Dynamic feature selection in medical predictive monitoring by reinforcement learning
Our approach addresses this limitation by enabling the selection of time-varying feature subsets for each patient.
TinyLLaVA Factory: A Modularized Codebase for Small-scale Large Multimodal Models
We present TinyLLaVA Factory, an open-source modular codebase for small-scale large multimodal models (LMMs) with a focus on simplicity of code implementations, extensibility of new features, and reproducibility of training results.
UniFS: Universal Few-shot Instance Perception with Point Representations
In this paper, we propose UniFS, a universal few-shot instance perception model that unifies a wide range of instance perception tasks by reformulating them into a dynamic point representation learning framework.
Ranked #1 on
Few-Shot Object Detection
on MS-COCO (5-shot)
Bayesian Example Selection Improves In-Context Learning for Speech, Text, and Visual Modalities
Large language models (LLMs) can adapt to new tasks through in-context learning (ICL) based on a few examples presented in dialogue history without any model parameter update.
M$^3$AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic Lecture Dataset
Although multiple academic video datasets have been constructed and released, few of them support both multimodal content recognition and understanding tasks, which is partially due to the lack of high-quality human annotations.
TinyLLaVA: A Framework of Small-scale Large Multimodal Models
We present the TinyLLaVA framework that provides a unified perspective in designing and analyzing the small-scale Large Multimodal Models (LMMs).
Ranked #148 on
Visual Question Answering
on MM-Vet
Transferring speech-generic and depression-specific knowledge for Alzheimer's disease detection
Apart from the knowledge from speech-generic representations, this paper also proposes to simultaneously transfer the knowledge from a speech depression detection task based on the high comorbidity rates of depression and AD.
QuadricsNet: Learning Concise Representation for Geometric Primitives in Point Clouds
This paper presents a novel framework to learn a concise geometric primitive representation for 3D point clouds.
Can Whisper perform speech-based in-context learning?
Language-level adaptation experiments using Chinese dialects showed that when applying SICL to isolated word ASR, consistent and considerable relative WER reductions can be achieved using Whisper models of any size on two dialects, which is on average 32. 3%.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
GKGNet: Group K-Nearest Neighbor based Graph Convolutional Network for Multi-Label Image Recognition
Multi-Label Image Recognition (MLIR) is a challenging task that aims to predict multiple object labels in a single image while modeling the complex relationships between labels and image regions.
Ranked #3 on
Multi-Label Classification
on PASCAL VOC 2007
Simple Data Augmentation Techniques for Chinese Disease Normalization
Nevertheless, the most significant obstacle to existing disease name normalization systems is the severe shortage of training data.
THiFLY Research at SemEval-2023 Task 7: A Multi-granularity System for CTR-based Textual Entailment and Evidence Retrieval
The NLI4CT task aims to entail hypotheses based on Clinical Trial Reports (CTRs) and retrieve the corresponding evidence supporting the justification.
How can Deep Learning Retrieve the Write-Missing Additional Diagnosis from Chinese Electronic Medical Record For DRG
Under the circumstance that countries generally start to adopt DRG enrollment and payment, the problem of write-missing diagnosis is a common and serious problem.
Understanding the Failure of Batch Normalization for Transformers in NLP
Batch Normalization (BN) is a core and prevalent technique in accelerating the training of deep neural networks and improving the generalization on Computer Vision (CV) tasks.
From Static to Dynamic Structures: Improving Binding Affinity Prediction with Graph-Based Deep Learning
Here, an MD dataset containing 3, 218 different protein-ligand complexes is curated, and Dynaformer, a graph-based deep learning model is further developed to predict the binding affinities by learning the geometric characteristics of the protein-ligand interactions from the MD trajectories.
Feature Forgetting in Continual Representation Learning
To study the feature forgetting problem, we create a synthetic dataset to identify and visualize the prevalence of feature forgetting in neural networks.
Table-based Fact Verification with Self-adaptive Mixture of Experts
The table-based fact verification task has recently gained widespread attention and yet remains to be a very challenging problem.
THiFly\_Queens at SemEval-2021 Task 9: Two-stage Statement Verification with Adaptive Ensembling and Slot-based Operation
This paper describes our system for verifying statements with tables at SemEval-2021 Task 9.
Distilling effective supervision for robust medical image segmentation with noisy labels
In particular, we explicitly estimate the uncertainty of every pixel as pixel-wise noise estimation, and propose pixel-wise robust learning by using both the original labels and pseudo labels.
Model information as an analysis tool in deep learning
We demonstrate using model information as a general analysis tool to gain insight into problems that arise in deep learning.
Information distance for neural network functions
We provide a practical distance measure in the space of functions parameterized by neural networks.
Multi-view Graph Contrastive Representation Learning for Drug-Drug Interaction Prediction
Drug-drug interaction(DDI) prediction is an important task in the medical health machine learning community.
Measuring Information Transfer in Neural Networks
We propose a practical measure of the generalizable information in a neural network model based on prequential coding, which we term Information Transfer ($L_{IT}$).
Applications of BERT Based Sequence Tagging Models on Chinese Medical Text Attributes Extraction
We convert the Chinese medical text attributes extraction task into a sequence tagging or machine reading comprehension task.
GCN-BMP: Investigating Graph Representation Learning for DDI Prediction Task
To alleviate this problem, we investigate the utilization of the end-to-end graph representation learning for the DDI prediction task.
Label-guided Learning for Text Classification
In this study, we propose a label-guided learning framework LguidedLearn for text representation and classification.
Tensor Graph Convolutional Networks for Text Classification
A new framework TensorGCN (tensor graph convolutional networks), is presented for this task.
Language-independent Cross-lingual Contextual Representations
Contextual representation models like BERT have achieved state-of-the-art performance on a diverse range of NLP tasks.
Learning Conceptual-Contextual Embeddings for Medical Text
External knowledge is often useful for natural language understanding tasks.
Pick-and-Learn: Automatic Quality Evaluation for Noisy-Labeled Image Segmentation
We propose a solution for network automatically evaluating the relative quality of the labels in the training set and using good ones to tune the network parameters.
Delta Embedding Learning
Evaluation also confirms the tuned word embeddings have better semantic properties.
Exploiting Sentence Embedding for Medical Question Answering
The learning framework consists of two main parts: 1) a sentence embedding producing module, and 2) a scoring module.
Medical Exam Question Answering with Large-scale Reading Comprehension
Reading and understanding text is one important component in computer aided diagnosis in clinical medicine, also being a major research problem in the field of NLP.
The Monkeytyping Solution to the YouTube-8M Video Understanding Challenge
This article describes the final solution of team monkeytyping, who finished in second place in the YouTube-8M video understanding challenge.
A Probabilistic Framework for Representing Dialog Systems and Entropy-Based Dialog Management through Dynamic Stochastic State Evolution
A Song-On-Demand task, with a total of 38117 songs and 12 attributes corresponding to each song, is used to test the performance of the proposed approach.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Simple Deep Random Model Ensemble
(ii) Based on the above two views, we propose a very simple deep learning algorithm, named deep random model ensemble (DRME).
