Search Results for author:
Found 92 papers, 43 papers with code
Noise Learning for Text Classification: A Benchmark
However, we find that noise learning in text classification is relatively underdeveloped: 1. many methods that have been proven effective in the image domain are not explored in text classification, 2. it is difficult to conduct a fair comparison between previous studies as they do experiments in different noise settings.
Sentiment Interpretable Logic Tensor Network for Aspect-Term Sentiment Analysis
SILTN is interpretable because it is a neurosymbolic formalism and a computational model that supports learning and reasoning about data with a differentiable first-order logic language (FOL).
Multimodal Multi-turn Conversation Stance Detection: A Challenge Dataset and Effective Model
To address this, we introduce a new multimodal multi-turn conversational stance detection dataset (called MmMtCSD).
ESA: Annotation-Efficient Active Learning for Semantic Segmentation
Active learning enhances annotation efficiency by selecting the most revealing samples for labeling, thereby reducing reliance on extensive human input.
Domain-invariant Representation Learning via Segment Anything Model for Blood Cell Classification
To address this issue, we propose a novel framework of domain-invariant representation learning (DoRL) via segment anything model (SAM) for blood cell classification.
Towards Cross-Domain Single Blood Cell Image Classification via Large-Scale LoRA-based Segment Anything Model
Accurate classification of blood cells plays a vital role in hematological analysis as it aids physicians in diagnosing various medical conditions.
Apple Intelligence Foundation Language Models
We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large server-based language model designed for Private Cloud Compute.
LSReGen: Large-Scale Regional Generator via Backward Guidance Framework
The third approach depends on phenomena specific to certain model architectures, complicating its application to large-scale image generation. To address these issues, we propose a novel controllable generation framework that offers a generalized interpretation of backward guidance without relying on specific assumptions.
RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Models
We present RodinHD, which can generate high-fidelity 3D avatars from a portrait image.
Core Knowledge Learning Framework for Graph Adaptation and Scalability Learning
\method{} comprises several key modules, including the core subgraph knowledge submodule, graph domain adaptation module, and few-shot learning module for downstream tasks.
White-Box 3D-OMP-Transformer for ISAC
In this work, we revisit the 3D-orthogonal matching pursuit (OMP) algorithm and demonstrate that the operation of 3D-OMP is analogous to a specific kind of transformer with 3D attention.
Structural Attention: Rethinking Transformer for Unpaired Medical Image Synthesis
To address this, we introduce UNet Structured Transformer (UNest), a novel architecture incorporating structural inductive biases for unpaired medical image synthesis.
Learning to Adapt Foundation Model DINOv2 for Capsule Endoscopy Diagnosis
Our method, inspired by the DINOv2 foundation model, applies low-rank adaptation learning to tailor foundation models for capsule endoscopy diagnosis effectively.
Pcc-tuning: Breaking the Contrastive Learning Ceiling in Semantic Textual Similarity
Semantic Textual Similarity (STS) constitutes a critical research direction in computational linguistics and serves as a key indicator of the encoding capabilities of embedding models.
Advancing Semantic Textual Similarity Modeling: A Regression Framework with Translated ReLU and Smooth K2 Loss
Since the introduction of BERT and RoBERTa, research on Semantic Textual Similarity (STS) has made groundbreaking progress.
MAVIN: Multi-Action Video Generation with Diffusion Models via Transition Video Infilling
The core challenge lies in generating smooth and natural transitions between these segments given the inherent complexity and variability of action transitions.
Leveraging Large Language Models for Semantic Query Processing in a Scholarly Knowledge Graph
DDM enables a fine-grained representation of the hierarchical structure and semantic relationships within academic papers, while KGQP leverages the KG structure to improve query accuracy and efficiency with LLMs.
Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Models
While Ferret seamlessly integrates regional understanding into the Large Language Model (LLM) to facilitate its referring and grounding capability, it poses certain limitations: constrained by the pre-trained fixed visual encoder and failed to perform well on broader tasks.
Ranked #83 on
Visual Question Answering
on MM-Vet
VeTraSS: Vehicle Trajectory Similarity Search Through Graph Modeling and Representation Learning
Trajectory similarity search plays an essential role in autonomous driving, as it enables vehicles to analyze the information and characteristics of different trajectories to make informed decisions and navigate safely in dynamic environments.
Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction
A principal issue is that in prior methods, the KG schema has to be included in the LLM prompt to generate valid triplets; larger and more complex schema easily exceed the LLMs' context window length.
Simple Techniques for Enhancing Sentence Embeddings in Generative Language Models
Sentence Embedding stands as a fundamental task within the realm of Natural Language Processing, finding extensive application in search engines, expert systems, and question-and-answer platforms.
MIPS at SemEval-2024 Task 3: Multimodal Emotion-Cause Pair Extraction in Conversations with Multimodal Language Models
This paper presents our winning submission to Subtask 2 of SemEval 2024 Task 3 on multimodal emotion cause analysis in conversations.
GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling
We introduce a radiance representation that is both structured and fully explicit and thus greatly facilitates 3D generative modeling.
Compress3D: a Compressed Latent Space for 3D Generation from a Single Image
In this paper, we present a triplane autoencoder, which encodes 3D models into a compact triplane latent space to effectively compress both the 3D geometry and texture information.
A Challenge Dataset and Effective Models for Conversational Stance Detection
Previous stance detection studies typically concentrate on evaluating stances within individual instances, thereby exhibiting limitations in effectively modeling multi-party discussions concerning the same specific topic, as naturally transpire in authentic social media interactions.
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
Through careful and comprehensive ablations of the image encoder, the vision language connector, and various pre-training data choices, we identified several crucial design lessons.
Ranked #30 on
Visual Question Answering
on MM-Vet
Decomposing Disease Descriptions for Enhanced Pathology Detection: A Multi-Aspect Vision-Language Pre-training Framework
Medical vision language pre-training (VLP) has emerged as a frontier of research, enabling zero-shot pathological recognition by comparing the query image with the textual descriptions for each disease.
MoZIP: A Multilingual Benchmark to Evaluate Large Language Models in Intellectual Property
In this paper, we contribute a new benchmark, the first Multilingual-oriented quiZ on Intellectual Property (MoZIP), for the evaluation of LLMs in the IP domain.
Source-Free Unsupervised Domain Adaptation with Hypothesis Consolidation of Prediction Rationale
Source-Free Unsupervised Domain Adaptation (SFUDA) is a challenging task where a model needs to be adapted to a new domain without access to target domain labels or source domain data.
Vision Reimagined: AI-Powered Breakthroughs in WiFi Indoor Imaging
This is the first research work to consider WiFi indoor imaging as a multi-modal image generation task that converts the measured WiFi power into a high-resolution indoor image.
Cross-target Stance Detection by Exploiting Target Analytical Perspectives
In this paper, we propose a Multi-Perspective Prompt-Tuning (MPPT) model for CTSD that uses the analysis perspective as a bridge to transfer knowledge.
WildlifeMapper: Aerial Image Analysis for Multi-Species Detection and Identification
We introduce WildlifeMapper (WM) a flexible model designed to detect locate and identify multiple species in aerial imagery.
A Logically Consistent Chain-of-Thought Approach for Stance Detection
Zero-shot stance detection (ZSSD) aims to detect stances toward unseen targets.
SegReg: Segmenting OARs by Registering MR Images and CT Annotations
This manual process is highly time-consuming and expensive, limiting the number of patients who can receive timely radiotherapy.
Multimodal Large Language Model for Visual Navigation
Recent efforts to enable visual navigation using large language models have mainly focused on developing complex prompt systems.
Ferret: Refer and Ground Anything Anywhere at Any Granularity
We introduce Ferret, a new Multimodal Large Language Model (MLLM) capable of understanding spatial referring of any shape or granularity within an image and accurately grounding open-vocabulary descriptions.
VeCLIP: Improving CLIP Training via Visual-enriched Captions
For example, VeCLIP achieves up to +25. 2% gain in COCO and Flickr30k retrieval tasks under the 12M setting.
Compression Ratio Learning and Semantic Communications for Video Imaging
In this article, we also investigate the data transmission methods for programmable sensors, where the performance of communication systems is evaluated by the reconstructed images or videos rather than the transmission of sensor data itself.
Reconstructing 3D Human Pose from RGB-D Data with Occlusions
Our key idea is to constrain the solution space of the human body by considering the occluded body parts and visible body parts separately: modeling all plausible poses where the occluded body parts do not penetrate the scene, and constraining the visible body parts using depth data.
Compressing LLMs: The Truth is Rarely Pure and Never Simple
Recently, several works have shown significant success in training-free and data-free compression (pruning and quantization) of LLMs that achieve 50 - 60% sparsity and reduce the bit width to 3 or 4 bits per weight, with negligible degradation of perplexity over the uncompressed baseline.
CoT-BERT: Enhancing Unsupervised Sentence Representation through Chain-of-Thought
In response to these challenges, this paper presents CoT-BERT, an innovative method that harnesses the progressive thinking of Chain-of-Thought reasoning to tap into the latent potential of pre-trained models like BERT.
Mobile V-MoEs: Scaling Down Vision Transformers via Sparse Mixture-of-Experts
We empirically show that our sparse Mobile Vision MoEs (V-MoEs) can achieve a better trade-off between performance and efficiency than the corresponding dense ViTs.
Likelihood-Based Text-to-Image Evaluation with Patch-Level Perceptual and Semantic Credit Assignment
In this paper, we propose to evaluate text-to-image generation performance by directly estimating the likelihood of the generated images using a pre-trained likelihood-based text-to-image generative model, i. e., a higher likelihood indicates better perceptual quality and better text-image alignment.
Category Feature Transformer for Semantic Segmentation
Aggregation of multi-stage features has been revealed to play a significant role in semantic segmentation.
Dynamic Token Pruning in Plain Vision Transformers for Semantic Segmentation
Experiments suggest that the proposed DToP architecture reduces on average $20\% - 35\%$ of computational cost for current semantic segmentation methods based on plain vision transformers without accuracy degradation.
MetaDiff: Meta-Learning with Conditional Diffusion for Few-Shot Learning
Its key idea is learning a deep model in a bi-level optimization manner, where the outer-loop process learns a shared gradient descent algorithm (i. e., its hyperparameters), while the inner-loop process leverage it to optimize a task-specific model by using only few labeled data.
Investigating the Learning Behaviour of In-context Learning: A Comparison with Supervised Learning
Large language models (LLMs) have shown remarkable capacity for in-context learning (ICL), where learning a new task from just a few training examples is done without being explicitly pre-trained.
Semantic-Aware Image Compressed Sensing
According to the base CS results, the encoder then employs a policy network to analyze the semantic information in images and determines the measurement matrix for different image areas.
MOFI: Learning Image Representations from Noisy Entity Annotated Images
Our approach involves employing a named entity recognition model to extract entities from the alt-text, and then using a CLIP model to select the correct entities as labels of the paired image.
BPKD: Boundary Privileged Knowledge Distillation For Semantic Segmentation
Current knowledge distillation approaches in semantic segmentation tend to adopt a holistic approach that treats all spatial locations equally.
SegViTv2: Exploring Efficient and Continual Semantic Segmentation with Plain Vision Transformers
This paper investigates the capability of plain Vision Transformers (ViTs) for semantic segmentation using the encoder-decoder framework and introduces \textbf{SegViTv2}.
Ranked #16 on
Semantic Segmentation
on ADE20K
Less is More: Removing Text-regions Improves CLIP Training Efficiency and Robustness
In this paper, we discuss two effective approaches to improve the efficiency and robustness of CLIP training: (1) augmenting the training dataset while maintaining the same number of optimization steps, and (2) filtering out samples that contain text regions in the image.
Investigating Chain-of-thought with ChatGPT for Stance Detection on Social Media
Stance detection predicts attitudes towards targets in texts and has gained attention with the rise of social media.
Large Language Models as Zero-Shot Human Models for Human-Robot Interaction
In this work, we explore the potential of large-language models (LLMs) -- which have consumed vast amounts of human-generated text data -- to act as zero-shot human models for HRI.
Semantic Communications with Variable-Length Coding for Extended Reality
Wireless extended reality (XR) has attracted wide attentions as a promising technology to improve users' mobility and quality of experience.
Geometric Deep Learning for Autonomous Driving: Unlocking the Power of Graph Neural Networks With CommonRoad-Geometric
Heterogeneous graphs offer powerful data representations for traffic, given their ability to model the complex interaction effects among a varying number of traffic participants and the underlying road infrastructure.
STAIR: Learning Sparse Text and Image Representation in Grounded Tokens
We extend the CLIP model and build a sparse text and image representation (STAIR), where the image and text are mapped to a sparse token space.
MV-Adapter: Multimodal Video Transfer Learning for Video Text Retrieval
The first is a Temporal Adaptation Module that is incorporated in the video branch to introduce global and local temporal contexts.
A separation logic for sequences in pointer programs and its decidability
To deal with properties on variable-length sequences and multilevel data structures, we propose sequence-heap separation logic which integrates sequences into logical reasoning on heap-manipulated programs.
How would Stance Detection Techniques Evolve after the Launch of ChatGPT?
ChatGPT has the potential to be the best AI model for stance detection tasks in NLP, or at least change the research paradigm of this field.
Semantic Sensing and Communications for Ultimate Extended Reality
In particular, semantic sensing is used to improve the sensing efficiency by exploring the spatial-temporal distributions of semantic information.
MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation
In this work, we propose an ID-preserving talking head generation framework, which advances previous methods in two aspects.
ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation
Recently, CLIP has been applied to pixel-level zero-shot learning tasks via a two-stage scheme.
Ranked #4 on
Zero-Shot Semantic Segmentation
on PASCAL VOC
Context-Matched Collage Generation for Underwater Invertebrate Detection
However, in many scenarios, it can be difficult to collect images for training, not to mention the costs associated with collecting annotations suitable for training these object detectors.
Safety-Constrained Policy Transfer with Successor Features
In this work, we focus on the problem of safe policy transfer in reinforcement learning: we seek to leverage existing policies when learning a new task with specified constraints.
SegViT: Semantic Segmentation with Plain Vision Transformers
We explore the capability of plain Vision Transformers (ViTs) for semantic segmentation and propose the SegVit.
Ranked #4 on
Semantic Segmentation
on COCO-Stuff test
Shape Completion with Points in the Shadow
The core challenge is to generate plausible geometries to fill the unobserved part of the object based on a partial scan, which is under-constrained and suffers from a huge solution space.
Context-Driven Detection of Invertebrate Species in Deep-Sea Video
Each video includes annotations indicating the start and end times of substrates across the video in addition to counts of species of interest.
CSI-fingerprinting Indoor Localization via Attention-Augmented Residual Convolutional Neural Network
On the other hand, considering the generality of a tracking system, we decouple the tracking system from the CSI environments so that one tracking system for all environments becomes possible.
Deep Learning-Based Perceptual Stimulus Encoder for Bionic Vision
Retinal implants have the potential to treat incurable blindness, yet the quality of the artificial vision they produce is still rudimentary.
StyleSwin: Transformer-based GAN for High-resolution Image Generation
To this end, we believe that local attention is crucial to strike the balance between computational efficiency and modeling capacity.
Ranked #1 on
Image Generation
on CelebA 256x256
(FID metric)
Margin Calibration for Long-Tailed Visual Recognition
The long-tailed class distribution in visual recognition tasks poses great challenges for neural networks on how to handle the biased predictions between head and tail classes, i. e., the model tends to classify tail classes as head classes.
Co-training Transformer with Videos and Images Improves Action Recognition
We term this approach as Co-training Videos and Images for Action Recognition (CoVeR).
Ranked #6 on
Action Classification
on MiT
(using extra training data)
FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling
However, like other modern SSL algorithms, FixMatch uses a pre-defined constant threshold for all classes to select unlabeled data that contribute to the training, thus failing to consider different learning status and learning difficulties of different classes.
Visually Grounded Concept Composition
We investigate ways to compose complex concepts in texts from primitive ones while grounding them in images.
Systematic Generalization on gSCAN: What is Nearly Solved and What is Next?
We analyze the grounded SCAN (gSCAN) benchmark, which was recently proposed to study systematic generalization for grounded language understanding.
Dynamic Neural Representational Decoders for High-Resolution Semantic Segmentation
This neural representation enables our decoder to leverage the smoothness prior in the semantic label space, and thus makes our decoder more efficient.
Enhanced Hyperspectral Image Super-Resolution via RGB Fusion and TV-TV Minimization
Such methods, however, cannot guarantee that the input measurements are satisfied in the recovered image, since the learned parameters by the network are applied to every test image.
CREATe: Clinical Report Extraction and Annotation Technology
Clinical case reports are written descriptions of the unique aspects of a particular clinical case, playing an essential role in sharing clinical experiences about atypical disease phenotypes and new therapies.
Instance and Panoptic Segmentation Using Conditional Convolutions
In the literature, top-performing instance segmentation methods typically follow the paradigm of Mask R-CNN and rely on ROI operations (typically ROIAlign) to attend to each instance.
A Hierarchical Multi-Modal Encoder for Moment Localization in Video Corpus
Identifying a short segment in a long video that semantically matches a text query is a challenging task that has important application potentials in language-based video search, browsing, and navigation.
Solving Sparse Linear Inverse Problems in Communication Systems: A Deep Learning Approach With Adaptive Depth
Sparse signal recovery problems from noisy linear measurements appear in many areas of wireless communications.
Learning to Represent Image and Text with Denotation Graph
Recent progresses have leveraged the ideas of pre-training (from language modeling) and attention layers in Transformers to learn representation from datasets containing images aligned with linguistic expressions that describe the images.
Online Action Detection in Streaming Videos with Time Buffers
We formulate the problem of online temporal action detection in live streaming videos, acknowledging one important property of live streaming videos that there is normally a broadcast delay between the latest captured frame and the actual frame viewed by the audience.
Enhancing Cross-target Stance Detection with Transferable Semantic-Emotion Knowledge
Specifically, a semantic-emotion heterogeneous graph is constructed from external semantic and emotion lexicons, which is then fed into a graph convolutional network to learn multi-hop semantic connections between words and emotion tags.
Challenging the adversarial robustness of DNNs based on error-correcting output codes
The existence of adversarial examples and the easiness with which they can be generated raise several security concerns with regard to deep learning systems, pushing researchers to develop suitable defense mechanisms.
Visual Storytelling via Predicting Anchor Word Embeddings in the Stories
To narrate a sequence of images, we use the predicted anchor word embeddings and the image features as the joint input to a seq2seq model.
Ranked #15 on
Visual Storytelling
on VIST
Attacking CNN-based anti-spoofing face authentication in the physical domain
In this paper, we study the vulnerability of anti-spoofing methods based on deep learning against adversarial perturbations.
Cryptography and Security
Cross-Modal and Hierarchical Modeling of Video and Text
Similarly, a paragraph may contain sentences with different topics, which collectively conveys a coherent message or story.
Few Shot Learning with Simplex
During testing, combined with the test sample and the points in the class, a new simplex is formed.
Real-time Action Recognition with Enhanced Motion Vector CNNs
The deep two-stream architecture exhibited excellent performance on video based action recognition.
Ranked #73 on
Action Recognition
on UCF101
