Search Results for author:
Found 457 papers, 178 papers with code
Shape Robust Text Detection with Progressive Scale Expansion Network
Due to the fact that there are large geometrical margins among the minimal scale kernels, our method is effective to split the close text instances, making it easier to use segmentation-based methods to detect arbitrary-shaped text instances.
Ranked #12 on
Scene Text Detection
on SCUT-CTW1500
Selective Kernel Networks
A building block called Selective Kernel (SK) unit is designed, in which multiple branches with different kernel sizes are fused using softmax attention that is guided by the information in these branches.
Ranked #98 on
Image Classification
on CIFAR-100
(using extra training data)
PVT v2: Improved Baselines with Pyramid Vision Transformer
We hope this work will facilitate state-of-the-art Transformer researches in computer vision.
Ranked #23 on
Object Detection
on COCO-O
Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection
Specifically, we merge the quality estimation into the class prediction vector to form a joint representation of localization quality and classification, and use a vector to represent arbitrary distribution of box locations.
Ranked #93 on
Object Detection
on COCO test-dev
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Unlike the recently-proposed Transformer model (e. g., ViT) that is specially designed for image classification, we propose Pyramid Vision Transformer~(PVT), which overcomes the difficulties of porting Transformer to various dense prediction tasks.
Ranked #5 on
Semantic Segmentation
on SynPASS
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Our work, for the first time, uncovers that properly aligning the visual features with an advanced large language model can possess numerous advanced multi-modal abilities demonstrated by GPT-4, such as detailed image description generation and website creation from hand-drawn drafts.
Ranked #9 on
Visual Question Answering
on BenchLMM
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning
Motivated by this, we target to build a unified interface for completing many vision-language tasks including image description, visual question answering, and visual grounding, among others.
Ranked #10 on
Visual Question Answering
on BenchLMM
Supervised Community Detection with Line Graph Neural Networks
We show that, in a data-driven manner and without access to the underlying generative models, they can match or even surpass the performance of the belief propagation algorithm on binary and multi-class stochastic block models, which is believed to reach the computational threshold.
Ranked #1 on
Community Detection
on Amazon
(Accuracy-NE metric, using extra
training data)
Generalized Focal Loss V2: Learning Reliable Localization Quality Estimation for Dense Object Detection
Such a property makes the distribution statistics of a bounding box highly correlated to its real localization quality.
Ranked #26 on
Object Detection
on COCO-O
Pelee: A Real-Time Object Detection System on Mobile Devices
In this study, we propose an efficient architecture named PeleeNet, which is built with conventional convolution instead.
Learning Tree-based Deep Model for Recommender Systems
In systems with large corpus, however, the calculation cost for the learnt model to predict all user-item preferences is tremendous, which makes full corpus retrieval extremely difficult.
Shape Robust Text Detection with Progressive Scale Expansion Network
To address these problems, we propose a novel Progressive Scale Expansion Network (PSENet), designed as a segmentation-based detector with multiple predictions for each text instance.
Ranked #12 on
Scene Text Detection
on ICDAR 2017 MLT
Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks
The Convolutional Neural Networks (CNNs) generate the feature representation of complex objects by collecting hierarchical and different parts of semantic sub-features.
Ranked #739 on
Image Classification
on ImageNet
ConvLab: Multi-Domain End-to-End Dialog System Platform
We present ConvLab, an open-source multi-domain end-to-end dialog system platform, that enables researchers to quickly set up experiments with reusable components and compare a large set of different approaches, ranging from conventional pipeline systems to end-to-end neural models, in common environments.
Evaluating Large Language Models for Radiology Natural Language Processing
The rise of large language models (LLMs) has marked a pivotal shift in the field of natural language processing (NLP).
ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems
We present ConvLab-2, an open-source toolkit that enables researchers to build task-oriented dialogue systems with state-of-the-art models, perform an end-to-end evaluation, and diagnose the weakness of systems.
PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text
By systematically comparing with existing scene text representations, we show that our kernel representation can not only describe arbitrarily-shaped text but also well distinguish adjacent text.
Video ChatCaptioner: Towards Enriched Spatiotemporal Descriptions
Video captioning aims to convey dynamic scenes from videos using natural language, facilitating the understanding of spatiotemporal information within our environment.
MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Training LLMs at this scale brings unprecedented challenges to training efficiency and stability.
Large Selective Kernel Network for Remote Sensing Object Detection
To the best of our knowledge, this is the first time that large and selective kernel mechanisms have been explored in the field of remote sensing object detection.
Ranked #1 on
Semantic Segmentation
on UAVid
LSKNet: A Foundation Lightweight Backbone for Remote Sensing
While a considerable amount of research has been dedicated to remote sensing classification, object detection and semantic segmentation, most of these studies have overlooked the valuable prior knowledge embedded within remote sensing scenarios.
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks
Conventional task- and modality-specific artificial intelligence (AI) models are inflexible in real-world deployment and maintenance for biomedicine.
Ranked #1 on
Text Summarization
on MeQSum
FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation
In this paper, we are interested in few-shot object segmentation where the number of annotated training examples are limited to 5 only.
Ranked #20 on
Few-Shot Semantic Segmentation
on FSS-1000 (5-shot)
TrustLLM: Trustworthiness in Large Language Models
This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions.
On the Convergence of FedAvg on Non-IID Data
In this paper, we analyze the convergence of \texttt{FedAvg} on non-iid data and establish a convergence rate of $\mathcal{O}(\frac{1}{T})$ for strongly convex and smooth problems, where $T$ is the number of SGDs.
Uniform Masking: Enabling MAE Pre-training for Pyramid-based Vision Transformers with Locality
Masked AutoEncoder (MAE) has recently led the trends of visual self-supervision area by an elegant asymmetric encoder-decoder design, which significantly optimizes both the pre-training efficiency and fine-tuning accuracy.
Ranked #37 on
Object Detection
on COCO minival
Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security
Next, we discuss several key challenges to achieve intelligent, efficient and secure Personal LLM Agents, followed by a comprehensive survey of representative solutions to address these challenges.
Dynamic Feature Fusion for Semantic Edge Detection
In this work, we propose a novel dynamic feature fusion strategy that assigns different fusion weights for different input images and locations adaptively.
SARDet-100K: Towards Open-Source Benchmark and ToolKit for Large-Scale SAR Object Detection
To the best of our knowledge, SARDet-100K is the first COCO-level large-scale multi-class SAR object detection dataset ever created.
Ranked #1 on
2D Object Detection
on SARDet-100K
(using extra training data)
Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark
Monocular depth estimation aims at predicting depth from a single image or video.
A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond
Building on our examination of the developmental trajectories, we further investigate the emerging synergies between code intelligence and broader machine intelligence, uncovering new cross-domain opportunities and illustrating the substantial influence of code intelligence across various domains.
Disentangling User Interest and Conformity for Recommendation with Causal Embedding
We further demonstrate that the learned embeddings successfully capture the desired causes, and show that DICE guarantees the robustness and interpretability of recommendation.
PseCo: Pseudo Labeling and Consistency Training for Semi-Supervised Object Detection
Specifically, for pseudo labeling, existing works only focus on the classification score yet fail to guarantee the localization precision of pseudo boxes; For consistency training, the widely adopted random-resize training only considers the label-level consistency but misses the feature-level one, which also plays an important role in ensuring the scale invariance.
DTG-SSOD: Dense Teacher Guidance for Semi-Supervised Object Detection
Specifically, we propose the Inverse NMS Clustering (INC) and Rank Matching (RM) to instantiate the dense supervision, without the widely used, conventional sparse pseudo labels.
Curriculum Temperature for Knowledge Distillation
In this paper, we propose a simple curriculum-based technique, termed Curriculum Temperature for Knowledge Distillation (CTKD), which controls the task difficulty level during the student's learning career through a dynamic and learnable temperature.
Few-Shot Charge Prediction with Discriminative Legal Attributes
Specifically, our model outperforms other baselines by more than 50{\%} in the few-shot scenario.
u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model
Recent advancements in multi-modal large language models (MLLMs) have led to substantial improvements in visual understanding, primarily driven by sophisticated modality alignment strategies.
CrossKD: Cross-Head Knowledge Distillation for Object Detection
Moreover, as mimicking the teacher's predictions is the target of KD, CrossKD offers more task-oriented information in contrast with feature imitation.
Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report
While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints.
Dynamic MLP for Fine-Grained Image Classification by Leveraging Geographical and Temporal Information
Therefore, it is helpful to leverage additional information, e. g., the locations and dates for data shooting, which can be easily accessible but rarely exploited.
Vision-Language Models in Remote Sensing: Current Progress and Future Trends
Existing AI-related research in remote sensing primarily focuses on visual understanding tasks while neglecting the semantic understanding of the objects and their relationships.
PromptKD: Unsupervised Prompt Distillation for Vision-Language Models
To our best knowledge, we are the first to (1) perform unsupervised domain-specific prompt-driven knowledge distillation for CLIP, and (2) establish a practical pre-storing mechanism of text features as shared class vectors between teacher and student.
Ranked #1 on
Prompt Engineering
on Oxford-IIIT Pet Dataset
StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learning
Reinforcement Learning (RL) can be considered as a sequence modeling task: given a sequence of past state-action-reward experiences, an agent predicts a sequence of next actions.
Neuron-level Structured Pruning using Polarization Regularizer
Experimentally, we show that structured pruning using polarization regularizer achieves much better results than using L1 regularizer.
MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation
The Segment Anything Model (SAM), a foundation model for general image segmentation, has demonstrated impressive zero-shot performance across numerous natural image segmentation tasks.
An End-to-end Chinese Text Normalization Model based on Rule-guided Flat-Lattice Transformer
Inspired by Flat-LAttice Transformer (FLAT), we propose an end-to-end Chinese text normalization model, which accepts Chinese characters as direct input and integrates expert knowledge contained in rules into the neural network, both contribute to the superior performance of proposed model for the text normalization task.
Community Detection with Graph Neural Networks
This graph inference task can be recast as a node-wise graph classification problem, and, as such, computational detection thresholds can be translated in terms of learning within appropriate models.
Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling
To strike a balance in this trade-off, we propose Elapsed-Time Sampling Delayed Feedback Model (ES-DFM), which models the relationship between the observed conversion distribution and the true conversion distribution.
Meta-Learning Triplet Network with Adaptive Margins for Few-Shot Named Entity Recognition
We design an improved triplet network to map samples and prototype vectors into a low-dimensional space that is easier to be classified and propose an adaptive margin for each entity type.
Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal
Specifically, a shadow image is fed into the first generator which produces a shadow detection mask.
Ranked #3 on
RGB Salient Object Detection
on ISTD
Forward Laplacian: A New Computational Framework for Neural Network-based Variational Monte Carlo
Neural network-based variational Monte Carlo (NN-VMC) has emerged as a promising cutting-edge technique of ab initio quantum chemistry.
Joint Optimization of Tree-based Index and Deep Model for Recommender Systems
The previous work Tree-based Deep Model (TDM) \cite{zhu2018learning} greatly improves recommendation accuracy using tree index.
RSGPT: A Remote Sensing Vision Language Model and Benchmark
This dataset consists of human-annotated captions and visual question-answer pairs, allowing for a comprehensive assessment of VLMs in the context of RS.
Non-Rigid Point Set Registration Networks
In contrast, the proposed point registration neural network (PR-Net) actively learns the registration pattern as a parametric function from a training dataset, consequently predict the desired geometric transformation to align a pair of point sets.
Learning from Graphs with Heterophily: Progress and Future
Graphs are structured data that models complex relations between real-world entities.
MonoOcc: Digging into Monocular Semantic Occupancy Prediction
However, existing methods rely on a complex cascaded framework with relatively limited information to restore 3D scenes, including a dependency on supervision solely on the whole network's output, single-frame input, and the utilization of a small backbone.
One-Stage Cascade Refinement Networks for Infrared Small Target Detection
Finally, we present a new research benchmark for infrared small target detection, consisting of the SIRST-V2 dataset of real-world, high-resolution single-frame targets, the normalized contrast evaluation metric, and the DeepInfrared toolkit for detection.
Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V
In this paper, we critically evaluate the capabilities of the state-of-the-art multimodal large language model, i. e., GPT-4 with Vision (GPT-4V), on Visual Question Answering (VQA) task.
Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation?
Diffusion models have recently achieved astonishing performance in generating high-fidelity photo-realistic images.
Finding Global Homophily in Graph Neural Networks When Meeting Heterophily
Further, for other homophilous nodes excluded in the neighborhood, they are ignored for information aggregation.
Ranked #2 on
Node Classification
on pokec
Speech Emotion Recognition with Global-Aware Fusion on Multi-scale Feature Representation
Speech Emotion Recognition (SER) is a fundamental task to predict the emotion label from speech data.
CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Models
The pursuit of modern models, like Diffusion Models (DMs), holds promise for achieving high-fidelity, real-time speech synthesis.
Ranking-Enhanced Unsupervised Sentence Representation Learning
In this work, we show that the semantic meaning of a sentence is also determined by nearest-neighbor sentences that are similar to the input sentence.
Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learning
Sequence modeling approaches have shown promising results in robot imitation learning.
Decoding Natural Images from EEG for Object Recognition
This paper presents a self-supervised framework to demonstrate the feasibility of learning image representations from EEG signals, particularly for object recognition.
DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4
The digitization of healthcare has facilitated the sharing and re-using of medical data but has also raised concerns about confidentiality and privacy.
Coherent Point Drift Networks: Unsupervised Learning of Non-Rigid Point Set Registration
In contrast to previous efforts (e. g. coherent point drift), CPD-Net can learn displacement field function to estimate geometric transformation from a training dataset, consequently, to predict the desired geometric transformation for the alignment of previously unseen pairs without any additional iterative optimization process.
Towards Spatial Equilibrium Object Detection
In this paper, we study the spatial disequilibrium problem of modern object detectors and propose to quantify this ``spatial bias'' by measuring the detection performance over zones.
Zone Evaluation: Revealing Spatial Bias in Object Detection
A fundamental limitation of object detectors is that they suffer from "spatial bias", and in particular perform less satisfactorily when detecting objects near image borders.
SESR: Single Image Super Resolution with Recursive Squeeze and Excitation Networks
Single image super resolution is a very important computer vision task, with a wide range of applications.
Ranked #34 on
Image Super-Resolution
on BSD100 - 4x upscaling
Multi-Modal Sarcasm Detection via Cross-Modal Graph Convolutional Network
Then, the descriptions of the objects are served as a bridge to determine the importance of the association between the objects of image modality and the contextual words of text modality, so as to build a cross-modal graph for each multi-modal instance.
Customizable Perturbation Synthesis for Robust SLAM Benchmarking
To this end, we propose a novel, customizable pipeline for noisy data synthesis, aimed at assessing the resilience of multi-modal SLAM models against various perturbations.
Exchanging-based Multimodal Fusion with Transformer
To solve these issues, in this paper, we propose a novel exchanging-based multimodal fusion model MuSE for text-vision fusion based on Transformer.
Mixed Link Networks
Basing on the analysis by revealing the equivalence of modern networks, we find that both ResNet and DenseNet are essentially derived from the same "dense topology", yet they only differ in the form of connection -- addition (dubbed "inner link") vs. concatenation (dubbed "outer link").
One-Shot Object Detection without Fine-Tuning
Deep learning has revolutionized object detection thanks to large-scale datasets, but their object categories are still arguably very limited.
Unified Style Transfer
Currently, it is hard to compare and evaluate different style transfer algorithms due to chaotic definitions of style and the absence of agreed objective validation methods in the study of style transfer.
VIPTR: A Vision Permutable Extractor for Fast and Efficient Scene Text Recognition
In this work, we propose the VIsion Permutable extractor for fast and efficient scene Text Recognition (VIPTR), which achieves an impressive balance between high performance and rapid inference speeds in the domain of STR.
Bridging Unpaired Facial Photos And Sketches By Line-drawings
In this paper, we propose a novel method to learn face sketch synthesis models by using unpaired data.
Towards Robust Referring Video Object Segmentation with Cyclic Relational Consensus
Referring Video Object Segmentation (R-VOS) is a challenging task that aims to segment an object in a video based on a linguistic expression.
Ranked #11 on
Referring Video Object Segmentation
on Refer-YouTube-VOS
Referring Expression Segmentation
Referring Video Object Segmentation
+2
nanoLM: an Affordable LLM Pre-training Benchmark via Accurate Loss Prediction across Scales
With around 14% of the one-time pre-training cost, we can accurately forecast the loss for models up to 52B.
Towards Robust Audiovisual Segmentation in Complex Environments with Quantization-based Semantic Decomposition
We propose a semantic decomposition method based on product quantization, where the multi-source semantics can be decomposed and represented by several disentangled and noise-suppressed single-source semantics.
$\text{R}^2$-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations
Referring perception, which aims at grounding visual objects with multimodal referring guidance, is essential for bridging the gap between humans, who provide instructions, and the environment where intelligent systems perceive.
Idea-2-3D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputs
The definition of an IDEA is the composition of multimodal inputs including text, image, and 3D models.
Directionally Constrained Fully Convolutional Neural Network For Airborne Lidar Point Cloud Classification
In this paper, we proposed a directionally constrained fully convolutional neural network (D-FCN) that can take the original 3D coordinates and LiDAR intensity as input; thus, it can directly apply to unstructured 3D point clouds for semantic labeling.
RecursiveMix: Mixed Learning with History
In semantic segmentation, RM also surpasses the baseline and CutMix by 1. 9 and 1. 1 mIoU points under UperNet on ADE20K, respectively.
Peekaboo: Text to Image Diffusion Models are Zero-Shot Segmentors
In this work, we explore how an off-the-shelf text-to-image diffusion model, trained without exposure to localization information, can ground various semantic phrases without segmentation-specific re-training.
Fine-Grained Visual Prompting
Previous works have suggested that incorporating visual prompts, such as colorful boxes or circles, can improve the ability of models to recognize objects of interest.
Answering Numerical Reasoning Questions in Table-Text Hybrid Contents with Graph-based Encoder and Tree-based Decoder
In the real-world question answering scenarios, hybrid form combining both tabular and textual contents has attracted more and more attention, among which numerical reasoning problem is one of the most typical and challenging problems.
MoStGAN-V: Video Generation with Temporal Motion Styles
Video generation remains a challenging task due to spatiotemporal complexity and the requirement of synthesizing diverse motions with temporal consistency.
Exploring Better Text Image Translation with Multimodal Codebook
Text image translation (TIT) aims to translate the source texts embedded in the image to target translations, which has a wide range of applications and thus has important research value.
AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Framework
The task of financial analysis primarily encompasses two key areas: stock trend prediction and the corresponding financial question answering.
ImpressionGPT: An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT
The 'Impression' section of a radiology report is a critical basis for communication between radiologists and other physicians, and it is typically written by radiologists based on the 'Findings' section.
ADNet: Lane Shape Prediction via Anchor Decomposition
In this paper, we revisit the limitations of anchor-based lane detection methods, which have predominantly focused on fixed anchors that stem from the edges of the image, disregarding their versatility and quality.
Benchmarking of DL Libraries and Models on Mobile Devices
Deploying deep learning (DL) on mobile devices has been a notable trend in recent years.
Limited Data, Unlimited Potential: A Study on ViTs Augmented by Masked Autoencoders
We explore the appropriate SSL tasks that can be optimized alongside the primary task, the training schemes for these tasks, and the data scale at which they can be most effective.
Arbicon-Net: Arbitrary Continuous Geometric Transformation Networks for Image Registration
To address this issue, we present an end-to-end trainable deep neural networks, named Arbitrary Continuous Geometric Transformation Networks (Arbicon-Net), to directly predict the dense displacement field for pairwise image alignment.
Heterogeneous Graph Contrastive Learning with Meta-path Contexts and Adaptively Weighted Negative Samples
In addition, we propose a variant model AdaMEOW that adaptively learns soft-valued weights of negative samples to further improve node representation.
Dual Intent Enhanced Graph Neural Network for Session-based New Item Recommendation
We design a dual-intent network to learn user intent from an attention mechanism and the distribution of historical data respectively, which can simulate users' decision-making process in interacting with a new item.
SAMAug: Point Prompt Augmentation for Segment Anything Model
Starting with an initial point prompt, SAM produces an initial mask, which is then fed into our proposed SAMAug to generate augmented point prompts.
Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems
In this paper, we propose a novel ensemble of retrieval-based and generation-based dialog systems in the open domain.
JointCL: A Joint Contrastive Learning Framework for Zero-Shot Stance Detection
In this paper, we propose a joint contrastive learning (JointCL) framework, which consists of stance contrastive learning and target-aware prototypical graph contrastive learning.
S$^3$HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering
In this paper, we propose a three-stage TextTableQA framework S3HQA, which comprises of retriever, selector, and reasoner.
The Image Local Autoregressive Transformer
To address these limitations, we propose a novel model -- image Local Autoregressive Transformer (iLAT), to better facilitate the locally guided image synthesis.
Modeling Users' Contextualized Page-wise Feedback for Click-Through Rate Prediction in E-commerce Search
In this paper, we propose a new perspective for context-aware users' behavior modeling by including the whole page-wisely exposed products and the corresponding feedback as contextualized page-wise feedback sequence.
Understanding Long Videos in One Multimodal Language Model Pass
In addition to faster inference, we discover the resulting models to yield surprisingly good accuracy on long-video tasks, even with no video specific information.
PC-GAIN: Pseudo-label Conditional Generative Adversarial Imputation Networks for Incomplete Data
Further, this classifier is incorporated into the generative adversarial framework to help the generator to yield higher quality imputation results.
LWSIS: LiDAR-guided Weakly Supervised Instance Segmentation for Autonomous Driving
In this paper, we present a more artful framework, LiDAR-guided Weakly Supervised Instance Segmentation (LWSIS), which leverages the off-the-shelf 3D data, i. e., Point Cloud, together with the 3D boxes, as natural weak supervisions for training the 2D image instance segmentation models.
Defense-VAE: A Fast and Accurate Defense against Adversarial Attacks
The proposed method is generic and can defend white-box and black-box attacks without the need of retraining the original CNN classifiers, and can further strengthen the defense by retraining CNN or end-to-end finetuning the whole pipeline.
An Orthogonal Classifier for Improving the Adversarial Robustness of Neural Networks
Neural networks are susceptible to artificially designed adversarial perturbations.
Ranked #1 on
Adversarial Attack
on CIFAR-10
Generative Dynamic Patch Attack
Extensive experiments on VGGFace, Traffic Sign and ImageNet show that GDPA achieves higher attack success rates than state-of-the-art patch attacks, while adversarially trained model with GDPA demonstrates superior robustness to adversarial patch attacks than competing methods.
SCARA: Scalable Graph Neural Networks with Feature-Oriented Optimization
Recent advances in data processing have stimulated the demand for learning graphs of very large scales.
CAT-probing: A Metric-based Approach to Interpret How Pre-trained Models for Programming Language Attend Code Structure
To interpret these models, some probing methods have been applied.
Knowledge Prompting in Pre-trained Language Model for Natural Language Understanding
In this paper, to address these problems, we introduce a seminal knowledge prompting paradigm and further propose a knowledge-prompting-based PLM framework KP-PLM.
Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration
Large Language Models (LLMs) are evolving at an unprecedented pace and have exhibited considerable capability in the realm of natural language processing (NLP) with world knowledge.
DialCoT Meets PPO: Decomposing and Exploring Reasoning Paths in Smaller Language Models
Chain-of-Thought (CoT) prompting has proven to be effective in enhancing the reasoning capabilities of Large Language Models (LLMs) with at least 100 billion parameters.
Soulstyler: Using Large Language Model to Guide Image Style Transfer for Target Object
We introduce a large language model to parse the text and identify stylization goals and specific styles.
A Concise yet Effective model for Non-Aligned Incomplete Multi-view and Missing Multi-label Learning
In aligning, we characterize the global and local structures of multiple labels to be high-rank and low-rank, respectively.
A Survey of Historical Learning: Learning Models with Learning History
The various types of elements, deposited in the training history, are a large amount of wealth for improving learning deep models.
A General Framework for Learning from Weak Supervision
Weakly supervised learning generally faces challenges in applicability to various scenarios with diverse weak supervision and in scalability due to the complexity of existing algorithms, thereby hindering the practical deployment.
Weakly Supervised Text Classification using Supervision Signals from a Language Model
In this paper, we propose to query a masked language model with cloze style prompts to obtain supervision signals.
Meta-Learning Siamese Network for Few-Shot Text Classification
Few-shot learning has been used to tackle the problem of label scarcity in text classification, of which meta-learning based methods have shown to be effective, such as the prototypical networks (PROTO).
3DCoMPaT$^{++}$: An improved Large-scale 3D Vision Dataset for Compositional Recognition
In this work, we present 3DCoMPaT$^{++}$, a multimodal 2D/3D dataset with 160 million rendered views of more than 10 million stylized 3D shapes carefully annotated at the part-instance level, alongside matching RGB point clouds, 3D textured meshes, depth maps, and segmentation masks.
Instance-level Heterogeneous Domain Adaptation for Limited-labeled Sketch-to-Photo Retrieval
Therefore, previous works pre-train their models on rich-labeled photo retrieval data (i. e., source domain) and then fine-tune them on the limited-labeled sketch-to-photo retrieval data (i. e., target domain).
Ranked #1 on
Image Retrieval
on PKU-Reid
Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives
Large language models (LLMs) have shown increasing power on various natural language processing (NLP) tasks.
Invisible Watermarking for Audio Generation Diffusion Models
Diffusion models have gained prominence in the image domain for their capabilities in data generation and transformation, achieving state-of-the-art performance in various tasks in both image and audio domains.
MediViSTA-SAM: Zero-shot Medical Video Analysis with Spatio-temporal SAM Adaptation for Echocardiography
The Segmentation Anything Model (SAM) has gained significant attention for its robust generalization capabilities across diverse downstream tasks.
Densely Connected Bidirectional LSTM with Applications to Sentence Classification
Deep neural networks have recently been shown to achieve highly competitive performance in many computer vision tasks due to their abilities of exploring in a much larger hypothesis space.
Automated Segmentation of Cervical Nuclei in Pap Smear Images using Deformable Multi-path Ensemble Model
Pap smear testing has been widely used for detecting cervical cancers based on the morphology properties of cell nuclei in microscopic image.
Structure-Enhanced Meta-Learning For Few-Shot Graph Classification
Graph classification is a highly impactful task that plays a crucial role in a myriad of real-world applications such as molecular property prediction and protein function prediction. Aiming to handle the new classes with limited labeled graphs, few-shot graph classification has become a bridge of existing graph classification solutions and practical usage. This work explores the potential of metric-based meta-learning for solving few-shot graph classification. We highlight the importance of considering structural characteristics in the solution and propose a novel framework which explicitly considers global structure and local structure of the input graph.
PaintSeg: Training-free Segmentation via Painting
The paper introduces PaintSeg, a new unsupervised method for segmenting objects without any training.
The Radiation Oncology NLP Database
ROND is specifically designed to address this gap in the domain of radiation oncology, a field that offers many opportunities for NLP exploration.
GradMA: A Gradient-Memory-based Accelerated Federated Learning with Alleviated Catastrophic Forgetting
Federated Learning (FL) has emerged as a de facto machine learning area and received rapid increasing research interests from the community.
Creative Birds: Self-Supervised Single-View 3D Style Transfer
Our focus lies primarily on birds, a popular subject in 3D reconstruction, for which no existing single-view 3D transfer methods have been developed. The method we propose seeks to generate a 3D mesh shape and texture of a bird from two single-view images.
FwdLLM: Efficient FedLLM using Forward Gradient
Federated Learning (FL), a method to preserve user data privacy, is often employed in fine-tuning LLMs to downstream mobile tasks, an approach known as FedLLM.
CGMN: A Contrastive Graph Matching Network for Self-Supervised Graph Similarity Learning
As most of the existing graph neural networks yield effective graph representations of a single graph, little effort has been made for jointly learning two graph representations and calculating their similarity score.
Modeling Dual Period-Varying Preferences for Takeaway Recommendation
Different from traditional recommendation, takeaway recommendation faces two main challenges: (1) Dual Interaction-Aware Preference Modeling.
Exploring Multimodal Approaches for Alzheimer's Disease Detection Using Patient Speech Transcript and Audio Data
As AD impairs the patient's language understanding and expression ability, the speech of AD patients can serve as an indicator of this disease.
You Can Backdoor Personalized Federated Learning
The resistance of pFL methods with parameter decoupling is attributed to the heterogeneous classifiers between malicious clients and benign counterparts.
Video State-Changing Object Segmentation
Learning about object state changes in Video Object Segmentation (VOS) is crucial for understanding and interacting with the visual world.
VA3: Virtually Assured Amplification Attack on Probabilistic Copyright Protection for Text-to-Image Generative Models
The booming use of text-to-image generative models has raised concerns about their high risk of producing copyright-infringing content.
NPHardEval4V: A Dynamic Reasoning Benchmark of Multimodal Large Language Models
Understanding the reasoning capabilities of Multimodal Large Language Models (MLLMs) is an important area of research.
FGCRec: Fine-Grained Geographical Characteristics Modeling for Point-of-Interest Recommendation
With the popularity of location-based social networks (LBSNs), Point-of-Interest (POI) recommendation has become an essential location-based service to help people explore novel locations.
Lexical Knowledge Internalization for Neural Dialog Generation
We propose knowledge internalization (KI), which aims to complement the lexical knowledge into neural dialog models.
A Neural Network Architecture for Program Understanding Inspired by Human Behaviors
In this paper, we consider human behaviors and propose the PGNN-EK model that consists of two main components.
Towards Robust Neural Machine Translation with Iterative Scheduled Data-Switch Training
Most existing methods on robust neural machine translation (NMT) construct adversarial examples by injecting noise into authentic examples and indiscriminately exploit two types of examples.
Rethinking the Reasonability of the Test Set for Simultaneous Machine Translation
Simultaneous machine translation (SimulMT) models start translation before the end of the source sentence, making the translation monotonically aligned with the source sentence.
FGRec: A Fine-Grained Point-of-Interest Recommendation Framework by Capturing Intrinsic Influences
Recent studies mainly utilize social information, categorical information and/or geographical information to supplement the highly sparse check-in data.
ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphs
To tackle these limitations, we propose a novel framework that leverages the power of ChatGPT for specific tasks, such as text classification, while improving its interpretability.
Improving Zero-shot Visual Question Answering via Large Language Models with Reasoning Question Prompts
This results in a set of candidate answers.
On better training the infinite restricted Boltzmann machines
To break this dependency between neighboring hidden units and speed up the convergence of training, a novel training strategy is proposed.
Neural Image Compression and Explanation
Explaining the prediction of deep neural networks (DNNs) and semantic image compression are two active research areas of deep learning with a numerous of applications in decision-critical systems, such as surveillance cameras, drones and self-driving cars, where interpretable decision is critical and storage/network bandwidth is limited.
Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More
We show that our Generative MMC (GMMC) can be trained discriminatively, generatively, or jointly for image classification and generation.
Interpretable Multi-hop Reasoning for Forecasting Future Links on Temporal Knowledge Graphs
The interpretability of the current temporal KG forecasting models is manifested in providing the reasoning paths.
Next Check-ins Prediction via History and Friendship on Location-Based Social Networks
In this paper, we propose a unified location prediction framework to integrate the effect of history check-in and the influence of social circles.
Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?
We investigate whether self-supervised learning (SSL) can improve online reinforcement learning (RL) from pixels.
CofeNet: Context and Former-Label Enhanced Net for Complicated Quotation Extraction
CofeNet is able to extract complicated quotations with components of variable lengths and complicated structures.
On de novo Bridging Paired-end RNA-seq Data
Methods have been proposed to bridge paired-end reads in the presence of reference genome (called reference-based bridging), but the algorithms are far away from scaling for de novo bridging as the underlying compacted de Bruijn graph(cdBG) used in the latter task often contains millions of vertices and edges.
The Hidden Dance of Phonemes and Visage: Unveiling the Enigmatic Link between Phonemes and Facial Features
This work unveils the enigmatic link between phonemes and facial features.
Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift
Theoretically, we find that Dropout would shift the variance of a specific neural unit when we transfer the state of that network from train to test.
Representing Joint Hierarchies with Box Embeddings
Box Embeddings [Vilnis et al., 2018, Li et al., 2019] represent concepts with hyperrectangles in $n$-dimensional space and are shown to be capable of modeling tree-like structures efficiently by training on a large subset of the transitive closure of the WordNet hypernym graph.
Collaborative Reflection-Augmented Autoencoder Network for Recommender Systems
As the deep learning techniques have expanded to real-world recommendation tasks, many deep neural network based Collaborative Filtering (CF) models have been developed to project user-item interactions into latent feature space, based on various neural architectures, such as multi-layer perceptron, auto-encoder and graph neural networks.
Asca: less audio data is more insightful
Audio recognition in specialized areas such as birdsong and submarine acoustics faces challenges in large-scale pre-training due to the limitations in available samples imposed by sampling environments and specificity requirements.
Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding
The recent success of large pre-trained language models (PLMs) heavily hinges on massive labeled data, which typically produces inferior performance in low-resource scenarios.
The NeurIPS 2022 Neural MMO Challenge: A Massively Multiagent Competition with Specialization and Trade
In this paper, we present the results of the NeurIPS-2022 Neural MMO Challenge, which attracted 500 participants and received over 1, 600 submissions.
TreeEval: Benchmark-Free Evaluation of Large Language Models through Tree Planning
Recently, numerous new benchmarks have been established to evaluate the performance of large language models (LLMs) via either computing a holistic score or employing another LLM as a judge.
Eye-gaze Guided Multi-modal Alignment Framework for Radiology
Additionally, we explore the impact of varying amounts of eye-gaze data on model performance, highlighting the feasibility and utility of integrating this auxiliary data into multi-modal pre-training.
A Statistical Analysis of Polyak-Ruppert Averaged Q-learning
We study Q-learning with Polyak-Ruppert averaging in a discounted Markov decision process in synchronous and tabular settings.
Mobile Foundation Model as Firmware
Concurrently, each app contributes a concise, offline fine-tuned "adapter" tailored to distinct downstream tasks.
Context-aware Session-based Recommendation with Graph Neural Networks
Session-based recommendation (SBR) is a task that aims to predict items based on anonymous sequences of user behaviors in a session.
Federated Learning via Consensus Mechanism on Heterogeneous Data: A New Perspective on Convergence
We theoretically show that the consensus mechanism can guarantee the convergence of the global objective.
Modeling 4D fMRI Data via Spatio-Temporal Convolutional Neural Networks (ST-CNN)
Simultaneous modeling of the spatio-temporal variation patterns of brain functional network from 4D fMRI data has been an important yet challenging problem for the field of cognitive neuroscience and medical image analysis.
Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Tasks
In this work, we propose to learn universal user representations across multiple tasks for more e ective personalization.
Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures
Embedding methods which enforce a partial order or lattice structure over the concept space, such as Order Embeddings (OE) (Vendrov et al., 2016), are a natural way to model transitive relational data (e. g. entailment graphs).
Adversarial Metric Learning
In distinguishment stage, a metric is exhaustively learned to try its best to distinguish both the adversarial pairs and the original training pairs.
Unsupervised Multi-Domain Image Translation with Domain-Specific Encoders/Decoders
Unsupervised Image-to-Image Translation achieves spectacularly advanced developments nowadays.
Medical Image Segmentation Based on Multi-Modal Convolutional Neural Network: Study on Image Fusion Schemes
Image analysis using more than one modality (i. e. multi-modal) has been increasingly applied in the field of biomedical imaging.
Image Segmentation and Classification for Sickle Cell Disease using Deformable U-Net
Reliable cell segmentation and classification from biomedical images is a crucial step for both scientific research and clinical practice.
Improved Representation Learning for Predicting Commonsense Ontologies
Recent work in learning ontologies (hierarchical and partially-ordered structures) has leveraged the intrinsic geometry of spaces of learned representations to make predictions that automatically obey complex structural constraints.
Developing Knowledge-enhanced Chronic Disease Risk Prediction Models from Regional EHR Repositories
Precision medicine requires the precision disease risk prediction models.
Dictionary Learning and Sparse Coding-based Denoising for High-Resolution Task Functional Connectivity MRI Analysis
In order to address the limitations of the unsupervised DLSC-based fMRI studies, we utilize the prior knowledge of task paradigm in the learning step to train a data-driven dictionary and to model the sparse representation.
Self-paced Convolutional Neural Network for Computer Aided Detection in Medical Imaging Analysis
However, the development of a robust and reliable deep learning model for computer-aided diagnosis is still highly challenging due to the combination of the high heterogeneity in the medical images and the relative lack of training samples.
Learning the Sparse and Low Rank PARAFAC Decomposition via the Elastic Net
In this article, we derive a Bayesian model to learning the sparse and low rank PARAFAC decomposition for the observed tensor with missing values via the elastic net, with property to find the true rank and sparse factor matrix which is robust to the noise.
LightRNN: Memory and Computation-Efficient Recurrent Neural Networks
Based on the 2-Component shared embedding, we design a new RNN algorithm and evaluate it using the language modeling task on several benchmark datasets.
Statistical Properties of the Single Linkage Hierarchical Clustering Estimator
Distance-based hierarchical clustering (HC) methods are widely used in unsupervised data analysis but few authors take account of uncertainty in the distance data.
Top-push Video-based Person Re-identification
Since only limited information can be exploited from still images, it is hard (if not impossible) to overcome the occlusion, pose and camera-view change, and lighting variation problems.
An Enhanced Deep Feature Representation for Person Re-identification
In this paper, we focus on the feature representation and claim that hand-crafted histogram features can be complementary to Convolutional Neural Network (CNN) features.
StalemateBreaker: A Proactive Content-Introducing Approach to Automatic Human-Computer Conversation
In this paper, we propose StalemateBreaker, a conversation system that can proactively introduce new content when appropriate.
Maximum Likelihood Estimation for Single Linkage Hierarchical Clustering
We derive a statistical model for estimation of a dendrogram from single linkage hierarchical clustering (SLHC) that takes account of uncertainty through noise or corruption in the measurements of separation of data.
Task-group Relatedness and Generalization Bounds for Regularized Multi-task Learning
We are mainly concerned with two theoretical questions: 1) under what conditions does RMTL perform better with a smaller task sample size than STL?
Adversarial Open-World Person Re-Identification
The main idea is learning to attack feature extractor on the target people by using GAN to generate very target-like images (imposters), and in the meantime the model will make the feature extractor learn to tolerate the attack by discriminative learning so as to realize group-based verification.
Multi-Estimator Full Left Ventricle Quantification through Ensemble Learning
The framework combines two 1st-level modules: direct estimation module and a segmentation module.
Network Modeling and Pathway Inference from Incomplete Data ("PathInf")
In this work, we developed a network inference method from incomplete data ("PathInf") , as massive and non-uniformly distributed missing values is a common challenge in practical problems.
Triple Attention Mixed Link Network for Single Image Super Resolution
Single image super resolution is of great importance as a low-level computer vision task.
Deep Diabetologist: Learning to Prescribe Hyperglycemia Medications with Hierarchical Recurrent Neural Networks
In healthcare, applying deep learning models to electronic health records (EHRs) has drawn considerable attention.
Improving the Robustness of Speech Translation
Although neural machine translation (NMT) has achieved impressive progress recently, it is usually trained on the clean parallel data set and hence cannot work well when the input sentence is the production of the automatic speech recognition (ASR) system due to the enormous errors in the source.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Group-Attention Single-Shot Detector (GA-SSD): Finding Pulmonary Nodules in Large-Scale CT Images
In this paper, we propose a novel and effective abnormality detector implementing the attention mechanism and group convolution on 3D single-shot detector (SSD) called group-attention SSD (GA-SSD).
Computed Tomography (CT)
Finding Pulmonary Nodules In Large-Scale Ct Images
Joint Task-Recursive Learning for Semantic Segmentation and Depth Estimation
In this paper, we propose a novel joint Task-Recursive Learning (TRL) framework for the closing-loop semantic segmentation and monocular depth estimation tasks.
Ranked #76 on
Semantic Segmentation
on NYU Depth v2
Smoothing the Geometry of Probabilistic Box Embeddings
However, the hard edges of the boxes present difficulties for standard gradient based optimization; that work employed a special surrogate function for the disjoint case, but we find this method to be fragile.
