Search Results for author:
Found 137 papers, 63 papers with code
PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Literature Parsing Task B: Table Recognition to HTML
In our method, we divide the table content recognition task into foursub-tasks: table structure recognition, text line detection, text line recognition, and box assignment. Our table structure recognition algorithm is customized based on MASTER [1], a robust image textrecognition algorithm.
Ranked #1 on
Table Recognition
on PubTabNet
A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise
They endow Large Language Models (LLMs) with powerful capabilities in visual understanding, enabling them to tackle diverse multi-modal tasks.
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
We present LLaMA-Adapter, a lightweight adaption method to efficiently fine-tune LLaMA into an instruction-following model.
Ranked #2 on
Music Question Answering
on MusicQA
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
This strategy effectively alleviates the interference between the two tasks of image-text alignment and instruction following and achieves strong multi-modal reasoning with only a small-scale image-text and instruction dataset.
Ranked #6 on
Visual Question Answering (VQA)
on InfiMM-Eval
Instruction Following
Optical Character Recognition (OCR)
+7
ImageBind-LLM: Multi-modality Instruction Tuning
During training, we adopt a learnable bind network to align the embedding space between LLaMA and ImageBind's image encoder.
UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning
For Something-Something V1 and V2, our UniFormer achieves new state-of-the-art performances of 60. 9% and 71. 2% top-1 accuracy respectively.
SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models
We present SPHINX, a versatile multi-modal large language model (MLLM) with a joint mixing of model weights, tuning tasks, and visual embeddings.
Ranked #2 on
Visual Question Answering
on BenchLMM
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models
We propose SPHINX-X, an extensive Multimodality Large Language Model (MLLM) series developed upon SPHINX.
Ranked #4 on
Video Question Answering
on MVBench
Personalize Segment Anything Model with One Shot
Driven by large-data pre-training, Segment Anything Model (SAM) has been demonstrated as a powerful and promptable framework, revolutionizing the segmentation models.
Ranked #1 on
Personalized Segmentation
on PerSeg
UniFormer: Unifying Convolution and Self-attention for Visual Recognition
Different from the typical transformer blocks, the relation aggregators in our UniFormer block are equipped with local and global token affinity respectively in shallow and deep layers, allowing to tackle both redundancy and dependency for efficient and effective representation learning.
Ranked #151 on
Image Classification
on ImageNet
SUG: Single-dataset Unified Generalization for 3D Point Cloud Classification
In this paper, different from previous 2D DG works, we focus on the 3D DG problem and propose a Single-dataset Unified Generalization (SUG) framework that only leverages a single source dataset to alleviate the unforeseen domain differences faced by a well-trained source model.
OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
LWC modulates the extreme values of weights by optimizing the clipping threshold.
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling
To further enhance CLIP's few-shot capability, CLIP-Adapter proposed to fine-tune a lightweight residual feature adapter and significantly improves the performance for few-shot classification.
Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification
On top of that, the performance of Tip-Adapter can be further boosted to be state-of-the-art on ImageNet by fine-tuning the cache model for 10$\times$ fewer epochs than existing methods, which is both effective and efficient.
Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learners
Our CaFo incorporates CLIP's language-contrastive knowledge, DINO's vision-contrastive knowledge, DALL-E's vision-generative knowledge, and GPT-3's language-generative knowledge.
ConvMAE: Masked Convolution Meets Masked Autoencoders
Masked auto-encoding for feature pretraining and multi-scale hybrid convolution-transformer architectures can further unleash the potentials of ViT, leading to state-of-the-art performances on image classification, detection and semantic segmentation.
Mimic before Reconstruct: Enhancing Masked Autoencoders with Feature Mimicking
To alleviate this, previous methods simply replace the pixel reconstruction targets of 75% masked tokens by encoded features from pre-trained image-image (DINO) or image-language (CLIP) contrastive learning.
OneLLM: One Framework to Align All Modalities with Language
In detail, we first train an image projection module to connect a vision encoder with LLM.
Ranked #73 on
Visual Question Answering
on MM-Vet
Parameter is Not All You Need: Starting from Non-Parametric Networks for 3D Point Cloud Analysis
We present a Non-parametric Network for 3D point cloud analysis, Point-NN, which consists of purely non-learnable components: farthest point sampling (FPS), k-nearest neighbors (k-NN), and pooling operations, with trigonometric functions.
Ranked #1 on
Training-free 3D Part Segmentation
on ShapeNet-Part
3D Point Cloud Classification
Training-free 3D Part Segmentation
+1
Starting From Non-Parametric Networks for 3D Point Cloud Analysis
We present a Non-parametric Network for 3D point cloud analysis, Point-NN, which consists of purely non-learnable components: farthest point sampling (FPS), k-nearest neighbors (k-NN), and pooling operations, with trigonometric functions.
No Time to Train: Empowering Non-Parametric Networks for Few-shot 3D Scene Segmentation
To reduce the reliance on large-scale datasets, recent works in 3D segmentation resort to few-shot learning.
You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure Correction
Challenging illumination conditions (low-light, under-exposure and over-exposure) in the real world not only cast an unpleasant visual appearance but also taint the computer vision tasks.
Ranked #2 on
Image Enhancement
on Exposure-Errors
CLIP-Adapter: Better Vision-Language Models with Feature Adapters
Large-scale contrastive vision-language pre-training has shown significant progress in visual representation learning.
Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following
We introduce Point-Bind, a 3D multi-modality model aligning point clouds with 2D image, language, audio, and video.
Ranked #5 on
3D Question Answering (3D-QA)
on 3D MM-Vet
LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models
Large Vision-Language Models (LVLMs) have recently played a dominant role in multimodal vision-language learning.
Tiny LVLM-eHub: Early Multimodal Experiments with Bard
Secondly, it conducts an in-depth analysis of LVLMs' predictions using the ChatGPT Ensemble Evaluation (CEE), which leads to a robust and accurate evaluation and exhibits improved alignment with human evaluation compared to the word matching approach.
MonoDETR: Depth-guided Transformer for Monocular 3D Object Detection
In this paper, we introduce the first DETR framework for Monocular DEtection with a depth-guided TRansformer, named MonoDETR.
PointCLIP: Point Cloud Understanding by CLIP
On top of that, we design an inter-view adapter to better extract the global feature and adaptively fuse the few-shot knowledge learned from 3D into CLIP pre-trained in 2D.
Ranked #3 on
3D Open-Vocabulary Instance Segmentation
on STPLS3D
3D Open-Vocabulary Instance Segmentation
Few-Shot Learning
+6
PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learning
In this paper, we first collaborate CLIP and GPT to be a unified 3D open-world learner, named as PointCLIP V2, which fully unleashes their potential for zero-shot 3D classification, segmentation, and detection.
Ranked #2 on
3D Open-Vocabulary Instance Segmentation
on STPLS3D
Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model
This paper presents Instruct2Act, a framework that utilizes Large Language Models to map multi-modal instructions to sequential actions for robotic manipulation tasks.
Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training
By fine-tuning on downstream tasks, Point-M2AE achieves 86. 43% accuracy on ScanObjectNN, +3. 36% to the second-best, and largely benefits the few-shot classification, part segmentation and 3D object detection with the hierarchical pre-training scheme.
Ranked #4 on
3D Point Cloud Linear Classification
on ModelNet40
(using extra training data)
Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoders
Pre-training by numerous image data has become de-facto for robust 2D representations.
Ranked #2 on
3D Point Cloud Linear Classification
on ModelNet40
(using extra training data)
3D Point Cloud Linear Classification
Few-Shot 3D Point Cloud Classification
End-to-End Object Detection with Adaptive Clustering Transformer
In this paper, a novel variant of transformer named Adaptive Clustering Transformer(ACT) has been proposed to reduce the computation cost for high-resolution input.
Fast Convergence of DETR with Spatially Modulated Co-Attention
The recently proposed Detection Transformer (DETR) model successfully applies Transformer to objects detection and achieves comparable performance with two-stage object detection frameworks, such as Faster-RCNN.
Fast Convergence of DETR with Spatially Modulated Co-Attention
However, DETR suffers from its slow convergence.
SCOPE: Scalable Composite Optimization for Learning on Spark
Recently, many distributed stochastic optimization~(DSO) methods have been proposed to solve the large-scale composite optimization problems, which have shown better performance than traditional batch methods.
Frozen CLIP Models are Efficient Video Learners
Video recognition has been dominated by the end-to-end learning paradigm -- first initializing a video recognition model with weights of a pretrained image model and then conducting end-to-end training on videos.
Ranked #26 on
Action Classification
on Kinetics-400
(using extra training data)
Multi-View Partial (MVP) Point Cloud Challenge 2021 on Completion and Registration: Methods and Results
Based on the MVP dataset, this paper reports methods and results in the Multi-View Partial Point Cloud Challenge 2021 on Completion and Registration.
Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement
The popularity of Contrastive Language-Image Pre-training (CLIP) has propelled its application to diverse downstream vision tasks.
Learning Where to Focus for Efficient Video Object Detection
Transferring existing image-based detectors to the video is non-trivial since the quality of frames is always deteriorated by part occlusion, rare pose, and motion blur.
Ranked #22 on
Video Object Detection
on ImageNet VID
Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer
The large pre-trained vision transformers (ViTs) have demonstrated remarkable performance on various visual tasks, but suffer from expensive computational and memory cost problems when deployed on resource-constrained devices.
A Simple Long-Tailed Recognition Baseline via Vision-Language Model
Recent advances in large-scale contrastive visual-language pretraining shed light on a new pathway for visual recognition.
Ranked #4 on
Long-tail Learning
on Places-LT
(using extra training data)
Container: Context Aggregation Network
Convolutional neural networks (CNNs) are ubiquitous in computer vision, with a myriad of effective and efficient variations.
Ranked #462 on
Image Classification
on ImageNet
Container: Context Aggregation Networks
Convolutional neural networks (CNNs) are ubiquitous in computer vision, with a myriad of effective and efficient variations.
ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning
Charts play a vital role in data visualization, understanding data patterns, and informed decision-making.
Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation
In this paper, we propose MUTR, a Multi-modal Unified Temporal transformer for Referring video object segmentation.
Prototypical Contrast Adaptation for Domain Adaptive Semantic Segmentation
Unsupervised Domain Adaptation (UDA) aims to adapt the model trained on the labeled source domain to an unlabeled target domain.
Ranked #14 on
Unsupervised Domain Adaptation
on SYNTHIA-to-Cityscapes
Less is More: Towards Efficient Few-shot 3D Semantic Segmentation via Training-free Networks
However, the prior pre-training stage not only introduces excessive time overhead, but also incurs a significant domain gap on `unseen' classes.
3D Semantic Segmentation
Few-shot 3D semantic segmentation
+1
An effective self-supervised framework for learning expressive molecular global representations to drug discovery
In MPG, we proposed a powerful GNN for modelling molecular graph named MolGNet, and designed an effective self-supervised strategy for pre-training the model at both the node and graph-level.
Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want
In this paper, we introduce the Draw-and-Understand project: a new model, a multi-domain dataset, and a challenging benchmark for visual prompting.
Q-DETR: An Efficient Low-Bit Quantized Detection Transformer
At the upper level, we introduce a new foreground-aware query matching scheme to effectively transfer the teacher information to distillation-desired features to minimize the conditional information entropy.
FeatAug-DETR: Enriching One-to-Many Matching for DETRs with Feature Augmentation
The first method is One-to-many Matching via Data Augmentation (denoted as DataAug-DETR).
RomeBERT: Robust Training of Multi-Exit BERT
In this paper, we leverage gradient regularized self-distillation for RObust training of Multi-Exit BERT (RomeBERT), which can effectively solve the performance imbalance problem between early and late exits.
POS-BERT: Point Cloud One-Stage BERT Pre-Training
We propose to use the dynamically updated momentum encoder as the tokenizer, which is updated and outputs the dynamic supervision signal along with the training process.
Recurrent Bilinear Optimization for Binary Neural Networks
To address this issue, Recurrent Bilinear Optimization is proposed to improve the learning process of BNNs (RBONNs) by associating the intrinsic bilinear variables in the back propagation process.
M$^{2}$Chat: Empowering VLM for Multimodal LLM Interleaved Text-Image Generation
Moreover, to further enhance the effectiveness of $M^{3}Adapter$ while preserving the coherence of semantic context comprehension, we introduce a two-stage $M^{3}FT$ fine-tuning strategy.
IDa-Det: An Information Discrepancy-aware Distillation for 1-bit Detectors
This explains why existing KD methods are less effective for 1-bit detectors, caused by a significant information discrepancy between the real-valued teacher and the 1-bit student.
Consecutive Pretraining: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domain
Currently, under supervised learning, a model pretrained by a large-scale nature scene dataset and then fine-tuned on a few specific task labeling data is the paradigm that has dominated the knowledge transfer learning.
Resilient Binary Neural Network
In this paper, we introduce a Resilient Binary Neural Network (ReBNN) to mitigate the frequent oscillation for better BNNs' training.
BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation
Large language models (LLMs) have demonstrated outstanding performance in various tasks, such as text summarization, text question-answering, and etc.
SAQL: A Stream-based Query System for Real-Time Abnormal System Behavior Detection
To facilitate the task of expressing anomalies based on expert knowledge, our system provides a domain-specific query language, SAQL, which allows analysts to express models for (1) rule-based anomalies, (2) time-series anomalies, (3) invariant-based anomalies, and (4) outlier-based anomalies.
Cryptography and Security Databases
Masked AutoDecoder is Effective Multi-Task Vision Generalist
Inspired by the success of general-purpose models in NLP, recent studies attempt to unify different vision tasks in the same sequence format and employ autoregressive Transformers for sequence prediction.
Enabling Efficient Cyber Threat Hunting With Cyber Threat Intelligence
Log-based cyber threat hunting has emerged as an important solution to counter sophisticated attacks.
A Complementary Tracking Model with Multiple Features
Discriminative Correlation Filters based tracking algorithms exploiting conventional handcrafted features have achieved impressive results both in terms of accuracy and robustness.
High Performance Visual Tracking with Circular and Structural Operators
Based on the proposed circular and structural operators, a set of primal confidence score maps can be obtained by circular correlating feature maps with their corresponding structural correlation filters.
A Novel Low-cost FPGA-based Real-time Object Tracking System
In current visual object tracking system, the CPU or GPU-based visual object tracking systems have high computational cost and consume a prohibitive amount of power.
A Novel Parallel Ray-Casting Algorithm
The Ray-Casting algorithm is an important method for fast real-time surface display from 3D medical images.
Large Margin Structured Convolution Operator for Thermal Infrared Object Tracking
To the best of our knowledge, we are the first to incorporate the advantages of DCF and SOSVM for TIR object tracking.
Question-Guided Hybrid Convolution for Visual Question Answering
Most state-of-the-art VQA methods fuse the high-level textual and visual features from the neural network and abandon the visual spatial information when learning multi-modal features. To address these problems, question-guided kernels generated from the input question are designed to convolute with visual features for capturing the textual and visual relationship in the early stage.
Ranked #14 on
Visual Question Answering (VQA)
on CLEVR
FPGA-based Acceleration System for Visual Tracking
In order to improve the tracking speed and reduce the overall power consumption of visual tracking, this paper proposes a real-time visual tracking algorithm based on DSST(Discriminative Scale Space Tracking) approach.
Efficient Multi-level Correlating for Visual Tracking
Correlation filter (CF) based tracking algorithms have demonstrated favorable performance recently.
Siamese Attentional Keypoint Network for High Performance Visual Tracking
In this paper, we investigate the impacts of three main aspects of visual tracking, i. e., the backbone network, the attentional mechanism, and the detection component, and propose a Siamese Attentional Keypoint Network, dubbed SATIN, for efficient tracking and accurate localization.
Learning Cascaded Siamese Networks for High Performance Visual Tracking
In order to achieve high performance visual tracking in various negative scenarios, a novel cascaded Siamese network is proposed and developed based on two different deep learning networks: a matching subnetwork and a classification subnetwork.
FPGA-based Binocular Image Feature Extraction and Matching System
Image feature extraction and matching is a fundamental but computation intensive task in machine vision.
Dynamic Fusion With Intra- and Inter-Modality Attention Flow for Visual Question Answering
It can robustly capture the high-level interactions between language and vision domains, thus significantly improves the performance of visual question answering.
Joint Multi-frame Detection and Segmentation for Multi-cell Tracking
Another UNet is utilized to acquire primary segmentation.
Multi-modality Latent Interaction Network for Visual Question Answering
The proposed module learns the cross-modality relationships between latent visual and language summarizations, which summarize visual regions and question into a small number of latent representations to avoid modeling uninformative individual region-word relations.
Learning Reinforced Attentional Representation for End-to-End Visual Tracking
Although numerous recent tracking approaches have made tremendous advances in the last decade, achieving high-performance visual tracking remains a challenge.
Research on Autonomous Maneuvering Decision of UCAV based on Approximate Dynamic Programming
To solve the problem of dimensional explosion in the air combat, the proposed method is implemented through feature selection, trajectory sampling, function approximation and Bellman backup operation in the air combat simulation environment.
Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks
We have experimented both (a) improving the fine-tuning of pre-trained language models on a task with a small dataset size, by leveraging datasets of similar tasks; and (b) incorporating the distributional representations of a KG onto the representations of pre-trained language models, via simply concatenation or multi-head attention.
Ranked #17 on
Common Sense Reasoning
on ReCoRD
Multi-Layer Content Interaction Through Quaternion Product For Visual Question Answering
To solve the issue for the intermediate layers, we propose an efficient Quaternion Block Network (QBN) to learn interaction not only for the last layer but also for all intermediate layers simultaneously.
Extreme Low-Light Imaging with Multi-granulation Cooperative Networks
Low-light imaging is challenging since images may appear to be dark and noised due to low signal-to-noise ratio, complex image content, and the variety in shooting scenes in extreme low-light condition.
Character Matters: Video Story Understanding with Character-Aware Relations
Without identifying the connection between appearing people and character names, a model is not able to obtain a genuine understanding of the plots.
Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers
Given an input video, its associated audio, and a brief caption, the audio-visual scene aware dialog (AVSD) task requires an agent to indulge in a question-answer dialog with a human about the audio-visual content.
Reconstruction Regularized Deep Metric Learning for Multi-label Image Classification
In this paper, we present a novel deep metric learning method to tackle the multi-label image classification problem.
Contrastive Visual-Linguistic Pretraining
We evaluate CVLP on several down-stream tasks, including VQA, GQA and NLVR2 to validate the superiority of contrastive learning on multi-modality representation learning.
Gradient Regularized Contrastive Learning for Continual Domain Adaptation
At the core of our method, gradient regularization plays two key roles: (1) enforces the gradient of contrastive loss not to increase the supervised training loss on the source domain, which maintains the discriminative power of learned features; (2) regularizes the gradient update on the new domain not to increase the classification loss on the old target domains, which enables the model to adapt to an in-coming target domain while preserving the performance of previously observed domains.
Multi-Pass Transformer for Machine Translation
In contrast with previous approaches where information flows only towards deeper layers of a stack, we consider a multi-pass transformer (MPT) architecture in which earlier layers are allowed to process information in light of the output of later layers.
A Predictive Autoscaler for Elastic Batch Jobs
Large batch jobs such as Deep Learning, HPC and Spark require far more computational resources and higher cost than conventional online service.
Multi-view Sensor Fusion by Integrating Model-based Estimation and Graph Learning for Collaborative Object Localization
Collaborative object localization aims to collaboratively estimate locations of objects observed from multiple views or perspectives, which is a critical ability for multi-agent systems such as connected vehicles.
Pre-training Entity Relation Encoder with Intra-span and Inter-span Information
In particular, a span encoder is trained to recover a random shuffling of tokens in a span, and a span pair encoder is trained to predict positive pairs that are from the same sentences and negative pairs that are from different sentences using contrastive loss.
Semi-supervised Active Learning for Instance Segmentation via Scoring Predictions
Active learning generally involves querying the most representative samples for human labeling, which has been widely studied in many fields such as image classification and object detection.
Learn molecular representations from large-scale unlabeled molecules for drug discovery
Here, we proposed a novel Molecular Pre-training Graph-based deep learning framework, named MPG, that leans molecular representations from large-scale unlabeled molecules.
A System for Efficiently Hunting for Cyber Threats in Computer Systems Using Threat Intelligence
Log-based cyber threat hunting has emerged as an important solution to counter sophisticated cyber attacks.
A System for Automated Open-Source Threat Intelligence Gathering and Management
SecurityKG collects OSCTI reports from various sources, uses a combination of AI and NLP techniques to extract high-fidelity knowledge about threat behaviors, and constructs a security knowledge graph.
Sharp upper bounds for moments of quadratic Dirichlet $L$-functions
We establish unconditional sharp upper bounds of the $k$-th moments of the family of quadratic Dirichlet $L$-functions at the central point for $0 \leq k \leq 2$.
Number Theory
Self-supervised learning for fast and scalable time series hyper-parameter tuning
Hyper-parameters of time series models play an important role in time series analysis.
PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Table Image Recognition to Latex
This paper presents our solution for the ICDAR 2021 Competition on Scientific Table Image Recognition to LaTeX.
Dual-stream Network for Visual Recognition
Transformers with remarkable global representation capacities achieve competitive results for visual tasks, but fail to consider high-level local pattern information in input images.
Scalable Transformers for Neural Machine Translation
In this paper, we propose a novel Scalable Transformers, which naturally contains sub-Transformers of different scales and have shared parameters.
Oriented Object Detection with Transformer
Object detection with Transformers (DETR) has achieved a competitive performance over traditional detectors, such as Faster R-CNN.
Winner Team Mia at TextVQA Challenge 2021: Vision-and-Language Representation Learning with Pre-trained Sequence-to-Sequence Model
TextVQA requires models to read and reason about text in images to answer questions about them.
ClueReader: Heterogeneous Graph Attention Network for Multi-hop Machine Reading Comprehension
Multi-hop machine reading comprehension is a challenging task in natural language processing as it requires more reasoning ability across multiple documents.
Dense Contrastive Visual-Linguistic Pretraining
To overcome these issues, we propose unbiased Dense Contrastive Visual-Linguistic Pretraining (DCVLP), which replaces the region regression and classification with cross-modality region contrastive learning that requires no annotations.
Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning
In previous work, we have proposed the Audio-Visual Scene-Aware Dialog (AVSD) task, collected an AVSD dataset, developed AVSD technologies, and hosted an AVSD challenge track at both the 7th and 8th Dialog System Technology Challenges (DSTC7, DSTC8).
Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks
Self-supervised learning has gradually emerged as a powerful technique for graph representation learning.
Asynchronous Collaborative Localization by Integrating Spatiotemporal Graph Learning with Model-Based Estimation
In this paper, we introduce a novel approach that integrates uncertainty-aware spatiotemporal graph learning and model-based state estimation for a team of robots to collaboratively localize objects.
Superpixel-Based Building Damage Detection from Post-earthquake Very High Resolution Imagery Using Deep Neural Networks
This paper presents a novel superpixel based approach combining DNN and a modified segmentation method, to detect damaged buildings from VHR imagery.
RestoreDet: Degradation Equivariant Representation for Object Detection in Low Resolution Images
Image restoration algorithms such as super resolution (SR) are indispensable pre-processing modules for object detection in degraded images.
TerViT: An Efficient Ternary Vision Transformer
Vision transformers (ViTs) have demonstrated great potential in various visual tasks, but suffer from expensive computational and memory cost problems when deployed on resource-constrained devices.
Distillation with Contrast is All You Need for Self-Supervised Point Cloud Representation Learning
Especially, we develop a variant of ViT for 3D point cloud feature extraction, which also achieves comparable results with existing backbones when combined with our framework, and visualization of the attention maps show that our model does understand the point cloud by combining the global shape information and multiple local structural information, which is consistent with the inspiration of our representation learning method.
CandidateDrug4Cancer: An Open Molecular Graph Learning Benchmark on Drug Discovery for Cancer
Anti-cancer drug discoveries have been serendipitous, we sought to present the Open Molecular Graph Learning Benchmark, named CandidateDrug4Cancer, a challenging and realistic benchmark dataset to facilitate scalable, robust, and reproducible graph machine learning research for anti-cancer drug discovery.
Learning Decoupling Features Through Orthogonality Regularization
Bearing this in mind, a two-branch deep network (KWS branch and SV branch) with the same network structure is developed and a novel decoupling feature learning method is proposed to push up the performance of KWS and SV simultaneously where speaker-invariant keyword representations and keyword-invariant speaker representations are expected respectively.
PASH at TREC 2021 Deep Learning Track: Generative Enhanced Model for Multi-stage Ranking
This paper describes the PASH participation in TREC 2021 Deep Learning Track.
SFE-AI at SemEval-2022 Task 11: Low-Resource Named Entity Recognition using Large Pre-trained Language Models
By assigning different weights to each model for different inputs, we adopted the Transformer layer to integrate the advantages of diverse models effectively.
Low Resource Named Entity Recognition
named-entity-recognition
+2
Collaboration of Pre-trained Models Makes Better Few-shot Learner
In this paper, we propose CoMo, a Collaboration of pre-trained Models that incorporates diverse prior knowledge from various pre-training paradigms for better few-shot learning.
HCL: Improving Graph Representation with Hierarchical Contrastive Learning
Contrastive learning has emerged as a powerful tool for graph representation learning.
Zebra: Deeply Integrating System-Level Provenance Search and Tracking for Efficient Attack Investigation
System auditing has emerged as a key approach for monitoring system call events and investigating sophisticated attacks.
Stare at What You See: Masked Image Modeling without Reconstruction
However, unlike the low-level features such as pixel values, we argue the features extracted by powerful teacher models already encode rich semantic correlation across regions in an intact image. This raises one question: is reconstruction necessary in Masked Image Modeling (MIM) with a teacher model?
Medical Knowledge Graph QA for Drug-Drug Interaction Prediction based on Multi-hop Machine Reading Comprehension
Drug-drug interaction prediction is a crucial issue in molecular biology.
Exploring Representation Learning for Small-Footprint Keyword Spotting
To address those challenges, we explore representation learning for KWS by self-supervised contrastive learning and self-training with pretrained model.
OGMN: Occlusion-guided Multi-task Network for Object Detection in UAV Images
One interaction for the guide is between two task decoders to address the feature confusion problem, and an occlusion decoupling head (ODH) is proposed to replace the general detection head.
Filter Pruning via Filters Similarity in Consecutive Layers
Filter pruning is widely adopted to compress and accelerate the Convolutional Neural Networks (CNNs), but most previous works ignore the relationship between filters and channels in different layers.
Enhanced Neural Beamformer with Spatial Information for Target Speech Extraction
In this paper, we propose a target speech extraction network that utilizes spatial information to enhance the performance of neural beamformer.
SparseMAE: Sparse Training Meets Masked Autoencoders
In this paper, we aim to reduce model complexity from large vision transformers pretrained by MAE with assistant of sparse training.
Improving Compositional Text-to-image Generation with Large Vision-Language Models
However, compositional text-to-image models frequently encounter difficulties in generating high-quality images that accurately align with input texts describing multiple objects, variable attributes, and intricate spatial relationships.
Digital Life Project: Autonomous 3D Characters with Social Intelligence
In this work, we present Digital Life Project, a framework utilizing language as the universal medium to build autonomous 3D characters, who are capable of engaging in social interactions and expressing with articulated body motions, thereby simulating life in a digital environment.
3DAxiesPrompts: Unleashing the 3D Spatial Task Capabilities of GPT-4V
In this work, we present a new visual prompting method called 3DAxiesPrompts (3DAP) to unleash the capabilities of GPT-4V in performing 3D spatial tasks.
LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding
In this paper, we introduce LiDAR-LLM, which takes raw LiDAR data as input and harnesses the remarkable reasoning capabilities of LLMs to gain a comprehensive understanding of outdoor 3D scenes.
Vision-Language Navigation with Embodied Intelligence: A Survey
As a long-term vision in the field of artificial intelligence, the core goal of embodied intelligence is to improve the perception, understanding, and interaction capabilities of agents and the environment.
Automated Design and Optimization of Distributed Filtering Circuits via Reinforcement Learning
Designing distributed filtering circuits (DFCs) is complex and time-consuming, with the circuit performance relying heavily on the expertise and experience of electronics engineers.
YOLO-TLA: An Efficient and Lightweight Small Object Detection Model based on YOLOv5
Additionally, we have incorporated a global attention mechanism into the backbone network.
In Defense and Revival of Bayesian Filtering for Thermal Infrared Object Tracking
Deep learning-based methods monopolize the latest research in the field of thermal infrared (TIR) object tracking.
Searching a Lightweight Network Architecture for Thermal Infrared Pedestrian Tracking
To expedite the search process, a random channel selection strategy is employed prior to assessing operation candidates.
MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?
To this end, we introduce MathVerse, an all-around visual math benchmark designed for an equitable and in-depth evaluation of MLLMs.
CT Synthesis with Conditional Diffusion Models for Abdominal Lymph Node Segmentation
To address these problems, we present a pipeline that integrates the conditional diffusion model for lymph node generation and the nnU-Net model for lymph node segmentation to improve the segmentation performance of abdominal lymph nodes through synthesizing a diversity of realistic abdominal lymph node data.
