Search Results for author:
Found 405 papers, 148 papers with code
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion.
Ranked #1 on
Zero-Shot Object Detection
on MSCOCO
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
Studying how people interact with large language models (LLMs) in real-world scenarios is increasingly important due to their widespread use in various applications.
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Evaluating large language model (LLM) based chat assistants is challenging due to their broad capabilities and the inadequacy of existing benchmarks in measuring human preferences.
Ranked #3 on
Long-Context Understanding
on Ada-LEval (TSort)
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
To address this issue, we introduce Chatbot Arena, an open platform for evaluating LLMs based on human preferences.
Efficient Memory Management for Large Language Model Serving with PagedAttention
On top of it, we build vLLM, an LLM serving system that achieves (1) near-zero waste in KV cache memory and (2) flexible sharing of KV cache within and across requests to further reduce memory usage.
DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
Depending on the task complexity, thousands to millions of labeled image pairs are needed to train a conditional GAN.
Ranked #2 on
Image-to-Image Translation
on Aerial-to-Map
DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
Compared to other models on the leaderboard, DINO significantly reduces its model size and pre-training data size while achieving better results.
Ranked #1 on
Real-Time Object Detection
on COCO 2017 val
Segment Everything Everywhere All at Once
In SEEM, we propose a novel decoding mechanism that enables diverse prompting for all types of segmentation tasks, aiming at a universal segmentation interface that behaves like large language models (LLMs).
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
We introduce Grounded SAM, which uses Grounding DINO as an open-set object detector to combine with the segment anything model (SAM).
Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation
In this paper we present Mask DINO, a unified object detection and segmentation framework.
Ranked #1 on
Panoptic Segmentation
on COCO test-dev
Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
We present Set-of-Mark (SoM), a new visual prompting method, to unleash the visual grounding abilities of large multimodal models (LMMs), such as GPT-4V.
Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning
Existing model-parallel training systems either require users to manually create a parallelization plan or automatically generate one from a limited space of model parallelism configurations.
AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving
Model parallelism is conventionally viewed as a method to scale a single large deep learning model beyond the memory limits of a single device.
DN-DETR: Accelerate DETR Training by Introducing Query DeNoising
Our method is universal and can be easily plugged into any DETR-like methods by adding dozens of lines of code to achieve a remarkable improvement.
Semantic-SAM: Segment and Recognize Anything at Any Granularity
In this paper, we introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity.
Visual In-Context Prompting
In-context prompting in large language models (LLMs) has become a prevalent approach to improve zero-shot capabilities, but this idea is less explored in the vision domain.
DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR
We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR.
Ranked #11 on
2D Object Detection
on SARDet-100K
detrex: Benchmarking Detection Transformers
To address this issue, we develop a unified, highly modular, and lightweight codebase called detrex, which supports a majority of the mainstream DETR-based instance recognition algorithms, covering various fundamental tasks, including object detection, segmentation, and pose estimation.
A Simple Framework for Open-Vocabulary Segmentation and Detection
We present OpenSeeD, a simple Open-vocabulary Segmentation and Detection framework that jointly learns from different segmentation and detection datasets.
Ranked #2 on
Instance Segmentation
on ADE20K val
(using extra training data)
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
Autoregressive decoding of large language models (LLMs) is memory bandwidth bounded, resulting in high latency and significant wastes of the parallel processing power of modern accelerators.
How Can Recommender Systems Benefit from Large Language Models: A Survey
In this paper, we conduct a comprehensive survey on this research direction from the perspective of the whole pipeline in real-world recommender systems.
A Strong and Reproducible Object Detector with Only Public Datasets
This work presents Focal-Stable-DINO, a strong and reproducible object detection model which achieves 64. 6 AP on COCO val2017 and 64. 8 AP on COCO test-dev using only 700M parameters without any test time augmentation.
Ranked #5 on
Object Detection
on COCO minival
(using extra training data)
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
LLaVA-Plus is a general-purpose multimodal assistant that expands the capabilities of large multimodal models.
Ranked #1 on
LMM real-life tasks
on Leaderboard
Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning
Some recent schedulers choose job resources for users, but do so without awareness of how DL training can be re-optimized to better utilize the provided resources.
Learning Implicit Fields for Generative Shape Modeling
We advocate the use of implicit fields for learning generative models of shapes and introduce an implicit field decoder, called IM-NET, for shape generation, aimed at improving the visual quality of the generated shapes.
Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning
We validate the learned sensor-level collision avoidance policy in a variety of simulated scenarios with thorough performance evaluations and show that the final learned policy is able to find time efficient, collision-free paths for a large-scale robot system.
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
To address this issue, we have created GVC data that allows for the combination of grounding and chat capabilities.
Neural Dual Contouring
We introduce neural dual contouring (NDC), a new data-driven approach to mesh reconstruction based on dual contouring (DC).
BSP-Net: Generating Compact Meshes via Binary Space Partitioning
The network is trained to reconstruct a shape using a set of convexes obtained from a BSP-tree built on a set of planes.
Learning Mesh Representations via Binary Space Partitioning Tree Networks
The network is trained to reconstruct a shape using a set of convexes obtained from a BSP-tree built over a set of planes, where the planes and convexes are both defined by learned network weights.
Detection Transformer with Stable Matching
We point out that the unstable matching in DETR is caused by a multi-optimization path problem, which is highlighted by the one-to-one matching design in DETR.
Lite DETR : An Interleaved Multi-Scale Encoder for Efficient DETR
Recent DEtection TRansformer-based (DETR) models have obtained remarkable performance.
From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Models
By simply equipping it with an MLP layer for alignment, DINO surpasses CLIP in fine-grained related perception tasks.
Neural Marching Cubes
To tackle these challenges, we re-cast MC from a deep learning perspective, by designing tessellation templates more apt at preserving geometric features, and learning the vertex positions and mesh topologies from training meshes, to account for contextual information from nearby cubes.
DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training
FlashAttention (Dao, 2023) effectively reduces the quadratic peak memory usage to linear in training transformer-based large language models (LLMs) on a single GPU.
DS-Fusion: Artistic Typography via Discriminated and Stylized Diffusion
We introduce a novel method to automatically generate an artistic typography by stylizing one or more letter fonts to visually convey the semantics of an input word, while ensuring that the output remains readable.
PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes
We introduce PQ-NET, a deep neural network which represents and generates 3D shapes via sequential part assembly.
MP-Former: Mask-Piloted Transformer for Image Segmentation
We present a mask-piloted Transformer which improves masked-attention in Mask2Former for image segmentation.
SketchyScene: Richly-Annotated Scene Sketches
We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level.
Span-based Localizing Network for Natural Language Video Localization
Given an untrimmed video and a text query, natural language video localization (NLVL) is to locate a matching span from the video that semantically corresponds to the query.
Automatic Photo Adjustment Using Deep Neural Networks
Many photographic styles rely on subtle adjustments that depend on the image content and even its semantics.
Interfacing Foundation Models' Embeddings
The proposed interface is adaptive to new tasks, and new models.
DECOR-GAN: 3D Shape Detailization by Conditional Refinement
During testing, a style code is fed into the generator to condition the refinement.
MPCFormer: fast, performant and private Transformer inference with MPC
Through extensive evaluations, we show that MPCFORMER significantly speeds up Transformer inference in MPC settings while achieving similar ML performance to the input model.
UKP-SQuARE v2: Explainability and Adversarial Attacks for Trustworthy QA
In this paper, we introduce SQuARE v2, the new version of SQuARE, to provide an explainability infrastructure for comparing models based on methods such as saliency maps and graph-based explanations.
UKP-SQuARE v3: A Platform for Multi-Agent QA Research
To ease research in multi-agent models, we extend UKP-SQuARE, an online platform for QA research, to support three families of multi-agent systems: i) agent selection, ii) early-fusion of agents, and iii) late-fusion of agents.
Token Shift Transformer for Video Classification
It is worth noticing that our TokShift transformer is a pure convolutional-free video transformer pilot with computational efficiency for video understanding.
ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishing
Zero-shot capability has been considered as a new revolution of deep learning, letting machines work on tasks without curated training data.
BAE-NET: Branched Autoencoder for Shape Co-Segmentation
The unsupervised BAE-NET is trained with a collection of un-segmented shapes, using a shape reconstruction loss, without any ground-truth labels.
TilinGNN: Learning to Tile with Self-Supervised Graph Neural Network
To start, we reformulate tiling as a graph problem by modeling candidate tile locations in the target shape as graph nodes and connectivity between tile locations as edges.
TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders
In this work, we propose TimeMAE, a novel self-supervised paradigm for learning transferrable time series representations based on transformer networks.
TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models
With this key idea, we design TeraPipe, a high-performance token-level pipeline parallel algorithm for synchronous model-parallel training of Transformer-based language models.
FaceDNeRF: Semantics-Driven Face Reconstruction, Prompt Editing and Relighting with Diffusion Models
The ability to create high-quality 3D faces from a single image has become increasingly important with wide applications in video conferencing, AR/VR, and advanced video editing in movie industries.
DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding
As phrase extraction can be regarded as a $1$D text segmentation problem, we formulate PEG as a dual detection problem and propose a novel DQ-DETR model, which introduces dual queries to probe different features from image and text for object prediction and phrase mask prediction.
Ranked #7 on
Referring Expression Comprehension
on RefCOCO
BalaGAN: Image Translation Between Imbalanced Domains via Cross-Modal Transfer
State-of-the-art image-to-image translation methods tend to struggle in an imbalanced domain setting, where one image domain lacks richness and diversity.
Video Corpus Moment Retrieval with Contrastive Learning
We adopt the first approach and introduce two contrastive learning objectives to refine video encoder and text encoder to learn video and text representations separately but with better alignment for VCMR.
WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling
To train an inference network jointly with a deep generative topic model, making it both scalable to big corpora and fast in out-of-sample prediction, we develop Weibull hybrid autoencoding inference (WHAI) for deep latent Dirichlet allocation, which infers posterior samples via a hybrid of stochastic-gradient MCMC and autoencoding variational Bayes.
UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification
The Fact Extraction and VERification (FEVER) shared task was launched to support the development of systems able to verify claims by extracting supporting or refuting facts from raw text.
CLLMs: Consistency Large Language Models
Parallel decoding methods such as Jacobi decoding show promise for more efficient LLM inference as it breaks the sequential nature of the LLM decoding process and transforms it into parallelizable computation.
LayoutGMN: Neural Graph Matching for Structural Layout Similarity
In particular, retrieval results by our network better match human judgement of structural layout similarity compared to both IoUs and other baselines including a state-of-the-art method based on graph neural networks and image convolution.
A Prototype-Oriented Framework for Unsupervised Domain Adaptation
Existing methods for unsupervised domain adaptation often rely on minimizing some statistical distance between the source and target samples in the latent space.
Roof-GAN: Learning to Generate Roof Geometry and Relations for Residential Houses
This paper presents Roof-GAN, a novel generative adversarial network that generates structured geometry of residential roof structures as a set of roof primitives and their relationships.
Estimation of genomic characteristics by analyzing k-mer frequency in de novo genome projects
Therefore, it is necessary to develop efficient assembly-independent methods for accurate estimation of these genomic characteristics.
GDPNet: Refining Latent Multi-View Graph for Relation Extraction
Recent advances on RE task are from BERT-based sequence modeling and graph-based modeling of relationships among the tokens in the sequence.
Ranked #4 on
Dialog Relation Extraction
on DialogRE
(F1c (v1) metric)
Discovering and Explaining the Representation Bottleneck of DNNs
This paper explores the bottleneck of feature representations of deep neural networks (DNNs), from the perspective of the complexity of interactions between input variables encoded in DNNs.
BIRNAT: Bidirectional Recurrent Neural Networks with Adversarial Training for Video Snapshot Compressive Imaging
This measurement and the modulation masks are fed into our Recurrent Neural Network (RNN) to reconstruct the desired high-speed frames.
Shape-IoU: More Accurate Metric considering Bounding Box Shape and Scale
As an important component of the detector localization branch, bounding box regression loss plays a significant role in object detection tasks.
Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning
In this paper, we propose an effective method to improve the model generalization by additionally penalizing the gradient norm of loss function during optimization.
Inner-IoU: More Effective Intersection over Union Loss with Auxiliary Bounding Box
Based on the above, we first analyzed the BBR model and concluded that distinguishing different regression samples and using different scales of auxiliary bounding boxes to calculate losses can effectively accelerate the bounding box regression process.
Ranked #1 on
Object Detection
on AI-TOD
(mAP50 metric)
HD-CNN: Hierarchical Deep Convolutional Neural Network for Large Scale Visual Recognition
In this paper, we introduce hierarchical deep CNNs (HD-CNNs) by embedding deep CNNs into a category hierarchy.
Ranked #174 on
Image Classification
on CIFAR-100
An End-to-End Neural Network for Image Cropping by Learning Composition from Aesthetic Photos
In this paper, we primarily focus on improving the accuracy of automatic image cropping, and on further exploring its potential in public datasets with high efficiency.
AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness
Scaling up model sizes can lead to fundamentally new capabilities in many machine learning (ML) tasks.
CompoNet: Learning to Generate the Unseen by Part Synthesis and Composition
Data-driven generative modeling has made remarkable progress by leveraging the power of deep neural networks.
RPM-Net: Recurrent Prediction of Motion and Parts from Point Cloud
We show results of simultaneous motion and part predictions from synthetic and real scans of 3D objects exhibiting a variety of part mobilities, possibly involving multiple movable parts.
Structured Generative Adversarial Networks
We study the problem of conditional generative modeling based on designated semantics or structures.
Predictive and Generative Neural Networks for Object Functionality
Given a 3D object in isolation, our functional similarity network (fSIM-NET), a variation of the triplet network, is trained to predict the functionality of the object by inferring functionality-revealing interaction contexts.
Memory-Efficient Network for Large-scale Video Compressive Sensing
With the knowledge of masks, optimization algorithms or deep learning methods are employed to reconstruct the desired high-speed video frames from this snapshot measurement.
Group Contextualization for Video Recognition
By utilizing calibrators to embed feature with four different kinds of contexts in parallel, the learnt representation is expected to be more resilient to diverse types of activities.
Ranked #3 on
Egocentric Activity Recognition
on EGTEA
MS-RNN: A Flexible Multi-Scale Framework for Spatiotemporal Predictive Learning
Spatiotemporal predictive learning, which predicts future frames through historical prior knowledge with the aid of deep learning, is widely used in many fields.
FED-NeRF: Achieve High 3D Consistency and Temporal Coherence for Face Video Editing on Dynamic NeRF
However, achieving simultaneously multi-view consistency and temporal coherence while editing video sequences remains a formidable challenge.
High-accuracy mass, spin, and recoil predictions of generic black-hole merger remnants
We present accurate fits for the remnant properties of generically precessing binary black holes, trained on large banks of numerical-relativity simulations.
General Relativity and Quantum Cosmology High Energy Astrophysical Phenomena
AutoLoss: Learning Discrete Schedules for Alternate Optimization
Many machine learning problems involve iteratively and alternately optimizing different task objectives with respect to different sets of parameters.
Symbolic Graph Reasoning Meets Convolutions
To cooperate with local convolutions, each SGR is constituted by three modules: a) a primal local-to-semantic voting module where the features of all symbolic nodes are generated by voting from local representations; b) a graph reasoning module propagates information over knowledge graph to achieve global semantic coherency; c) a dual semantic-to-local mapping module learns new associations of the evolved symbolic nodes with local representations, and accordingly enhances local features.
Ranked #81 on
Semantic Segmentation
on ADE20K val
Physical Interaction: Reconstructing Hand-object Interactions with Physics
Single view-based reconstruction of hand-object interaction is challenging due to the severe observation missing caused by occlusions.
MetaSCI: Scalable and Adaptive Reconstruction for Video Compressive Sensing
To capture high-speed videos using a two-dimensional detector, video snapshot compressive imaging (SCI) is a promising system, where the video frames are coded by different masks and then compressed to a snapshot measurement.
Can learning from natural image denoising be used for seismic data interpolation?
We propose a convolutional neural network (CNN) denoising based method for seismic data interpolation.
FLNeRF: 3D Facial Landmarks Estimation in Neural Radiance Fields
This paper presents the first significant work on directly predicting 3D face landmarks on neural radiance fields (NeRFs).
De novo Drug Design using Reinforcement Learning with Multiple GPT Agents
A central challenge in this field is to generate molecules with specific properties while also producing a wide range of diverse candidates.
MS-DETR: Natural Language Video Localization with Sampling Moment-Moment Interaction
Given a query, the task of Natural Language Video Localization (NLVL) is to localize a temporal moment in an untrimmed video that semantically matches the query.
ARO-Net: Learning Implicit Fields from Anchored Radial Observations
We introduce anchored radial observations (ARO), a novel shape encoding for learning implicit field representation of 3D shapes that is category-agnostic and generalizable amid significant shape variations.
Semi-supervised URL Segmentation with Recurrent Neural NetworksPre-trained on Knowledge Graph Entities
Breaking domain names such as openresearch into component words open and research is important for applications like Text-to-Speech synthesis and web search.
Semi-supervised URL Segmentation with Recurrent Neural Networks Pre-trained on Knowledge Graph Entities
Breaking domain names such as openresearch into component words open and research is important for applications like Text-to-Speech synthesis and web search.
GANHopper: Multi-Hop GAN for Unsupervised Image-to-Image Translation
Instead of executing translation directly, we steer the translation by requiring the network to produce in-between images that resemble weighted hybrids between images from the input domains.
Adaptive Split-Fusion Transformer
Neural networks for visual content understanding have recently evolved from convolutional ones (CNNs) to transformers.
High-throughput, high-resolution registration-free generated adversarial network microscopy
We combine generative adversarial network (GAN) with light microscopy to achieve deep learning super-resolution under a large field of view (FOV).
Focaler-IoU: More Focused Intersection over Union Loss
Existing researchs improve regression performance by utilizing the geometric relationship between bounding boxes, while ignoring the impact of difficult and easy sample distribution on bounding box regression.
Hybrid Neural Networks for On-device Directional Hearing
On-device directional hearing requires audio source separation from a given direction while achieving stringent human-imperceptible latency requirements.
Ranked #1 on
Real-time Directional Hearing
on VCTK
Manifoldron: Direct Space Partition via Manifold Discovery
A neural network with the widely-used ReLU activation has been shown to partition the sample space into many convex polytopes for prediction.
Neural Eigenfunctions Are Structured Representation Learners
Unlike prior spectral methods such as Laplacian Eigenmap that operate in a nonparametric manner, Neural Eigenmap leverages NeuralEF to parametrically model eigenfunctions using a neural network.
BSD-GAN: Branched Generative Adversarial Network for Scale-Disentangled Representation Learning and Image Synthesis
The key feature of BSD-GAN is that it is trained in multiple branches, progressively covering both the breadth and depth of the network, as resolutions of the training images increase to reveal finer-scale features.
Variational Hetero-Encoder Randomized GANs for Joint Image-Text Modeling
For bidirectional joint image-text modeling, we develop variational hetero-encoder (VHE) randomized generative adversarial network (GAN), a versatile deep generative model that integrates a probabilistic text decoder, probabilistic image encoder, and GAN into a coherent end-to-end multi-modality learning framework.
Spin-Orbit Protection of Induced Superconductivity in Majorana Nanowires
Spin-orbit interaction (SOI) plays a key role in creating Majorana zero modes in semiconductor nanowires proximity coupled to a superconductor.
Mesoscale and Nanoscale Physics
Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage
Small and imbalanced datasets commonly seen in healthcare represent a challenge when training classifiers based on deep learning models.
RIM-Net: Recursive Implicit Fields for Unsupervised Learning of Hierarchical Shape Structures
Each level of the tree corresponds to an assembly of shape parts, represented as implicit functions, to reconstruct the input shape.
Language Model Decomposition: Quantifying the Dependency and Correlation of Language Models
A goodness-of-fit metric for LMD similar to the coefficient of determination is defined and used to measure the linear dependency of a set of LMs.
DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentation
We present an unsupervised 3D shape co-segmentation method which learns a set of deformable part templates from a shape collection.
Multi-Task Dense Prediction via Mixture of Low-Rank Experts
Furthermore, to control the parameters and computational cost brought by the increase in the number of experts, we take inspiration from LoRA and propose to leverage the low-rank format of a vanilla convolution in the expert network.
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
The quadratic complexity and weak length extrapolation of Transformers limits their ability to scale to long sequences, and while sub-quadratic solutions like linear attention and state space models exist, they empirically underperform Transformers in pretraining efficiency and downstream task accuracy.
Downstream Transformer Generation of Question-Answer Pairs with Preprocessing and Postprocessing Pipelines
We present a system called TP3 to perform a downstream task of transformers on generating question-answer pairs (QAPs) from a given article.
Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion
Through the text semantic encoder and semantic interaction fusion decoder, Text-IF is accessible to the all-in-one infrared and visible image degradation-aware processing and the interactive flexible fusion outcomes.
Interventional Video Grounding with Dual Contrastive Learning
2) Meanwhile, we introduce a dual contrastive learning approach (DCL) to better align the text and video by maximizing the mutual information (MI) between query and video clips, and the MI between start/end frames of a target moment and the others within a video to learn more informative visual representations.
COSY: COunterfactual SYntax for Cross-Lingual Understanding
Pre-trained multilingual language models, e. g., multilingual-BERT, are widely used in cross-lingual tasks, yielding the state-of-the-art performance.
Multi-relation Message Passing for Multi-label Text Classification
These examples motivate the modelling of multiple types of bi-directional relationships between labels.
Multi-Label Classification
Multi-Label Image Classification
+4
Parameterization of Cross-Token Relations with Relative Positional Encoding for Vision MLP
They use token-mixing layers to capture cross-token interactions, as opposed to the multi-head self-attention mechanism used by Transformers.
ShaDDR: Interactive Example-Based Geometry and Texture Generation via 3D Shape Detailization and Differentiable Rendering
Furthermore, we showcase the ability of our method to learn geometric details and textures from shapes reconstructed from real-world photos.
Distantly-Supervised Long-Tailed Relation Extraction Using Constraint Graphs
On top of that, we further propose a novel constraint graph-based relation extraction framework(CGRE) to handle the two challenges simultaneously.
Heterogeneous Autoencoder Empowered by Quadratic Neurons
Inspired by the complexity and diversity of biological neurons, a quadratic neuron is proposed to replace the inner product in the current neuron with a simplified quadratic function.
Incorporating Instructional Prompts into a Unified Generative Framework for Joint Multiple Intent Detection and Slot Filling
Specifically, we design 5-type templates as instructional prompts, and each template includes a question that acts as the driver to teach UGEN to grasp the paradigm, options that list the candidate intents or slots to reduce the answer search space, and the context denotes original utterance.
MeaCap: Memory-Augmented Zero-shot Image Captioning
The framework of MeaCap achieves the state-of-the-art performance on a series of zero-shot IC settings.
FAME: 3D Shape Generation via Functionality-Aware Model Evolution
Our tool supports constrained modeling, allowing users to restrict or steer the model evolution with functionality labels.
Graphics
EnsLM: Ensemble Language Model for Data Diversity by Semantic Clustering
As a result, the backbone learns the shared knowledge among all clusters while modulated weights extract the cluster-specific features.
SAC-GAN: Structure-Aware Image Composition
Specifically, our network takes the semantic layout features from the input scene image, features encoded from the edges and silhouette in the input object patch, as well as a latent code as inputs, and generates a 2D spatial affine transform defining the translation and scaling of the object patch.
A Variational Edge Partition Model for Supervised Graph Representation Learning
This paper introduces a graph generative process to model how the observed edges are generated by aggregating the node interactions over a set of overlapping node communities, each of which contributes to the edges via a logical OR mechanism.
Long-term Leap Attention, Short-term Periodic Shift for Video Classification
By replacing a vanilla 2D attention with the LAPS, we could adapt a static transformer into a video one, with zero extra parameters and neglectable computation overhead ($\sim$2. 6\%).
Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planning
Automaton based approaches have enabled robots to perform various complex tasks.
NoisywikiHow: A Benchmark for Learning with Real-world Noisy Labels in Natural Language Processing
To mitigate the effects of label noise, learning with noisy labels (LNL) methods are designed to achieve better generalization performance.
TLM: Token-Level Masking for Transformers
Structured dropout approaches, such as attention dropout and DropHead, have been investigated to regularize the multi-head attention mechanism in Transformers.
Revisiting Single Image Reflection Removal In the Wild
This research focuses on the issue of single-image reflection removal (SIRR) in real-world conditions, examining it from two angles: the collection pipeline of real reflection pairs and the perception of real reflection locations.
Wavelet Regularization Benefits Adversarial Training
Since adversarial vulnerability can be regarded as a high-frequency phenomenon, it is essential to regulate the adversarially-trained neural network models in the frequency domain.
Computron: Serving Distributed Deep Learning Models with Model Parallel Swapping
In anticipation of workloads that involve serving many of such large models to handle different tasks, we develop Computron, a system that uses memory swapping to serve multiple distributed models on a shared GPU cluster.
Parameter-Efficient Conversational Recommender System as a Language Processing Task
Conversational recommender systems (CRS) aim to recommend relevant items to users by eliciting user preference through natural language conversation.
Interpretable Complex-Valued Neural Networks for Privacy Protection
Previous studies have found that an adversary attacker can often infer unintended input information from intermediate-layer features.
Quantification and Analysis of Layer-wise and Pixel-wise Information Discarding
We define two types of entropy-based metrics, i. e. (1) the discarding of pixel-wise information used in the forward propagation, and (2) the uncertainty of the input reconstruction, to measure input information contained by a specific layer from two perspectives.
Deep N-ary Error Correcting Output Codes
To facilitate the training of N-ary ECOC with deep learning base learners, we further propose three different variants of parameter sharing architectures for deep N-ary ECOC.
An Embarrassingly Simple Model for Dialogue Relation Extraction
Dialogue relation extraction (RE) is to predict the relation type of two entities mentioned in a dialogue.
Ranked #9 on
Dialog Relation Extraction
on DialogRE
Unlocking the Potential of Large Language Models for Explainable Recommendations
In this study, we propose LLMXRec, a simple yet effective two-stage explainable recommendation framework aimed at further boosting the explanation quality by employing LLMs.
Contrastive Attraction and Contrastive Repulsion for Representation Learning
We realize this strategy with contrastive attraction and contrastive repulsion (CACR), which makes the query not only exert a greater force to attract more distant positive samples but also do so to repel closer negative samples.
Combined Invariant Subspace \& Frequency-Domain Subspace Method for Identification of Discrete-Time MIMO Linear Systems
Recently, a novel system identification method based on invariant subspace theory is introduced, aiming to address the identification problem of continuous-time (CT) linear time-invariant (LTI) systems by combining time-domain and frequency-domain methods.
Empirical Evidence for the Fragment level Understanding on Drug Molecular Structure of LLMs
AI for drug discovery has been a research hotspot in recent years, and SMILES-based language models has been increasingly applied in drug molecular design.
Alternating Synthetic and Real Gradients for Neural Language Modeling
Empirically, we demonstrate the effectiveness of alternating training with synthetic and real gradients after periodic warm restarts on language modeling tasks.
Sentence Bag Graph Formulation for Biomedical Distant Supervision Relation Extraction
We introduce a novel graph-based framework for alleviating key challenges in distantly-supervised relation extraction and demonstrate its effectiveness in the challenging and important domain of biomedical data.
P2P-NET: Bidirectional Point Displacement Net for Shape Transform
We introduce P2P-NET, a general-purpose deep neural network which learns geometric transformations between point-based shape representations from two domains, e. g., meso-skeletons and surfaces, partial and complete scans, etc.
Semi-Supervised Co-Analysis of 3D Shape Styles from Projected Lines
We present a semi-supervised co-analysis method for learning 3D shape styles from projected feature lines, achieving style patch localization with only weak supervision.
On the Selection of Anchors and Targets for Video Hyperlinking
A principle way of hyperlinking can be carried out by picking centers of clusters as anchors and from there reach out to targets within or outside of clusters with consideration of neighborhood complexity.
Cavs: A Vertex-centric Programming Interface for Dynamic Neural Networks
Recent deep learning (DL) models have moved beyond static network architectures to dynamic ones, handling data where the network structure changes every example, such as sequences of variable lengths, trees, and graphs.
Efficient and Effective Single-Document Summarizations and A Word-Embedding Measurement of Quality
We show that the orderings of the ROUGE and WESM scores of our algorithms are highly comparable, suggesting that WESM may serve as a viable alternative for measuring the quality of a summary.
Mining Deep And-Or Object Structures via Cost-Sensitive Question-Answer-Based Active Annotations
The loss is defined for nodes in all layers of the AOG, including the generative loss (measuring the likelihood of the images) and the discriminative loss (measuring the fitness to human answers).
Generative Semantic Manipulation with Contrasting GAN
Generative Adversarial Networks (GANs) have recently achieved significant improvement on paired/unpaired image-to-image translation, such as photo$\rightarrow$ sketch and artist painting style transfer.
Ranked #4 on
Facial Expression Translation
on CelebA
Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters
We show that Poseidon enables Caffe and TensorFlow to achieve 15. 5x speed-up on 16 single-GPU machines, even with limited bandwidth (10GbE) and the challenging VGG19-22K network for image classification.
GRASS: Generative Recursive Autoencoders for Shape Structures
We introduce a novel neural network architecture for encoding and synthesis of 3D shapes, particularly their structures.
SCAN: Structure Correcting Adversarial Network for Organ Segmentation in Chest X-rays
Through this adversarial process the critic network learns the higher order structures and guides the segmentation model to achieve realistic segmentation outcomes.
Recurrent Topic-Transition GAN for Visual Paragraph Generation
The proposed Recurrent Topic-Transition Generative Adversarial Network (RTT-GAN) builds an adversarial framework between a structured paragraph generator and multi-level paragraph discriminators.
Generative Adversarial Network
Image Paragraph Captioning
+1
ZM-Net: Real-time Zero-shot Image Manipulation Network
We cast this problem as manipulating an input image according to a parametric model whose key parameters can be conditionally generated from any guiding signal (even unseen ones).
Sequence-based Multimodal Apprenticeship Learning For Robot Perception and Decision Making
Apprenticeship learning has recently attracted a wide attention due to its capability of allowing robots to learn physical tasks directly from demonstrations provided by human experts.
Simultaneous Feature and Body-Part Learning for Real-Time Robot Awareness of Human Behaviors
We formulate FABL as a regression-like optimization problem with structured sparsity-inducing norms to model interrelationships of body parts and features.
Space-Time Representation of People Based on 3D Skeletal Data: A Review
Spatiotemporal human representation based on 3D visual perception data is a rapidly growing research area.
Learning Concept Taxonomies from Multi-modal Data
We study the problem of automatically building hypernym taxonomies from textual and visual data.
Self-Reflective Risk-Aware Artificial Cognitive Modeling for Robot Response to Human Behaviors
In order for cooperative robots ("co-robots") to respond to human behaviors accurately and efficiently in human-robot collaboration, interpretation of human actions, awareness of new situations, and appropriate decision making are all crucial abilities for co-robots.
Enforcing Template Representability and Temporal Consistency for Adaptive Sparse Tracking
Sparse representation has been widely studied in visual tracking, which has shown promising tracking performance.
Combining the Best of Convolutional Layers and Recurrent Layers: A Hybrid Network for Semantic Segmentation
State-of-the-art results of semantic segmentation are established by Fully Convolutional neural Networks (FCNs).
On the Reducibility of Submodular Functions
The scalability of submodular optimization methods is critical for their usability in practice.
Poseidon: A System Architecture for Efficient GPU-based Deep Learning on Multiple Machines
To investigate how to adapt existing frameworks to efficiently support distributed GPUs, we propose Poseidon, a scalable system architecture for distributed inter-machine communication in existing DL frameworks.
Online Markov decision processes with policy iteration
The online Markov decision process (MDP) is a generalization of the classical Markov decision process that incorporates changing reward functions.
Task Selection for Bandit-Based Task Assignment in Heterogeneous Crowdsourcing
Task selection (picking an appropriate labeling task) and worker selection (assigning the labeling task to a suitable worker) are two major challenges in task assignment for crowdsourcing.
Bandit-Based Task Assignment for Heterogeneous Crowdsourcing
We consider a task assignment problem in crowdsourcing, which is aimed at collecting as many reliable labels as possible within a limited budget.
Statistical models and regularization strategies in statistical image reconstruction of low-dose X-ray CT: a survey
Statistical image reconstruction (SIR) methods have shown potential to substantially improve the image quality of low-dose X-ray computed tomography (CT) as compared to the conventional filtered back-projection (FBP) method for various clinical tasks.
Spatial-Spectral Boosting Analysis for Stroke Patients' Motor Imagery EEG in Rehabilitation Training
Current studies about motor imagery based rehabilitation training systems for stroke subjects lack an appropriate analytic method, which can achieve a considerable classification accuracy, at the same time detects gradual changes of imagery patterns during rehabilitation process and disinters potential mechanisms about motor function recovery.
Dual-label Deep LSTM Dereverberation For Speaker Verification
In this paper, we present a reverberation removal approach for speaker verification, utilizing dual-label deep neural networks (DNNs).
Semantic WordRank: Generating Finer Single-Document Summarizations
We present Semantic WordRank (SWR), an unsupervised method for generating an extractive summary of a single document.
SCORES: Shape Composition with Recursive Substructure Priors
The network may significantly alter the geometry and structure of the input parts and synthesize a novel shape structure based on the inputs, while adding or removing parts to minimize a structure plausibility loss.
Toward Understanding the Impact of Staleness in Distributed Machine Learning
Many distributed machine learning (ML) systems adopt the non-synchronous execution in order to alleviate the network communication bottleneck, resulting in stale parameters that do not reflect the latest updates.
Event Representation through Semantic Roles: Evaluation of Coverage
Semantic role theory is a widely used approach for event representation.
Towards Verifying Semantic Roles Co-occurrence
Semantic role theory considers roles as a small universal set of unanalyzed entities.
Hartley Spectral Pooling for Deep Learning
In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size.
Deep Poisson gamma dynamical systems
We develop deep Poisson-gamma dynamical systems (DPGDS) to model sequentially observed multivariate count data, improving previously proposed models by not only mining deep hierarchical latent structure from the data, but also capturing both first-order and long-range temporal dependencies.
Nearly-tight bounds on linear regions of piecewise linear neural networks
The developments of deep neural networks (DNN) in recent years have ushered a brand new era of artificial intelligence.
Fast and Accurate Reordering with ITG Transition RNN
Compared to a strong baseline of attention-based RNN, our ITG RNN re-ordering model can reach the same reordering accuracy with only 1/10 of the training data and is 2. 5x faster in decoding.
Learning Multi-Instance Enriched Image Representations via Non-Greedy Ratio Maximization of the l1-Norm Distances
To tackle these two challenges, in this paper we propose a novel image representation learning method that can integrate the local patches (the instances) of an input image (the bag) and its holistic representation into one single-vector representation.
Generative Semantic Manipulation with Mask-Contrasting GAN
Despite the promising results on paired/unpaired image-to-image translation achieved by Generative Adversarial Networks (GANs), prior works often only transfer the low-level information (e. g. color or texture changes), but fail to manipulate high-level semantic meanings (e. g., geometric structure or content) of different object regions.
DATNet: Dual Adversarial Transfer for Low-resource Named Entity Recognition
We propose a new architecture termed Dual Adversarial Transfer Network (DATNet) for addressing low-resource Named Entity Recognition (NER).
Low Resource Named Entity Recognition
named-entity-recognition
+2
VHEGAN: Variational Hetero-Encoder Randomized GAN for Zero-Shot Learning
To extract and relate visual and linguistic concepts from images and textual descriptions for text-based zero-shot learning (ZSL), we develop variational hetero-encoder (VHE) that decodes text via a deep probabilisitic topic model, the variational posterior of whose local latent variables is encoded from an image via a Weibull distribution based inference network.
Simplex-Based 3D Spatio-Temporal Feature Description for Action Recognition
In addition, the results show that our SOD descriptor is a superior individual descriptor for action recognition.
Sparse Dictionary Learning for Edit Propagation of High-Resolution Images
Previous approaches for edit propagation typically employ a global optimization over the whole set of image pixels, incurring a prohibitively high memory and time consumption for high-resolution images.
HD-CNN: Hierarchical Deep Convolutional Neural Networks for Large Scale Visual Recognition
In this paper, we introduce hierarchical deep CNNs (HD-CNNs) by embedding deep CNNs into a category hierarchy.
AdaCoSeg: Adaptive Shape Co-Segmentation with Group Consistency Loss
While the part prior network can be trained with noisy and inconsistently segmented shapes, the final output of AdaCoSeg is a consistent part labeling for the input set, with each shape segmented into up to (a user-specified) K parts.
LOGAN: Unpaired Shape Transform in Latent Overcomplete Space
Our network consists of an autoencoder to encode shapes from the two input domains into a common latent space, where the latent codes concatenate multi-scale shape features, resulting in an overcomplete representation.
DenseAttentionSeg: Segment Hands from Interacted Objects Using Depth Input
We propose a real-time DNN-based technique to segment hand and object of interacting motions from depth inputs.
Multisensory Omni-directional Long-term Place Recognition: Benchmark Dataset and Analysis
We introduce a new dataset Multisensory Omnidirectional Long-term Place recognition (MOLP) comprising omnidirectional intensity and disparity images.
Robotics
Constrained low-tubal-rank tensor recovery for hyperspectral images mixed noise removal by bilateral random projections
In this paper, we propose a novel low-tubal-rank tensor recovery model, which directly constrains the tubal rank prior for effectively removing the mixed Gaussian and sparse noise in hyperspectral images.
A Hybrid Precipitation Prediction Method based on Multicellular Gene Expression Programming
Prompt and accurate precipitation forecast is very important for development management of regional water resource, flood disaster prevention and people's daily activity and production plan; however, non-linear and nonstationary characteristics of precipitation data and noise seriously affect forecast accuracy.
A Seft-adaptive Multicellular GEP Algorithm Based On Fuzzy Control For Function Optimization
According to the concentration and dispersion of individual fitness values in population, the crossover rate, mutation rate and real number set mutation rate of genetic operation are dynamically adjusted.
GRAINS: Generative Recursive Autoencoders for INdoor Scenes
We present a generative neural network which enables us to generate plausible 3D indoor scenes in large quantities and varieties, easily and highly efficiently.
Graphics
Improving Performance of End-to-End ASR on Numeric Sequences
This out-of-vocabulary (OOV) issue is addressed in conventional ASR systems by training part of the model on spoken domain utterances (e. g.
Dual Adversarial Neural Transfer for Low-Resource Named Entity Recognition
We propose a new neural transfer method termed Dual Adversarial Transfer Network (DATNet) for addressing low-resource Named Entity Recognition (NER).
