Search Results for author:
Found 229 papers, 76 papers with code
Can We Really Trust Explanations? Evaluating the Stability of Feature Attribution Explanation Methods via Adversarial Attack
“Explanations can increase the transparency of neural networks and make them more trustworthy.
Rethinking Sentiment Style Transfer
To overcome this issue, we propose a graph-based method to extract attribute content and attribute-independent content from input sentences in the YELP dataset and IMDB dataset.
A Versatile Adaptive Curriculum Learning Framework for Task-oriented Dialogue Policy Learning
It supports evaluating the difficulty of dialogue tasks only using the learning experiences of dialogue policy and skip-level selection according to their learning needs to maximize the learning efficiency.
A Flexible Recurrent Residual Pyramid Network for Video Frame Interpolation
Inspired by classical pyramid energy minimization optical flow algorithms, this paper proposes a recurrent residual pyramid network (RRPN) for video frame interpolation.
A Simple Yet Effective Corpus Construction Method for Chinese Sentence Compression
To remedy this shortcoming, we present a dependency-tree-based method to construct a Chinese corpus with 151k pairs of sentences and compression based on Chinese language-specific characteristics.
Bias Analysis and Mitigation through Protected Attribute Detection and Regard Classification
Large language models (LLMs) acquire general linguistic knowledge from massive-scale pretraining.
SEGA: Drivable 3D Gaussian Head Avatar from a Single Image
Creating photorealistic 3D head avatars from limited input has become increasingly important for applications in virtual reality, telepresence, and digital entertainment.
Information Gain-Guided Causal Intervention for Autonomous Debiasing Large Language Models
Despite significant progress, recent studies indicate that current large language models (LLMs) may still capture dataset biases and utilize them during inference, leading to the poor generalizability of LLMs.
Seaweed-7B: Cost-Effective Training of Video Generation Foundation Model
This technical report highlights the key design decisions that enhance the performance of the medium-sized diffusion model.
MLKV: Efficiently Scaling up Large Embedding Model Training with Disk-based Key-Value Storage
As increasingly diverse ML applications utilize embedding models and embedding tables continue to grow in size and number, there has been a surge in the ad-hoc development of specialized frameworks targeted to train large embedding models for specific tasks.
TransDiffSBDD: Causality-Aware Multi-Modal Structure-Based Drug Design
To address both challenges, we propose TransDiffSBDD, an integrated framework combining autoregressive transformers and diffusion models for SBDD.
Diff-Palm: Realistic Palmprint Generation with Polynomial Creases and Intra-Class Variation Controllable Diffusion Models
However, without employing real data fine-tuning, the performance of the recognition model trained on these synthetic datasets would drastically decline, indicating a large gap between generated and real palmprints.
PathoHR: Breast Cancer Survival Prediction on High-Resolution Pathological Images
Breast cancer survival prediction in computational pathology presents a remarkable challenge due to tumor heterogeneity.
MVD-HuGaS: Human Gaussians from a Single Image via 3D Human Multi-view Diffusion Prior
We first generate multi-view images from the single reference image with an enhanced multi-view diffusion model, which is well fine-tuned on high-quality 3D human datasets to incorporate 3D geometry priors and human structure priors.
Efficient Jailbreaking of Large Models by Freeze Training: Lower Layers Exhibit Greater Sensitivity to Harmful Content
With the widespread application of Large Language Models across various domains, their security issues have increasingly garnered significant attention from both academic and industrial communities.
Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models
Existing Multimodal Large Language Models (MLLMs) are predominantly trained and tested on consistent visual-textual inputs, leaving open the question of whether they can handle inconsistencies in real-world, layout-rich content.
DOEI: Dual Optimization of Embedding Information for Attention-Enhanced Class Activation Maps
This alignment of activation responses with semantic information strengthens the propagation and decoupling of target features, enabling the generated embeddings to more accurately represent target features in high-level semantic space.
Weakly supervised Semantic Segmentation
Weakly-Supervised Semantic Segmentation
Poster: SpiderSim: Multi-Agent Driven Theoretical Cybersecurity Simulation for Industrial Digitalization
Rapid industrial digitalization has created intricate cybersecurity demands that necessitate effective validation methods.
PedDet: Adaptive Spectral Optimization for Multimodal Pedestrian Detection
Pedestrian detection in intelligent transportation systems has made significant progress but faces two critical challenges: (1) insufficient fusion of complementary information between visible and infrared spectra, particularly in complex scenarios, and (2) sensitivity to illumination changes, such as low-light or overexposed conditions, leading to degraded performance.
Fluid Antenna Enabled Over-the-Air Federated Learning: Joint Optimization of Positioning, Beamforming, and User Selection
Over-the-air (OTA) federated learning (FL) effectively utilizes communication bandwidth, yet it is vulnerable to errors during analog aggregation.
Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selection
Large language models (LLMs) have shown great potential across various industries due to their remarkable ability to generalize through instruction tuning.
3DMolFormer: A Dual-channel Framework for Structure-based Drug Discovery
Structure-based drug discovery, encompassing the tasks of protein-ligand docking and pocket-aware 3D drug design, represents a core challenge in drug discovery.
MedConv: Convolutions Beat Transformers on Long-Tailed Bone Density Prediction
Bone density prediction via CT scans to estimate T-scores is crucial, providing a more precise assessment of bone health compared to traditional methods like X-ray bone density tests, which lack spatial resolution and the ability to detect localized changes.
GENIE: Generative Note Information Extraction model for structuring EHR data
Traditional methods for structuring EHR free-text data, such as rule-based systems and multi-stage pipelines, are often limited by their time-consuming configurations and inability to adapt across clinical notes from diverse healthcare settings.
Movable Antenna-Aided Cooperative ISAC Network with Time Synchronization error and Imperfect CSI
We analyze the impact of CSI errors on achievable rates and introduce a hybrid Cramer-Rao lower bound (HCRLB) to evaluate the effect of TS errors on target localization accuracy.
Deep Reinforcement Learning
Integrated sensing and communication
+1
SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration
Video restoration poses non-trivial challenges in maintaining fidelity while recovering temporally consistent details from unknown degradations in the wild.
ProjectedEx: Enhancing Generation in Explainable AI for Prostate Cancer
Additionally, we conduct comprehensive experiments on both the generator and classifier, demonstrating the clinical relevance and effectiveness of ProjectedEx in enhancing interpretability and supporting the adoption of AI in medical settings.
Optimizing SSD Caches for Cloud Block Storage Systems Using Machine Learning Approaches
The management of data writes to SSD caches plays a crucial role in improving overall system performance, reducing latency, and extending the lifespan of storage devices.
Dynamic Optimization of Storage Systems Using Reinforcement Learning Techniques
Traditional heuristics employed for storage performance optimization often fail to adapt to the variability and complexity of contemporary workloads, leading to significant performance bottlenecks and resource inefficiencies.
Dynamic Adaptation in Data Storage: Real-Time Machine Learning for Enhanced Prefetching
The exponential growth of data storage demands has necessitated the evolution of hierarchical storage management strategies [1].
SegKAN: High-Resolution Medical Image Segmentation with Long-Distance Dependencies
Hepatic vessels in computed tomography scans often suffer from image fragmentation and noise interference, making it difficult to maintain vessel integrity and posing significant challenges for vessel segmentation.
Fast inverse lithography based on a model-driven block stacking convolutional neural network
In the realm of lithography, Optical Proximity Correction (OPC) is a crucial resolution enhancement technique that optimizes the transmission function of photomasks on a pixel-based to effectively counter Optical Proximity Effects (OPE).
Latency Minimization for Movable Antennas-Enabled Relay-aided D2D Mobile Edge Computing Communication Systems
Specifically, the core of MA technology lies in optimizing the antenna positions to increase system capacity.
Multimodal Class-aware Semantic Enhancement Network for Audio-Visual Video Parsing
However, each segment may contain multiple events, resulting in semantically mixed holistic features that can lead to semantic interference during intra- or cross-modal interactions: the event semantics of one segment may incorporate semantics of unrelated events from other segments.
Lightweight Multiplane Images Network for Real-Time Stereoscopic Conversion from Planar Video
This paper proposes a planar video real-time stereoscopic conversion network based on multi-plane images (MPI), which consists of a detail branch for generating MPI and a depth-semantic branch for perceiving depth information.
Controlling the Latent Diffusion Model for Generative Image Shadow Removal via Residual Generation
Large-scale generative models have achieved remarkable advancements in various visual tasks, yet their application to shadow removal in images remains challenging.
Multi-task CNN Behavioral Embedding Model For Transaction Fraud Detection
The burgeoning e-Commerce sector requires advanced solutions for the detection of transaction fraud.
Movable Antenna-Aided Federated Learning with Over-the-Air Aggregation: Joint Optimization of Positioning, Beamforming, and User Selection
Federated learning (FL) in wireless computing effectively utilizes communication bandwidth, yet it is vulnerable to errors during the analog aggregation process.
Image Understanding Makes for A Good Tokenizer for Image Generation
Currently, pixel reconstruction (e. g., VQGAN) dominates the training objective for image tokenizers.
Corporate Fundamentals and Stock Price Co-Movement
We introduce an innovative framework that leverages advanced big data techniques to analyze dynamic co-movement between stocks and their underlying fundamentals using high-frequency stock market data.
CIT: Rethinking Class-incremental Semantic Segmentation with a Class Independent Transformation
Class-incremental semantic segmentation (CSS) requires that a model learn to segment new classes without forgetting how to segment previous ones: this is typically achieved by distilling the current knowledge and incorporating the latest data.
Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning
This paper proposes a deep reinforcement learning-based model for decision-making in multi-role UAV cooperative pursuit-evasion game, to address the challenge of enabling UAV to autonomously make decisions in complex game environments.
Distribution alignment based transfer fusion frameworks on quantum devices for seeking quantum advantages
Two transfer fusion frameworks are proposed in this paper to predict the labels of a target domain data by aligning its distribution to a different but related labelled source domain on quantum devices.
How Far is Video Generation from World Model: A Physical Law Perspective
Our scaling experiments show perfect generalization within the distribution, measurable scaling behavior for combinatorial generalization, but failure in out-of-distribution scenarios.
Power Source Allocation for RIS-aided Integrating Sensing, Communication, and Power Transfer Systems Based on NOMA
For the PS factor optimization problem, the SCA algorithm is proposed.
Conditional Uncertainty Quantification for Tensorized Topological Neural Networks
Graph Neural Networks (GNNs) have become the de facto standard for analyzing graph-structured data, leveraging message-passing techniques to capture both structural and node feature information.
Medical AI for Early Detection of Lung Cancer: A Survey
The recent emergence of deep learning has revolutionized medical image analysis, driving substantial advancements in this field.
Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation
These abilities, which are developed using proprietary and unavailable training data, make existing continual instruction tuning methods ineffective.
MMLF: Multi-modal Multi-class Late Fusion for Object Detection with Uncertainty Estimation
Autonomous driving necessitates advanced object detection techniques that integrate information from multiple modalities to overcome the limitations associated with single-modal approaches.
Toward Scalable Image Feature Compression: A Content-Adaptive and Diffusion-Based Approach
Extensive experiments demonstrate the effectiveness of the proposed framework in both image reconstruction and downstream machine vision tasks such as object detection, segmentation, and facial landmark detection, achieving superior perceptual quality compared to state-of-the-art methods.
Ordinal Preference Optimization: Aligning Human Preferences via NDCG
In this work, we propose a novel listwise approach named Ordinal Preference Optimization (OPO), which employs the Normalized Discounted Cumulative Gain (NDCG), a widely-used ranking metric, to better utilize relative proximity within ordinal multiple responses.
Loong: Generating Minute-level Long Videos with Autoregressive Language Models
Autoregressive large language models (LLMs) have achieved great success in generating coherent and long sequences of tokens in the domain of natural language processing, while the exploration of autoregressive LLMs for video generation is limited to generating short videos of several seconds.
NTIRE 2024 Challenge on Stereo Image Super-Resolution: Methods and Results
This paper summarizes the 3rd NTIRE challenge on stereo image super-resolution (SR) with a focus on new solutions and results.
Robust Beamforming Design for Near-Field DMA-NOMA mmWave Communications With Imperfect Position Information
For millimeter-wave (mmWave) non-orthogonal multiple access (NOMA) communication systems, we propose an innovative near-field (NF) transmission framework based on dynamic metasurface antenna (DMA) technology.
Supervised Fine-Tuning Achieve Rapid Task Adaption Via Alternating Attention Head Activation Patterns
We find that: (1) LLMs selectively activate task-specific attention heads during SFT; (2) activation patterns for complex tasks are combinations of basic task patterns; and (3) changes in a few parameters can significantly impact activation patterns after SFT on a small number of samples. Based on these insights, experiments are conducted to actually enhance the efficiency and effectiveness of SFT.
Delay Minimization for Movable Antennas-Enabled Anti-Jamming Communications With Mobile Edge Computing
The increase in DoF enhances the system's anti-jamming capabilities and reduces system delay.
MSDet: Receptive Field Enhanced Multiscale Detection for Tiny Pulmonary Nodule
Pulmonary nodules are critical indicators for the early diagnosis of lung cancer, making their detection essential for timely treatment.
Hybrid Cost Volume for Memory-Efficient Optical Flow
Compared to the recurrent flow methods based the all-pairs cost volumes, our HCVFlow significantly reduces memory consumption while ensuring high accuracy.
StimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Models
To address this issue, we propose StimuVAR, a spatiotemporal Stimuli-aware framework for Video Affective Reasoning (VAR) with MLLMs.
MedDet: Generative Adversarial Distillation for Efficient Cervical Disc Herniation Detection
Additionally, we customize the second-order nmODE to improve the model's resistance to noise in MRI.
Passenger hazard perception based on EEG signals for highly automated driving vehicles
Enhancing the safety of autonomous vehicles is crucial, especially given recent accidents involving automated systems.
ESA: Annotation-Efficient Active Learning for Semantic Segmentation
Active learning enhances annotation efficiency by selecting the most revealing samples for labeling, thereby reducing reliance on extensive human input.
Causal-Guided Active Learning for Debiasing Large Language Models
Although achieving promising performance, recent analyses show that current generative large language models (LLMs) may still capture dataset biases and utilize them for generation, leading to poor generalizability and harmfulness of LLMs.
Generative Diffusion Model-based Downscaling of Observed Sea Surface Height over Kuroshio Extension since 2000
Satellite altimetry has been widely utilized to monitor global sea surface dynamics, enabling investigation of upper ocean variability from basin-scale to localized eddy ranges.
HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priors
The prior learning phase leverages 3D head priors derived from a large-scale multi-view dynamic dataset, and the avatar creation phase applies these priors for few-shot personalization.
SegStitch: Multidimensional Transformer for Robust and Efficient Medical Imaging Segmentation
Medical imaging segmentation plays a significant role in the automatic recognition and analysis of lesions.
Apple Intelligence Foundation Language Models
We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large server-based language model designed for Private Cloud Compute.
MIMO Channel Shaping and Rate Maximization Using Beyond Diagonal RIS
This paper investigates the limits to which a passive Reconfigurable Intelligent Surface (RIS) can reshape a point-to-point Multiple-Input Multiple-Output (MIMO) in terms of singular values for improved wireless (e. g., rate and power) performance.
CodeV: Empowering LLMs for Verilog Generation through Multi-Level Summarization
Instruction-tuned large language models (LLMs) have demonstrated remarkable performance in automatically generating code for general-purpose programming languages like Python.
Multi-Scenario Combination Based on Multi-Agent Reinforcement Learning to Optimize the Advertising Recommendation System
This paper explores multi-scenario optimization on large platforms using multi-agent reinforcement learning (MARL).
Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding
Based on the tree, our ToL agent not only comprehends the content of the indicated area but also articulates the layout and spatial relationships between elements.
Learning Traffic Crashes as Language: Datasets, Benchmarks, and What-if Causal Analyses
Leveraging this rich dataset, we further formulate the crash event feature learning as a novel text reasoning problem and further fine-tune various large language models (LLMs) to predict detailed accident outcomes, such as crash types, severity and number of injuries, based on contextual and environmental factors.
When Will Gradient Regularization Be Harmful?
Meanwhile, we note that scalable models tend to rely more on the GR warmup, where the performance can be improved by up to 3\% on Cifar10 compared to baseline GR.
Research on Driver Facial Fatigue Detection Based on Yolov8 Model
In a society where traffic accidents frequently occur, fatigue driving has emerged as a grave issue.
Predict Click-Through Rates with Deep Interest Network Model in E-commerce Advertising
This paper proposes new methods to enhance click-through rate (CTR) prediction models using the Deep Interest Network (DIN) model, specifically applied to the advertising system of Alibaba's Taobao platform.
Optimizing Search Advertising Strategies: Integrating Reinforcement Learning with Generalized Second-Price Auctions for Enhanced Ad Ranking and Bidding
This paper explores the integration of strategic optimization methods in search advertising, focusing on ad ranking and bidding mechanisms within E-commerce platforms.
Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motion
Firstly, we proposed a novel agent-based approach named Motion Avatar, which allows for the automatic generation of high-quality customizable human and animal avatars with motions through text queries.
FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion
In this work, we propose FreeBind, an idea that treats multimodal representation spaces as basic units, and freely augments pre-trained unified space by integrating knowledge from extra expert spaces via "space bonds".
Improving Subject-Driven Image Synthesis with Subject-Agnostic Guidance
In subject-driven text-to-image synthesis, the synthesis process tends to be heavily influenced by the reference images provided by users, often overlooking crucial attributes detailed in the text prompt.
Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study
Forecasting the short-term spread of an ongoing disease outbreak is a formidable challenge due to the complexity of contributing factors, some of which can be characterized through interlinked, multi-modality variables such as epidemiological time series data, viral biology, population demographics, and the intersection of public policy and human behavior.
MEBS: Multi-task End-to-end Bid Shading for Multi-slot Display Advertising
Through extensive offline and online experiments, we demonstrate the effectiveness and efficiency of our method, and we obtain a 7. 01% lift in Gross Merchandise Volume, a 7. 42% lift in Return on Investment, and a 3. 26% lift in ad buy count.
OHTA: One-shot Hand Avatar via Data-driven Implicit Priors
In this paper, we delve into the creation of one-shot hand avatars, attaining high-fidelity and drivable hand representations swiftly from a single image.
Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning
Through pretraining on a corpus with various sources, Large Language Models (LLMs) have gained impressive performance.
A novel spatial-frequency domain network for zero-shot incremental learning
The proposed SFDNet has the ability to effectively extract spatial-frequency feature representation from input images, improve the accuracy of image classification, and fundamentally alleviate catastrophic forgetting.
Federated Learning with New Knowledge: Fundamentals, Advances, and Futures
Federated Learning (FL) is a privacy-preserving distributed learning approach that is rapidly developing in an era where privacy protection is increasingly valued.
Audio-Infused Automatic Image Colorization by Exploiting Audio Scene Semantics
Automatic image colorization is inherently an ill-posed problem with uncertainty, which requires an accurate semantic understanding of scenes to estimate reasonable colors for grayscale images.
Empirical Evidence for the Fragment level Understanding on Drug Molecular Structure of LLMs
AI for drug discovery has been a research hotspot in recent years, and SMILES-based language models has been increasingly applied in drug molecular design.
Instruct-Imagen: Image Generation with Multi-modal Instruction
We introduce *multi-modal instruction* for image generation, a task representation articulating a range of generation intents with precision.
Deep Video Inverse Tone Mapping Based on Temporal Clues
Although many deep image ITM methods can generate impressive results the field of video ITM is still to be explored.
Comparing roughness maps generated by five roughness descriptors for LiDAR-derived digital elevation models
Terrain surface roughness, often described abstractly, poses challenges in quantitative characterisation with various descriptors found in the literature.
Multi-Modal Domain Adaptation Across Video Scenes for Temporal Video Grounding
Thus, to address this issue and enhance model performance on new scenes, we explore the TVG task in an unsupervised domain adaptation (UDA) setting across scenes for the first time, where the video-query pairs in the source scene (domain) are labeled with temporal boundaries, while those in the target scene are not.
De novo Drug Design using Reinforcement Learning with Multiple GPT Agents
A central challenge in this field is to generate molecules with specific properties while also producing a wide range of diverse candidates.
CoRTEx: Contrastive Learning for Representing Terms via Explanations with Applications on Constructing Biomedical Knowledge Graphs
Due to a lack of knowledge, previous contrastive learning models trained with Unified Medical Language System (UMLS) synonyms struggle at clustering difficult terms and do not generalize well beyond UMLS terms.
Adapting Vision Transformer for Efficient Change Detection
Most change detection models based on vision transformers currently follow a "pretraining then fine-tuning" strategy.
DreamInpainter: Text-Guided Subject-Driven Image Inpainting with Diffusion Models
Our extensive experiments demonstrate the superior performance of our method in terms of visual quality, identity preservation, and text control, showcasing its effectiveness in the context of text-guided subject-driven image inpainting.
HiFi Tuner: High-Fidelity Subject-Driven Fine-Tuning for Diffusion Models
We further extend our method to a novel image editing task: substituting the subject in an image through textual manipulations.
MobileDiffusion: Instant Text-to-Image Generation on Mobile Devices
The deployment of large-scale text-to-image diffusion models on mobile devices is impeded by their substantial model size and slow inference speed.
Cut-and-Paste: Subject-Driven Video Editing with Attention Control
This paper presents a novel framework termed Cut-and-Paste for real-word semantic video editing under the guidance of text prompt and additional reference image.
UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANs
Text-to-image diffusion models have demonstrated remarkable capabilities in transforming textual prompts into coherent images, yet the computational cost of their inference remains a persistent challenge.
Exploring Federated Unlearning: Analysis, Comparison, and Insights
The increasing demand for privacy-preserving machine learning has spurred interest in federated unlearning, which enables the selective removal of data from models trained in federated systems.
Extending Multi-modal Contrastive Representations
Inspired by recent C-MCR, this paper proposes Extending Multimodal Contrastive Representation (Ex-MCR), a training-efficient and paired-data-free method to flexibly learn unified contrastive representation space for more than three modalities by integrating the knowledge of existing MCR spaces.
TranDRL: A Transformer-Driven Deep Reinforcement Learning Enabled Prescriptive Maintenance Framework
Industrial systems demand reliable predictive maintenance strategies to enhance operational efficiency and reduce downtime.
Cross-Dataset-Robust Method for Blind Real-World Image Quality Assessment
First, many individual models based on popular and state-of-the-art (SOTA) Swin-Transformer (SwinT) are trained on different real-world BIQA datasets respectively.
BenchTemp: A General Benchmark for Evaluating Temporal Graph Neural Networks
To handle graphs in which features or connectivities are evolving over time, a series of temporal graph neural networks (TGNNs) have been proposed.
Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes
This paper presents Chat-3D, which combines the 3D visual perceptual ability of pre-trained 3D representations and the impressive reasoning and conversation capabilities of advanced LLMs to achieve the first universal dialogue systems for 3D scenes.
RPG-Palm: Realistic Pseudo-data Generation for Palmprint Recognition
Palmprint recently shows great potential in recognition applications as it is a privacy-friendly and stable biometric.
3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding
3D visual grounding aims to localize the target object in a 3D point cloud by a free-form language description.
Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond
An authentic face restoration system is becoming increasingly demanding in many computer vision applications, e. g., image enhancement, video communication, and taking portrait.
Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Grounding
To accomplish this, we design a novel semantic matching model that analyzes the semantic similarity between object proposals and sentences in a coarse-to-fine manner.
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs
Our experiments show that BuboGPT achieves impressive multi-modality understanding and visual grounding abilities during the interaction with human.
DVFO: Learning-Based DVFS for Energy-Efficient Edge-Cloud Collaborative Inference
Due to limited resources on edge and different characteristics of deep neural network (DNN) models, it is a big challenge to optimize DNN inference performance in terms of energy consumption and end-to-end latency on edge devices.
CLIP3Dstyler: Language Guided 3D Arbitrary Neural Style Transfer
In this paper, we propose a novel language-guided 3D arbitrary neural style transfer method (CLIP3Dstyler).
Connecting Multi-modal Contrastive Representations
This paper proposes a novel training-efficient method for learning MCR without paired data called Connecting Multi-modal Contrastive Representations (C-MCR).
RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texture
The proposed framework consists of two significant components: Geometry Guided Diffusion and Mesh Optimization.
Pink-Eggs Dataset V1: A Step Toward Invasive Species Management Using Deep Learning Embedded Solutions
We introduce a novel dataset consisting of images depicting pink eggs that have been identified as Pomacea canaliculata eggs, accompanied by corresponding bounding box annotations.
E2TIMT: Efficient and Effective Modal Adapter for Text Image Machine Translation
Furthermore, the ablation studies verify the generalization of our method, where the proposed modal adapter is effective to bridge various OCR and MT models.
Multi-Teacher Knowledge Distillation For Text Image Machine Translation
Text image machine translation (TIMT) has been widely used in various real-world applications, which translates source language texts in images into another target language sentence.
Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing
Instant on-device Neural Radiance Fields (NeRFs) are in growing demand for unleashing the promise of immersive AR/VR experiences, but are still limited by their prohibitive training time.
BCEdge: SLO-Aware DNN Inference Services with Adaptive Batching on Edge Platforms
As deep neural networks (DNNs) are being applied to a wide range of edge intelligent applications, it is critical for edge inference platforms to have both high-throughput and low-latency at the same time.
Identity Encoder for Personalized Diffusion
Our approach greatly reduces the overhead for personalized image generation and is more applicable in many potential applications.
DATE: Domain Adaptive Product Seeker for E-commerce
Then, we present two cooperative seekers to simultaneously search the image for PR and localize the product for PG.
Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models
This paper proposes a method for generating images of customized objects specified by users.
Harmonizing Base and Novel Classes: A Class-Contrastive Approach for Generalized Few-Shot Segmentation
Current methods for few-shot segmentation (FSSeg) have mainly focused on improving the performance of novel classes while neglecting the performance of base classes.
CoopInit: Initializing Generative Adversarial Networks via Cooperative Learning
The proposed algorithm consists of two learning stages: (i) Cooperative initialization stage: The discriminator of GAN is treated as an energy-based model (EBM) and is optimized via maximum likelihood estimation (MLE), with the help of the GAN's generator to provide synthetic data to approximate the learning gradients.
Deformation measurement of a soil mixing retaining wall using terrestrial laser scanning
Retaining walls are often built to prevent excessive lateral movements of the ground surrounding an excavation site.
Revisiting the Stack-Based Inverse Tone Mapping
In this paper, we revisit the stack-based ITM approaches and propose a novel method to reconstruct HDR radiance from a single image, which only needs to estimate two exposure images.
Dilation-Erosion for Single-Frame Supervised Temporal Action Localization
To balance the annotation labor and the granularity of supervision, single-frame annotation has been introduced in temporal action localization.
Life-long Learning for Multilingual Neural Machine Translation with Knowledge Distillation
To alleviate the CF, we investigate knowledge distillation based life-long learning methods.
Stereo Image Rain Removal via Dual-View Mutual Attention
Stereo images, containing left and right view images with disparity, are utilized in solving low-vision tasks recently, e. g., rain removal and super-resolution.
Boosting Semi-Supervised 3D Object Detection with Semi-Sampling
When training a teacher-student semi-supervised framework, we randomly select gt samples and pseudo samples to both labeled frames and unlabeled frames, making a strong data augmentation for them.
NASA: Neural Architecture Search and Acceleration for Hardware Inspired Hybrid Networks
Multiplication is arguably the most cost-dominant operation in modern deep neural networks (DNNs), limiting their achievable efficiency and thus more extensive deployment in resource-constrained applications.
ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design
Specifically, on the algorithm level, ViTCoD prunes and polarizes the attention maps to have either denser or sparser fixed patterns for regularizing two levels of workloads without hurting the accuracy, largely reducing the attention computations while leaving room for alleviating the remaining dominant data movements; on top of that, we further integrate a lightweight and learnable auto-encoder module to enable trading the dominant high-cost data movements for lower-cost computations.
Behavioral graph fraud detection in E-commerce
Our experiments show that embedding features learned from similarity based behavioral graph have achieved significant performance increase to the baseline fraud detection model in various business scenarios.
CoopHash: Cooperative Learning of Multipurpose Descriptor and Contrastive Pair Generator via Variational MCMC Teaching for Supervised Image Hashing
Leveraging supervised information can lead to superior retrieval performance in the image hashing domain but the performance degrades significantly without enough labeled data.
Improving End-to-End Text Image Translation From the Auxiliary Text Translation Task
End-to-end text image translation (TIT), which aims at translating the source language embedded in images to the target language, has attracted intensive attention in recent research.
Video-Guided Curriculum Learning for Spoken Video Grounding
To rectify the discriminative phonemes and extract video-related information from noisy audio, we develop a novel video-guided curriculum learning (VGCL) during the audio pre-training process, which can make use of the vital visual perceptions to help understand the spoken language and suppress the external noise.
Learning an Efficient Multimodal Depth Completion Model
With the wide application of sparse ToF sensors in mobile devices, RGB image-guided sparse depth completion has attracted extensive attention recently, but still faces some problems.
Depth-Assisted ResiDualGAN for Cross-Domain Aerial Images Semantic Segmentation
Generative methods are common approaches to minimizing the domain gap of aerial images which improves the performance of the downstream tasks, e. g., cross-domain semantic segmentation.
e-G2C: A 0.14-to-8.31 $μ$J/Inference NN-based Processor with Continuous On-chip Adaptation for Anomaly Detection and ECG Conversion from EGM
This work presents the first silicon-validated dedicated EGM-to-ECG (G2C) processor, dubbed e-G2C, featuring continuous lightweight anomaly detection, event-driven coarse/precise conversion, and on-chip adaptation.
Turning to a Teacher for Timestamp Supervised Temporal Action Segmentation
To obtain more information to optimize the model, the existing method generated pseudo frame-wise labels iteratively based on the output of a segmentation model and the timestamp annotations.
AntPivot: Livestream Highlight Detection via Hierarchical Attention Mechanism
In recent days, streaming technology has greatly promoted the development in the field of livestream.
BRIGHT -- Graph Neural Networks in Real-Time Fraud Detection
Apart from rule-based and machine learning filters that are already deployed in production, we want to enable efficient real-time inference with graph neural networks (GNNs), which is useful to catch multihop risk propagation in a transaction graph.
Age Minimization in Outdoor and Indoor Communications with Relay-aided Dual RIS
In this paper, we investigate an outdoor and indoor wireless communication network with the assistance of a novel relay-aided double-sided reconfigurable intelligent surface (RIS).
Indoor simultaneous localization and mapping based on fringe projection profilometry
Simultaneous Localization and Mapping (SLAM) plays an important role in outdoor and indoor applications ranging from autonomous driving to indoor robotics.
Modelling graph dynamics in fraud detection with "Attention"
At online retail platforms, detecting fraudulent accounts and transactions is crucial to improve customer experience, minimize loss, and avoid unauthorized transactions.
Randomized Sharpness-Aware Training for Boosting Computational Efficiency in Deep Learning
Optimizers in RST would perform a Bernoulli trial at each iteration to choose randomly from base algorithms (SGD) and sharpness-aware algorithms (SAM) with a probability arranged by a predefined scheduling function.
Spatio-temporal Gait Feature with Global Distance Alignment
In this paper, we try to enhance the discrimination of spatio-temporal gait features from two aspects: effective extraction of spatio-temporal gait features and reasonable refinement of extracted features.
Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning
In this paper, we propose an effective method to improve the model generalization by additionally penalizing the gradient norm of loss function during optimization.
ResiDualGAN: Resize-Residual DualGAN for Cross-Domain Remote Sensing Images Semantic Segmentation
The performance of a semantic segmentation model for remote sensing (RS) images pretrained on an annotated dataset would greatly decrease when testing on another unannotated dataset because of the domain gap.
Neighborhood Region Smoothing Regularization for Finding Flat Minima In Deep Neural Networks
NRS leverages the finding that models would benefit from converging to flat minima, and tries to regularize the neighborhood region in weight space to yield approximate outputs.
Equivalence between algorithmic instability and transition to replica symmetry breaking in perceptron learning systems
Binary perceptron is a fundamental model of supervised learning for the non-convex optimization, which is a root of the popular deep learning.
Graph Neural Networks in Real-Time Fraud Detection with Lambda Architecture
Transaction checkout fraud detection is an essential risk control components for E-commerce marketplaces.
Rethinking Deep Face Restoration
While existing approaches for face restoration make significant progress in generating high-quality faces, they often fail to preserve facial features and cannot authentically reconstruct the faces.
TransTCN: An Attention-based TCN Framework for Sequential Modeling
The model was evaluated across a wide range of the tasks in time series, which are commonly used to the benchmark of TCN and recurrent networks.
Multi-frame Joint Enhancement for Early Interlaced Videos
Early interlaced videos usually contain multiple and interlacing and complex compression artifacts, which significantly reduce the visual quality.
Ranked #1 on
Video Deinterlacing
on MSU Deinterlacer Benchmark
D$^2$-GCN: Data-Dependent GCNs for Boosting Both Efficiency and Scalability
Graph Convolutional Networks (GCNs) have gained an increasing attention thanks to their state-of-the-art (SOTA) performance in graph-based learning tasks.
2-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiency
The recent breakthroughs of deep neural networks (DNNs) and the advent of billions of Internet of Things (IoT) devices have excited an explosive demand for intelligent IoT devices equipped with domain-specific DNN accelerators.
EAR-NET: Error Attention Refining Network For Retinal Vessel Segmentation
The precise detection of blood vessels in retinal images is crucial to the early diagnosis of the retinal vascular diseases, e. g., diabetic, hypertensive and solar retinopathies.
Cascaded Prediction Network via Segment Tree for Temporal Video Grounding
Temporal video grounding aims to localize the target segment which is semantically aligned with the given sentence in an untrimmed video.
Multi-Scale Context Aggregation Network with Attention-Guided for Crowd Counting
In this paper, we propose a multi-scale context aggregation network (MSCANet) based on single-column encoder-decoder architecture for crowd counting, which consists of an encoder based on a dense context-aware module (DCAM) and a hierarchical attention-guided decoder.
Estimating the Generalization in Deep Neural Networks via Sparsity
By training DNNs with a wide range of generalization gap on popular datasets, we show that our key quantities and linear model could be efficient tools for estimating the generalization gap of DNNs.
Super-Resolving Compressed Video in Coding Chain
Scaling and lossy coding are widely used in video transmission and storage.
HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark
To design HW-NAS-Bench, we carefully collected the measured/estimated hardware performance of all the networks in the search spaces of both NAS-Bench-201 and FBNet, on six hardware devices that fall into three categories (i. e., commercial edge devices, FPGA, and ASIC).
Hardware Aware Neural Architecture Search
Neural Architecture Search
Quantitative Performance Assessment of CNN Units via Topological Entropy Calculation
We show that by investigating the feature entropy of units on only training data, it could give discrimination between networks with different generalization ability from the view of the effectiveness of feature representations.
Interaction between optical pulse and tumor using finite element analysis
As a non-destructive biological tissue imaging technology, photoacoustic imaging has important application value in the field of biomedicine.
Biological Physics
Waveform and Beamforming Design for Intelligent Reflecting Surface Aided Wireless Power Transfer: Single-User and Multi-User Solutions
This paper highlights the fact that IRS can provide an extra passive beamforming gain on output DC power over conventional WPT designs and significantly influence the waveform design by leveraging the benefit of passive beamforming, frequency diversity and energy harvester nonlinearity.
Information Theory Signal Processing Information Theory
SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training
Results show that: 1) applied to inference, SD achieves up to 2. 44x energy efficiency as evaluated via real hardware implementations; 2) applied to training, SD leads to 10. 56x and 4. 48x reduction in the storage and training energy, with negligible accuracy loss compared to state-of-the-art training baselines.
SDA: Improving Text Generation with Self Data Augmentation
Data augmentation has been widely used to improve deep neural networks in many research fields, such as computer vision.
HW-NAS-Bench: Hardware-Aware Neural Architecture Search Benchmark
To design HW-NAS-Bench, we carefully collected the measured/estimated hardware performance (e. g., energy cost and latency) of all the networks in the search space of both NAS-Bench-201 and FBNet, considering six hardware devices that fall into three categories (i. e., commercial edge devices, FPGA, and ASIC).
Hardware Aware Neural Architecture Search
Neural Architecture Search
Benchmark Platform for Ultra-Fine-Grained Visual Categorization Beyond Human Performance
The proposed UFG image dataset and evaluation protocols is intended to serve as a benchmark platform that can advance research of visual classification from approaching human performance to beyond human ability, via facilitating benchmark data of artificial intelligence (AI) not to be limited by the labels of human intelligence (HI).
Learning Energy-Based Generative Models via Coarse-to-Fine Expanding and Sampling
Energy-based models (EBMs) for generative modeling parametrize a single net and can be directly trained by maximum likelihood estimation.
FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training
Recent breakthroughs in deep neural networks (DNNs) have fueled a tremendous demand for intelligent edge devices featuring on-site learning, while the practical realization of such systems remains a challenge due to the limited resources available at the edge and the required massive training costs for state-of-the-art (SOTA) DNNs.
A Comprehensive Survey of 6G Wireless Communications
First, we give an overview of 6G from perspectives of technologies, security and privacy, and applications.
Suspicious Massive Registration Detection via Dynamic Heterogeneous Graph Neural Networks
Massive account registration has raised concerns on risk management in e-commerce companies, especially when registration increases rapidly within a short time frame.
IRS-Aided SWIPT: Joint Waveform, Active and Passive Beamforming Design Under Nonlinear Harvester Model
To facilitate practical implementation, we also propose a low-complexity design based on closed-form adaptive waveform schemes.
Information Theory Signal Processing Information Theory
ReMP: Rectified Metric Propagation for Few-Shot Learning
Few-shot learning features the capability of generalizing from a few examples.
Knowledge Graph Enhanced Neural Machine Translation via Multi-task Learning on Sub-entity Granularity
Previous studies combining knowledge graph (KG) with neural machine translation (NMT) have two problems: i) Knowledge under-utilization: they only focus on the entities that appear in both KG and training sentence pairs, making much knowledge in KG unable to be fully utilized.
Unpaired Image-to-Image Translation via Latent Energy Transport
Instead of explicitly extracting the two codes and applying adaptive instance normalization to combine them, our latent EBM can implicitly learn to transport the source style code to the target style code while preserving the content code, an advantage over existing image translation methods.
Rethinking deinterlacing for early interlaced videos
Traditional deinterlacing approaches are mainly focused on early interlacing scanning systems and thus cannot handle the complex and complicated artifacts in real-world early interlaced videos.
xFraud: Explainable Fraud Transaction Detection
At online retail platforms, it is crucial to actively detect the risks of transactions to improve customer experience and minimize financial loss.
Role Taxonomy of Units in Deep Neural Networks
Identifying the role of network units in deep neural networks (DNNs) is critical in many aspects including giving understandings on the mechanisms of DNNs and building basic connections between deep learning and neuroscience.
Deconstruct to Reconstruct a Configurable Evaluation Metric for Open-Domain Dialogue Systems
For instance, specificity is mandatory in a food-ordering dialogue task, whereas fluency is preferred in a language-teaching dialogue system.
Dynamic Context Selection for Document-level Neural Machine Translation via Reinforcement Learning
Specifically, we introduce a selection module that is independent of the translation module to score each candidate context sentence.
Q-learning with Language Model for Edit-based Unsupervised Summarization
Q-learning is leveraged to train the agent to produce proper edit actions.
FaultNet: A Deep Convolutional Neural Network for bearing fault classification
In this work, we analyze vibration signal data of mechanical systems with bearings by combining different signal processing methods and coupling them with machine learning techniques to classify different types of bearing faults.
Ranked #3 on
Classification
on CWRU Bearing Dataset
(using extra training data)
Structure-Aware Human-Action Generation
Generating long-range skeleton-based human actions has been a challenging problem since small deviations of one frame can cause a malformed action sequence.
Ranked #2 on
Human action generation
on NTU RGB+D 2D
CASIA's System for IWSLT 2020 Open Domain Translation
This paper describes the CASIA{'}s system for the IWSLT 2020 open domain translation task.
Attacks to Federated Learning: Responsive Web User Interface to Recover Training Data from User Gradients
In a case where the aggregator is untrusted and LDP is not applied to each user gradient, the aggregator can recover sensitive user data from these gradients.
SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation
We present SmartExchange, an algorithm-hardware co-design framework to trade higher-cost memory storage/access for lower-cost computation, for energy-efficient inference of deep neural networks (DNNs).
TIMELY: Pushing Data Movements and Interfaces in PIM Accelerators Towards Local and in Time Domain
Resistive-random-access-memory (ReRAM) based processing-in-memory (R$^2$PIM) accelerators show promise in bridging the gap between Internet of Thing devices' constrained resources and Convolutional/Deep Neural Networks' (CNNs/DNNs') prohibitive energy cost.
Bayesian Meta Sampling for Fast Uncertainty Adaptation
Specifically, we propose a Bayesian meta sampling framework consisting of two main components: a meta sampler and a sample adapter.
Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck
Variational autoencoders (VAEs) are essential tools in end-to-end representation learning.
Local Differential Privacy based Federated Learning for Internet of Things
To avoid the privacy threat and reduce the communication cost, in this paper, we propose to integrate federated learning and local differential privacy (LDP) to facilitate the crowdsourcing applications to achieve the machine learning model.
Feature Quantization Improves GAN Training
The instability in GAN training has been a long-standing problem despite remarkable research efforts.
Ranked #1 on
Image-to-Image Translation
on anime-to-selfie
Dual-discriminator GAN: A GAN way of profile face recognition
A wealth of angle problems occur when facial recognition is performed: At present, the feature extraction network presents eigenvectors with large differences between the frontal face and profile face recognition of the same person in many cases.
A New MRAM-based Process In-Memory Accelerator for Efficient Neural Network Training with Floating Point Precision
The excellent performance of modern deep neural networks (DNNs) comes at an often prohibitive training cost, limiting the rapid development of DNN innovations and raising various environmental concerns.
DNN-Chip Predictor: An Analytical Performance Predictor for DNN Accelerators with Various Dataflows and Hardware Architectures
The recent breakthroughs in deep neural networks (DNNs) have spurred a tremendously increased demand for DNN accelerators.
AutoDNNchip: An Automated DNN Chip Predictor and Builder for Both FPGAs and ASICs
Specifically, AutoDNNchip consists of two integrated enablers: (1) a Chip Predictor, built on top of a graph-based accelerator representation, which can accurately and efficiently predict a DNN accelerator's energy, throughput, and area based on the DNN model parameters, hardware configuration, technology-based IPs, and platform constraints; and (2) a Chip Builder, which can automatically explore the design space of DNN chips (including IP selection, block configuration, resource balancing, etc.
