Search Results for author:
Found 84 papers, 26 papers with code
ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Model
Humans possess a unified cognitive ability to perceive, comprehend, and interact with the physical world.
DeFiScope: Detecting Various DeFi Price Manipulations with LLM Reasoning
DeFi (Decentralized Finance) is one of the most important applications of today's cryptocurrencies and smart contracts.
Personalized Language Model Learning on Text Data Without User Identifiers
In many practical natural language applications, user data are highly sensitive, requiring anonymous uploads of text data from mobile devices to the cloud without user identifiers.
RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation
To the best of our knowledge, RoboMIND is the largest multi-embodiment teleoperation dataset collected on a unified platform, providing large-scale and high-quality robotic training data.
EDT: An Efficient Diffusion Transformer Framework Inspired by Human-like Sketching
Transformer-based Diffusion Probabilistic Models (DPMs) have shown more potential than CNN-based DPMs, yet their extensive computational requirements hinder widespread practical applications.
Disentangled Representation Learning for Parametric Partial Differential Equations
Specifically, we introduce DisentangO, a novel hyper-neural operator architecture designed to unveil and disentangle the latent physical factors of variation embedded within the black-box neural operator parameters.
AlterMOMA: Fusion Redundancy Pruning for Camera-LiDAR Fusion Models with Alternative Modality Masking
The redundant parameters can be pruned by our proposed importance score evaluation function, Alternative Evaluation (AlterEva), which is based on the observation of the loss changes when certain modality parameters are activated and deactivated.
TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation
Vision-Language-Action (VLA) models have shown remarkable potential in visuomotor control and instruction comprehension through end-to-end learning processes.
Nonlocal Attention Operator: Materializing Hidden Knowledge Towards Interpretable Physics Discovery
In this work, we propose a novel neural operator architecture based on the attention mechanism, which we coin Nonlocal Attention Operator (NAO), and explore its capability towards developing a foundation physical model.
Autoregressive Enzyme Function Prediction with Multi-scale Multi-modality Fusion
Accurate prediction of enzyme function is crucial for elucidating biological mechanisms and driving innovation across various sectors.
Large language models, physics-based modeling, experimental measurements: the trinity of data-scarce learning of polymer properties
Our framework features a two-phase training strategy: (1) utilizing the large-in-amount while less accurate synthetic data for supervised pretraining, and (2) finetuning the phase-1 model with limited experimental data.
MMRo: Are Multimodal LLMs Eligible as the Brain for In-Home Robotics?
It is fundamentally challenging for robots to serve as useful assistants in human environments because this requires addressing a spectrum of sub-problems across robotics, including perception, language understanding, reasoning, and planning.
SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner
Jailbreaking is an emerging adversarial attack that bypasses the safety alignment deployed in off-the-shelf large language models (LLMs) and has evolved into multiple categories: human-based, optimization-based, generation-based, and the recent indirect and multilingual jailbreaks.
Deep Neural Operator Enabled Digital Twin Modeling for Additive Manufacturing
In terms of Laser Powder Bed Fusion (L-PBF) based additive manufacturing (AM), a DT can predict the current and future states of the melt pool and the resulting defects corresponding to the input laser parameters, evolve itself by assimilating in-situ sensor data, and optimize the laser parameters to mitigate defect formation.
STMGF: An Effective Spatial-Temporal Multi-Granularity Framework for Traffic Forecasting
Accurate Traffic Prediction is a challenging task in intelligent transportation due to the spatial-temporal aspects of road networks.
Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment
Based on T2VQA-DB, we propose a novel transformer-based model for subjective-aligned Text-to-Video Quality Assessment (T2VQA).
2023 Low-Power Computer Vision Challenge (LPCVC) Summary
This article describes the 2023 IEEE Low-Power Computer Vision Challenge (LPCVC).
Mipha: A Comprehensive Overhaul of Multimodal Assistant with Small Language Models
Multimodal Large Language Models (MLLMs) have showcased impressive skills in tasks related to visual understanding and reasoning.
Ranked #183 on
Visual Question Answering
on MM-Vet
LLMs Can Defend Themselves Against Jailbreaking in a Practical Manner: A Vision Paper
Our key insight is that regardless of the kind of jailbreak strategies employed, they eventually need to include a harmful prompt (e. g., "how to make a bomb") in the prompt sent to LLMs, and we found that existing LLMs can effectively recognize such harmful prompts that violate their safety policies.
EPSD: Early Pruning with Self-Distillation for Efficient Model Compression
Instead of a simple combination of pruning and SD, EPSD enables the pruned network to favor SD by keeping more distillable weights before training to ensure better distillation of the pruned network.
Learning from Imperfect Demonstrations with Self-Supervision for Robotic Manipulation
High-quality segments from the failed data are used to expand the training dataset.
Assessing Large Language Models in Mechanical Engineering Education: A Study on Mechanics-Focused Conceptual Understanding
This study is a pioneering endeavor to investigate the capabilities of Large Language Models (LLMs) in addressing conceptual questions within the domain of mechanical engineering with a focus on mechanics.
Peridynamic Neural Operators: A Data-Driven Nonlocal Constitutive Model for Complex Material Responses
In this work, we introduce a novel integral neural operator architecture called the Peridynamic Neural Operator (PNO) that learns a nonlocal constitutive law from data.
LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model
In this paper, we introduce LLaVA-$\phi$ (LLaVA-Phi), an efficient multi-modal assistant that harnesses the power of the recently advanced small language model, Phi-2, to facilitate multi-modal dialogues.
Ranked #212 on
Visual Question Answering
on MM-Vet
FCDNet: Frequency-Guided Complementary Dependency Modeling for Multivariate Time-Series Forecasting
Additionally, adopting a frequency-based perspective can effectively mitigate the influence of noise within MTS data, which helps capture more genuine dependencies.
Harnessing the Power of Neural Operators with Automatically Encoded Conservation Laws
In this work, we introduce conservation law-encoded neural operators (clawNOs), a suite of NOs that endow inference with automatic satisfaction of such conservation laws.
Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites
The fine-grained object attributes and behaviors non-existent in the image may still be generated but not measured by the current evaluation methods.
MV-CLIP: Multi-View CLIP for Zero-shot 3D Shape Recognition
Consequently, this paper aims to improve the confidence with view selection and hierarchical prompts.
Large Language Models for Robotics: A Survey
Understanding and assessing this intelligence is a complex task.
Learning Interpretable Rules for Scalable Data Representation and Classification
Rule-based models, e. g., decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity.
FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering
Knowledge base question answering (KBQA) is a critical yet challenging task due to the vast number of entities within knowledge bases and the diversity of natural language questions posed by users.
FLAME-based Multi-View 3D Face Reconstruction
Our work mainly addresses the problem of combining parametric models of faces with multi-view face 3D reconstruction and explores the implementation of a Flame based multi-view training and testing framework for contributing to the field of face 3D reconstruction.
StableVQA: A Deep No-Reference Quality Assessment Model for Video Stability
Indeed, most existing quality assessment models evaluate video quality as a whole without specifically taking the subjective experience of video stability into consideration.
Do-GOOD: Towards Distribution Shift Evaluation for Pre-Trained Visual Document Understanding Models
Results from the experiments demonstrate that there is a significant performance gap between the in-distribution (ID) and OOD settings for document images, and that fine-grained analysis of distribution shifts can reveal the brittle nature of existing pre-trained VDU models and OOD generalization algorithms.
Can LLMs like GPT-4 outperform traditional AI tools in dementia diagnosis? Maybe, but not today
In this paper, we explore the potential of LLMs such as GPT-4 to outperform traditional AI tools in dementia diagnosis.
Fascinating Supervisory Signals and Where to Find Them: Deep Anomaly Detection with Scale Learning
Due to the unsupervised nature of anomaly detection, the key to fueling deep models is finding supervisory signals.
T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answering
To address these issues, we propose a novel method termed T-SciQ that aims at teaching science question answering with LLM signals.
Bridging the Language Gap: Knowledge Injected Multilingual Question Answering
As a popular topic in natural language processing tasks, extractive question answering task (extractive QA) has gained extensive attention in the past few years.
CP$^3$: Channel Pruning Plug-in for Point-based Networks
CP$^3$ is elaborately designed to leverage the characteristics of point clouds and PNNs in order to enable 2D channel pruning methods for PNNs.
CMG-Net: An End-to-End Contact-Based Multi-Finger Dexterous Grasping Network
Our work significantly outperforms the state-of-the-art for three-finger robotic hands.
ICL-D3IE: In-Context Learning with Diverse Demonstrations Updating for Document Information Extraction
To this end, we propose a simple but effective in-context learning framework called ICL-D3IE, which enables LLMs to perform DIE with different types of demonstration examples.
Gauges and Accelerated Optimization over Smooth and/or Strongly Convex Sets
We consider feasibility and constrained optimization problems defined over smooth and/or strongly convex sets.
CP3: Channel Pruning Plug-In for Point-Based Networks
Directly implementing the 2D CNN channel pruning methods to PNNs undermine the performance of PNNs because of the different representations of 2D images and 3D point clouds as well as the network architecture disparity.
ScaleKD: Distilling Scale-Aware Knowledge in Small Object Detector
Unlike existing works that struggle to balance the trade-off between inference speed and SOD performance, in this paper, we propose a novel Scale-aware Knowledge Distillation (ScaleKD), which transfers knowledge of a complex teacher model to a compact student model.
INO: Invariant Neural Operators for Learning Complex Physical Systems with Momentum Conservation
Neural operators, which emerge as implicit solution operators of hidden governing equations, have recently become popular tools for learning responses of complex real-world physical systems.
Alignment-Enriched Tuning for Patch-Level Pre-trained Document Image Models
In this paper, we propose a new model architecture with alignment-enriched tuning (dubbed AETNet) upon pre-trained document image models, to adapt downstream tasks with the joint task-specific supervised and alignment-aware contrastive objective.
CAEN: A Hierarchically Attentive Evolution Network for Item-Attribute-Change-Aware Recommendation in the Growing E-commerce Environment
Traditional recommendation systems mainly focus on modeling user interests.
FuncFooler: A Practical Black-box Attack Against Learning-based Binary Code Similarity Detection Methods
The binary code similarity detection (BCSD) method measures the similarity of two binary executable codes.
Effective Few-Shot Named Entity Linking by Meta-Learning
In this paper, we endeavor to solve the problem of few-shot entity linking, which only requires a minimal amount of in-domain labeled data and is more practical in real situations.
An Automatic and Efficient BERT Pruning for Edge AI Systems
On MRPC, we obtain a 4. 6 higher score than the SOTA at the same overall pruning ratio of 0. 5.
A No-reference Quality Assessment Metric for Point Cloud Based on Captured Video Sequences
Point cloud is one of the most widely used digital formats of 3D models, the visual quality of which is quite sensitive to distortions such as downsampling, noise, and compression.
Differentiable Multi-Agent Actor-Critic for Multi-Step Radiology Report Summarization
A cascade of tasks are required to automatically generate an abstractive summary of the typical information-rich radiology report.
A Computational Efficient Maximum Likelihood Direct Position Determination Approach for Multiple Emitters Using Angle and Doppler Measurements
Emitter localization is widely applied in the military and civilian _elds.
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Systematical evaluation on accuracy, training speed, and memory footprint are conducted, where the proposed MEST framework consistently outperforms representative SOTA works.
CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization
Deep convolutional neural networks are shown to be overkill with high parametric and computational redundancy in many application scenarios, and an increasing number of works have explored model pruning to obtain lightweight and efficient networks.
Scalable Rule-Based Representation Learning for Interpretable Classification
Rule-based models, e. g., decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity.
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Based on our analysis, we summarize a guideline for parameter settings in regards of specific architecture characteristics, which we hope to catalyze the research progress on the topic of lottery ticket hypothesis.
Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI
More importantly, our method does not require extra hardware cost compared to the traditional two-column mapping scheme.
Beyond Outlier Detection: Outlier Interpretation by Attention-Guided Triplet Deviation Network
We obtain an optimal attention-guided embedding space with expanded high-level information and rich semantics, and thus outlying behaviors of the queried outlier can be better unfolded.
Hierarchical Adaptive Pooling by Capturing High-order Dependency for Graph Representation Learning
Graph neural networks (GNN) have been proven to be mature enough for handling graph-structured data on node-level graph representation learning tasks.
IPAPRec: A promising tool for learning high-performance mapless navigation skills with deep reinforcement learning
This paper studies how to improve the generalization performance and learning speed of the navigation agents trained with deep reinforcement learning (DRL).
Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?
In deep model compression, the recent finding "Lottery Ticket Hypothesis" (LTH) (Frankle & Carbin, 2018) pointed out that there could exist a winning ticket (i. e., a properly pruned sub-network together with original weight initialization) that can achieve competitive performance than the original dense network.
RRL: A Scalable Classifier for Interpretable Rule-Based Representation Learning
Rule-based models, e. g., decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity.
Shape Self-Correction for Unsupervised Point Cloud Understanding
Then the destroyed shape and the normal shape are sent into a point cloud network to get representations, which are employed to segment points that belong to distorted parts and further reconstruct them to restore the shape to normal.
DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search
The convolutional neural network has achieved great success in fulfilling computer vision tasks despite large computation overhead against efficient deployment.
Learn to Navigate Maplessly with Varied LiDAR Configurations: A Support Point-Based Approach
Deep reinforcement learning (DRL) demonstrates great potential in mapless navigation domain.
Dynamics of Momentum Distribution and Structure Factor in a Weakly Interacting Bose Gas with a Periodical Modulation
The momentum distribution and dynamical structure factor in a weakly interacting Bose gas with a time-dependent periodic modulation in terms of the Bogoliubov treatment are investigated.
Quantum Gases
Transparent Classification with Multilayer Logical Perceptrons and Random Binarization
In this paper, we propose a new hierarchical rule-based model for classification tasks, named Concept Rule Sets (CRS), which has both a strong expressive ability and a transparent inner structure.
Deep Compressed Pneumonia Detection for Low-Power Embedded Devices
Deep neural networks (DNNs) have been expanded into medical fields and triggered the revolution of some medical applications by extracting complex features and achieving high accuracy and performance, etc.
REQ-YOLO: A Resource-Aware, Efficient Quantization Framework for Object Detection on FPGAs
To achieve real-time, highly-efficient implementations on FPGA, we present the detailed hardware implementation of block circulant matrices on CONV layers and develop an efficient processing element (PE) structure supporting the heterogeneous weight quantization, CONV dataflow and pipelining techniques, design optimization, and a template-based automatic synthesis framework to optimally exploit hardware resource.
Universal Physical Camouflage Attacks on Object Detectors
In this paper, we study physical adversarial attacks on object detectors in the wild.
Distinguishing Individual Red Pandas from Their Faces
We are going to release our database and model in the public domain to promote the research on automatic animal identification and particularly on the technique for protecting red pandas.
A Stochastic-Computing based Deep Learning Framework using Adiabatic Quantum-Flux-Parametron SuperconductingTechnology
Further, the application of SC has been investigated in DNNs in prior work, and the suitability has been illustrated as SC is more compatible with approximate computations.
AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates
This work proposes AutoCompress, an automatic structured pruning framework with the following key performance improvements: (i) effectively incorporate the combination of structured pruning schemes in the automatic process; (ii) adopt the state-of-art ADMM-based structured weight pruning as the core algorithm, and propose an innovative additional purification step for further weight reduction without accuracy loss; and (iii) develop effective heuristic search method enhanced by experience-based guided search, replacing the prior deep reinforcement learning technique which has underlying incompatibility with the target pruning problem.
Knowledge-Aware Deep Dual Networks for Text-Based Mortality Prediction
To address the above issues, we propose novel Knowledge-aware Deep Dual Networks (K-DDN) for the text-based mortality prediction task.
ADCrowdNet: An Attention-injective Deformable Convolutional Network for Crowd Understanding
We propose an attention-injective deformable convolutional network called ADCrowdNet for crowd understanding that can address the accuracy degradation problem of highly congested noisy scenes.
Ranked #2 on
Crowd Counting
on TRANCOS
StructADMM: A Systematic, High-Efficiency Framework of Structured Weight Pruning for DNNs
Without loss of accuracy on the AlexNet model, we achieve 2. 58X and 3. 65X average measured speedup on two GPUs, clearly outperforming the prior work.
Structured Weight Matrices-Based Hardware Accelerators in Deep Neural Networks: FPGAs and ASICs
For FPGA implementations on deep convolutional neural networks (DCNNs), we achieve at least 152X and 72X improvement in performance and energy efficiency, respectively using the SWM-based framework, compared with the baseline of IBM TrueNorth processor under same accuracy constraints using the data set of MNIST, SVHN, and CIFAR-10.
VIBNN: Hardware Acceleration of Bayesian Neural Networks
In this paper, we propose VIBNN, an FPGA-based hardware accelerator design for variational inference on BNNs.
Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data
This paper presents the first deep reinforcement learning (DRL) framework to estimate the optimal Dynamic Treatment Regimes from observational medical data.
FFT-Based Deep Learning Deployment in Embedded Systems
The large model size of DNNs, while providing excellent accuracy, also burdens the embedded platforms with intensive computation and storage.
CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices
As the size of DNNs continues to grow, it is critical to improve the energy efficiency and performance while maintaining accuracy.
A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning
Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to (partially) solve the resource allocation problem adaptively in the cloud computing system.
