Search Results for author:
Found 80 papers, 37 papers with code
VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting
In this work, we propose VOTE, an efficient and general framework for the optimization and acceleration of VLA models.
LettinGo: Explore User Profile Generation for Recommendation System
This work enhances profile generation as a key innovation for next-generation recommendation systems.
Werewolf: A Straightforward Game Framework with TTS for Improved User Engagement
The growing popularity of social deduction game systems for both business applications and AI research has greatly benefited from the rapid advancements in Large Language Models (LLMs), which now demonstrate stronger reasoning and persuasion capabilities.
Enabling Flexible Multi-LLM Integration for Scalable Knowledge Aggregation
To address these issues, we propose a framework that adaptively selects and aggregates knowledge from diverse LLMs to build a single, stronger model, avoiding the high memory overhead of ensemble and inflexible weight merging.
RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning
Instead of relying on handcrafted rules or stylistic rewrites, our method trains a language model to generate structured, self-reflective prompts by optimizing for image-level outcomes.
Taming Diffusion for Dataset Distillation with High Representativeness
Specifically, we adopt DDIM inversion to map the latents of the full dataset from a low-normality latent domain to a high-normality Gaussian domain, preserving information and ensuring structural consistency to generate representative latents for the distilled dataset.
Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality
We highlight its potential to drive new model architectures and learning strategies that improve robustness, increase interpretability, and better align with the objectives of generative modeling.
Structured Agent Distillation for Large Language Model
Large language models (LLMs) exhibit strong capabilities as decision-making agents by interleaving reasoning and actions, as seen in ReAct-style frameworks.
FastCar: Cache Attentive Replay for Fast Auto-Regressive Video Generation on the Edge
In this paper, we propose the \textbf{FastCar} framework to accelerate the decode phase for the AR video generation by exploring the temporal redundancy.
DraftAttention: Fast Video Diffusion via Low-Resolution Attention Guidance
To address this, we propose the DraftAttention, a training-free framework for the acceleration of video diffusion transformers with dynamic sparse attention on GPUs.
UFO2: The Desktop AgentOS
Recent Computer-Using Agents (CUAs), powered by multimodal large language models (LLMs), offer a promising direction for automating complex desktop workflows through natural language.
QuartDepth: Post-Training Quantization for Real-Time Depth Estimation on the Edge
Monocular Depth Estimation (MDE) has emerged as a pivotal task in computer vision, supporting numerous real-world applications.
Lean and Mean: Decoupled Value Policy Optimization with Global Value Guidance
Proximal Policy Optimization (PPO)-based Reinforcement Learning from Human Feedback (RLHF) is essential for aligning large language models (LLMs) with human preferences.
Efficient Reasoning with Hidden Thinking
Meanwhile, we design corresponding Heima Decoder with traditional Large Language Models (LLMs) to adaptively interpret the hidden representations into variable-length textual sequence, reconstructing reasoning processes that closely resemble the original CoTs.
Skeleton-Guided-Translation: A Benchmarking Framework for Code Repository Translation with Fine-Grained Quality Evaluation
The advancement of large language models has intensified the need to modernize enterprise applications and migrate legacy systems to secure, versatile languages.
RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation
Although Low-Rank Adaptation (LoRA) is widely used and effective for fine-tuning, we have observed that its scaling factor can limit or even reduce performance as the rank size increases.
WarriorCoder: Learning from Expert Battles to Augment Code Large Language Models
Despite recent progress achieved by code large language models (LLMs), their remarkable abilities are largely dependent on fine-tuning on the high-quality data, posing challenges for data collection and annotation.
Numerical Pruning for Efficient Autoregressive Models
Transformers have emerged as the leading architecture in deep learning, proving to be versatile and highly effective across diverse domains beyond language and image processing.
LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformers
In this paper, we show that performing the full computation of the model at each diffusion step is unnecessary, as some computations can be skipped by lazily reusing the results of previous steps.
Large Action Models: From Inception to Implementation
As AI continues to advance, there is a growing demand for systems that go beyond language-based assistance and move toward intelligent agents capable of performing real-world actions.
Open-Source Acceleration of Stable-Diffusion.cpp Deployable on All Devices
To address this, stable-diffusion. cpp (Sdcpp) emerges as an efficient inference framework to accelerate the diffusion models.
Fully Open Source Moxin-7B Technical Report
Recently, Large Language Models (LLMs) have undergone a significant transformation, marked by a rapid rise in both their popularity and capabilities.
Fast and Memory-Efficient Video Diffusion Using Streamlined Inference
The rapid progress in artificial intelligence-generated content (AIGC), especially with diffusion models, has significantly advanced development of high-quality video generation.
Self-Evolved Reward Learning for LLMs
Reinforcement Learning from Human Feedback (RLHF) is a crucial technique for aligning language models with human preferences, playing a pivotal role in the success of conversational models like GPT-4, ChatGPT, and Llama 2.
Pruning Foundation Models for High Accuracy without Retraining
To deal with this problem, post-training pruning methods are proposed to prune LLMs in one-shot without retraining.
Rethinking Token Reduction for State Space Models
While token reduction techniques offer a straightforward post-training strategy, we find that applying existing methods directly to SSMs leads to substantial performance drops.
Lotus: learning-based online thermal and latency variation management for two-stage detectors on edge devices
Two-stage object detectors exhibit high accuracy and precise localization, especially for identifying small objects that are favorable for various edge applications.
Exploring Token Pruning in Vision State Space Models
Inspired by the observations that the final prediction in vision transformers (ViTs) is only based on a subset of most informative tokens, we take the novel step of enhancing the efficiency of SSM-based vision models through token-based pruning.
Search for Efficient Large Language Models
Large Language Models (LLMs) have long held sway in the realms of artificial intelligence research.
AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation
Moreover, to increase the difficulty diversity of generated planning tasks, we propose a bidirectional evolution method, Bi-Evol, that evolves planning tasks from easier and harder directions to synthesize a task set with a smoother difficulty curve.
Arena Learning: Build Data Flywheel for LLMs Post-training via Simulated Chatbot Arena
In this paper, we introduce Arena Learning, an innovative offline strategy designed to simulate these arena battles using AI-driven annotations to evaluate battle outcomes, thus facilitating the continuous improvement of the target model through both supervised fine-tuning and reinforcement learning.
Thread: A Logic-Based Data Organization Paradigm for How-To Question Answering with Retrieval Augmented Generation
Recent advances in retrieval-augmented generation have significantly improved the performance of question-answering systems, particularly on factoid '5Ws' questions.
Toward Adaptive Large Language Models Structured Pruning via Hybrid-grained Weight Importance Assessment
For example, HyWIA surpasses the cutting-edge LLM-Pruner by an average margin of 2. 82% in accuracy across seven downstream tasks when pruning LLaMA-7B by 50%.
InstructGIE: Towards Generalizable Image Editing
In response to this challenge, our study introduces a novel image editing framework with enhanced generalization robustness by boosting in-context learning capability and unifying language instruction.
DiffClass: Diffusion-Based Class Incremental Learning
On top of that, Exemplar-free Class Incremental Learning is even more challenging due to forbidden access to previous task data.
Nissist: An Incident Mitigation Copilot based on Troubleshooting Guides
Effective incident management is pivotal for the smooth operation of enterprises-level cloud services.
EdgeQAT: Entropy and Distribution Guided Quantization-Aware Training for the Acceleration of Lightweight LLMs on the Edge
In this paper, we propose EdgeQAT, the Entropy and Distribution Guided QAT for the optimization of lightweight LLMs to achieve inference acceleration on Edge devices.
Detection and Recovery Against Deep Neural Network Fault Injection Attacks Based on Contrastive Learning
Deep Neural Network (DNN) models when implemented on executing devices as the inference engines are susceptible to Fault Injection Attacks (FIAs) that manipulate model parameters to disrupt inference execution with disastrous performance.
Contrastive Learning with Negative Sampling Correction
To correct the negative sampling bias, we propose a novel contrastive learning method named Positive-Unlabeled Contrastive Learning (PUCL).
Why does Prediction Accuracy Decrease over Time? Uncertain Positive Learning for Cloud Failure Prediction
Using two real-world datasets of disk failure prediction and conducting node prediction experiments in Microsoft Azure, which is a top-tier cloud provider that serves millions of users, we demonstrate Uptake can significantly improve the failure prediction accuracy by 5% on average.
TaskWeaver: A Code-First Agent Framework
TaskWeaver provides support for rich data structures, flexible plugin usage, and dynamic plugin selection, and leverages LLM coding capabilities for complex logic.
Self-Guard: Empower the LLM to Safeguard Itself
Currently, there are two main approaches to address jailbreak attacks: safety training and safeguards.
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
Large language models (LLMs), such as GPT-4, have shown remarkable performance in natural language processing (NLP) tasks, including challenging mathematical reasoning.
Ranked #54 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Diffusion-based Time Series Data Imputation for Microsoft 365
However, they suffer from poor data quality like data missing in model training and prediction, which limits the performance.
Robust Positive-Unlabeled Learning via Noise Negative Sample Self-correction
To mitigate the impact of label uncertainty and improve the robustness of learning with positive and unlabeled data, we propose a new robust PU learning method with a training strategy motivated by the nature of human learning: easy cases should be learned first.
WizardCoder: Empowering Code Large Language Models with Evol-Instruct
Moreover, our model even outperforms the largest closed LLMs, Anthropic's Claude and Google's Bard, on HumanEval and HumanEval+.
Ranked #8 on
Code Generation
on CodeContests
Empower Large Language Model to Perform Better on Industrial Domain-Specific Question Answering
Large Language Model (LLM) has gained popularity and achieved remarkable results in open-domain tasks, but its performance in real industrial domain-specific scenarios is average due to its lack of specific domain knowledge.
Introspective Tips: Large Language Model for In-Context Decision Making
The emergence of large language models (LLMs) has substantially influenced natural language processing, demonstrating exceptional results across various tasks.
Augmented Large Language Models with Parametric Knowledge Guiding
We demonstrate that our PKG framework can enhance the performance of "black-box" LLMs on a range of domain knowledge-intensive tasks that require factual (+7. 9%), tabular (+11. 9%), medical (+3. 0%), and multimodal (+8. 1%) knowledge.
WizardLM: Empowering Large Language Models to Follow Complex Instructions
In this paper, we show an avenue for creating large amounts of instruction data with varying levels of complexity using LLM instead of humans.
Less is More: Data Pruning for Faster Adversarial Training
Deep neural networks (DNNs) are sensitive to adversarial examples, resulting in fragile and unreliable performance in the real world.
LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Retrieval
The conventional dense retrieval paradigm relies on encoding images and texts into dense representations using dual-stream encoders, however, it faces challenges with low retrieval speed in large-scale retrieval scenarios.
LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Sparse Retrieval
To address this issue, we propose a novel sparse retrieval paradigm for ITR that exploits sparse representations in the vocabulary space for images and texts.
Pruning Parameterization With Bi-Level Optimization for Efficient Semantic Segmentation on the Edge
With the ever-increasing popularity of edge devices, it is necessary to implement real-time segmentation on the edge for autonomous driving and many other applications.
All-in-One: A Highly Representative DNN Pruning Framework for Edge Devices with Dynamic Power Management
By re-configuring the model to the corresponding pruning ratio for a specific execution frequency (and voltage), we are able to achieve stable inference speed, i. e., keeping the difference in speed performance under various execution frequencies as small as possible.
MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation
First, it is the largest multi-modal conversation dataset by the number of dialogues by 88x.
Ranked #2 on
Multimodal Intent Recognition
on MMDialog
Advancing Model Pruning via Bi-level Optimization
To reduce the computation overhead, various efficient 'one-shot' pruning methods have been developed, but these schemes are usually unable to find winning tickets as good as IMP.
Efficient Multi-Prize Lottery Tickets: Enhanced Accuracy, Training, and Inference Speed
Recently, Diffenderfer and Kailkhura proposed a new paradigm for learning compact yet highly accurate binary neural networks simply by pruning and quantizing randomly weighted full precision neural networks.
Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution
Instead of measuring the speed on mobile devices at each iteration during the search process, a speed model incorporated with compiler optimizations is leveraged to predict the inference latency of the SR block with various width configurations for faster convergence.
Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization
By combining the structural reparameterization and PaS, we successfully searched out a new family of VGG-like and lightweight networks, which enable the flexibility of arbitrary width with respect to each layer instead of each stage.
Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration
Weight pruning is an effective model compression technique to tackle the challenges of achieving real-time deep neural network (DNN) inference on mobile devices.
Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search
Though recent years have witnessed remarkable progress in single image super-resolution (SISR) tasks with the prosperous development of deep neural networks (DNNs), the deep learning methods are confronted with the computation and memory consumption issues in practice, especially for resource-limited platforms such as mobile devices.
Achieving Real-Time Object Detection on MobileDevices with Neural Pruning Search
Object detection plays an important role in self-driving cars for security development.
A Compression-Compilation Framework for On-mobile Real-time BERT Applications
In this paper, we propose a compression-compilation co-design framework that can guarantee the identified model to meet both resource and real-time specifications of mobile devices.
High-Robustness, Low-Transferability Fingerprinting of Neural Networks
This paper proposes Characteristic Examples for effectively fingerprinting deep neural networks, featuring high-robustness to the base model against model pruning as well as low-transferability to unassociated models.
Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device
3D object detection is an important task, especially in the autonomous driving application domain.
NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
With the increasing demand to efficiently deploy DNNs on mobile edge devices, it becomes much more important to reduce unnecessary computation and increase the execution speed.
Learning to Generate Image Source-Agnostic Universal Adversarial Perturbations
The resulting scheme for meta-learning a UAP generator (i) has better performance (50% higher ASR) than baselines such as Projected Gradient Descent, (ii) has better performance (37% faster) than the vanilla L2O and MAML frameworks (when applicable), and (iii) is able to simultaneously handle UAP generation for different victim models and image data sources.
Real-Time Execution of Large-scale Language Models on Mobile
Our framework can guarantee the identified model to meet both resource and real-time specifications of mobile devices, thus achieving real-time execution of large transformer-based models like BERT variants.
Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness
In this work, we propose to employ mode connectivity in loss landscapes to study the adversarial robustness of deep neural networks, and provide novel methods for improving this robustness.
Towards Real-Time DNN Inference on Mobile Platforms with Model Pruning and Compiler Optimization
High-end mobile platforms rapidly serve as primary computing devices for a wide range of Deep Neural Network (DNN) applications.
Defending against Backdoor Attack on Deep Neural Networks
Although deep neural networks (DNNs) have achieved a great success in various computer vision tasks, it is recently found that they are vulnerable to adversarial attacks.
Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent
Despite the great achievements of the modern deep neural networks (DNNs), the vulnerability/robustness of state-of-the-art DNNs raises security concerns in many application domains requiring high reliability.
BLK-REW: A Unified Block-based DNN Pruning Framework using Reweighted Regularization Method
Accelerating DNN execution on various resource-limited computing platforms has been a long-standing problem.
On the Design of Black-box Adversarial Examples by Leveraging Gradient-free Optimization and Operator Splitting Method
Robust machine learning is currently one of the most prominent topics which could potentially help shaping a future of advanced AI platforms that not only perform well in average cases but also in worst cases or adverse situations.
Fault Sneaking Attack: a Stealthy Framework for Misleading Deep Neural Networks
Despite the great achievements of deep neural networks (DNNs), the vulnerability of state-of-the-art DNNs raises security concerns of DNNs in many application domains requiring high reliability. We propose the fault sneaking attack on DNNs, where the adversary aims to misclassify certain input images into any target labels by modifying the DNN parameters.
Interpreting Adversarial Examples by Activation Promotion and Suppression
It is widely known that convolutional neural networks (CNNs) are vulnerable to adversarial examples: images with imperceptible perturbations crafted to fool classifiers.
Defensive Dropout for Hardening Deep Neural Networks under Adversarial Attacks
Based on the observations of the effect of test dropout rate on test accuracy and attack success rate, we propose a defensive dropout algorithm to determine an optimal test dropout rate given the neural network model and the attacker's strategy for generating adversarial examples. We also investigate the mechanism behind the outstanding defense effects achieved by the proposed defensive dropout.
Structured Adversarial Attack: Towards General Implementation and Better Interpretability
When generating adversarial examples to attack deep neural networks (DNNs), Lp norm of the added perturbation is usually used to measure the similarity between original image and adversarial example.
An ADMM-Based Universal Framework for Adversarial Attacks on Deep Neural Networks
In the literature, the added distortions are usually measured by L0, L1, L2, and L infinity norms, namely, L0, L1, L2, and L infinity attacks, respectively.
