Search Results for author:
Found 58 papers, 25 papers with code
Pangu DeepDiver: Adaptive Search Intensity Scaling via Open-Web Reinforcement Learning
Building on this dataset, we propose DeepDiver, a Reinforcement Learning (RL) framework that promotes SIS by encouraging adaptive search policies through exploration under a real-world open-web environment.
Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition
This work presents Pangu Embedded, an efficient Large Language Model (LLM) reasoner developed on Ascend Neural Processing Units (NPUs), featuring flexible fast and slow thinking capabilities.
Pangu Light: Weight Re-Initialization for Pruning and Accelerating LLMs
While structured pruning offers a promising avenue for model compression, existing methods often struggle with the detrimental effects of aggressive, simultaneous width and depth reductions, leading to substantial performance degradation.
EAM: Enhancing Anything with Diffusion Transformers for Blind Super-Resolution
We introduce a novel block, $\Psi$-DiT, which effectively guides the DiT to enhance image restoration.
DSPO: Direct Semantic Preference Optimization for Real-World Image Super-Resolution
Recent advances in diffusion models have improved Real-World Image Super-Resolution (Real-ISR), but existing methods lack human feedback integration, risking misalignment with human preference and may leading to artifacts, hallucinations and harmful content generation.
A Physics-guided Multimodal Transformer Path to Weather and Climate Sciences
With the rapid development of machine learning in recent years, many problems in meteorology can now be addressed using AI models.
Transferable text data distillation by trajectory matching
In the realm of large language model (LLM), as the size of large models increases, it also brings higher training costs.
U-REPA: Aligning Diffusion U-Nets to ViTs
Representation Alignment (REPA) that aligns Diffusion Transformer (DiT) hidden-states with ViT visual encoders has proven highly effective in DiT training, demonstrating superior convergence properties, but it has not been validated on the canonical diffusion U-Net architecture that shows faster convergence compared to DiTs.
Ranked #15 on
Image Generation
on ImageNet 256x256
Autoregressive Image Generation Guided by Chains of Thought
In the field of autoregressive (AR) image generation, models based on the 'next-token prediction' paradigm of LLMs have shown comparable performance to diffusion models by reducing inductive biases.
Unshackling Context Length: An Efficient Selective Attention Approach through Query-Key Compression
Handling long-context sequences efficiently remains a significant challenge in large language models (LLMs).
RaSS: Improving Denoising Diffusion Samplers with Reinforced Active Sampling Scheduler
The RaSS is a plug-and-play module, which is applicable to multiple denoising diffusion samplers of diffusion models.
DiC: Rethinking Conv3x3 Designs in Diffusion Models
Diffusion models have shown exceptional performance in visual generation tasks.
Learning Quantized Adaptive Conditions for Diffusion Models
The curvature of ODE trajectories in diffusion models hinders their ability to generate high-quality images in a few number of function evaluations (NFE).
One Step Diffusion-based Super-Resolution with Time-Aware Distillation
Specifically, we introduce a novel score distillation strategy to align the data distribution between the outputs of the student and teacher models after minor noise perturbation.
Omni-Dimensional Frequency Learner for General Time Series Analysis
Frequency domain representation of time series feature offers a concise representation for handling real-world time series data with inherent complexity and dynamic nature.
Multi-Granularity Semantic Revision for Large Language Model Distillation
At the sequence level, we propose a sequence correction and re-generation (SCRG) strategy.
Instruct-IPT: All-in-One Image Processing Transformer via Weight Modulation
Due to the unaffordable size and intensive computation costs of low-level vision models, All-in-One models that are designed to address a handful of low-level vision tasks simultaneously have been popular.
Ranked #1 on
Single Image Desnowing
on CSD
GIM: A Million-scale Benchmark for Generative Image Manipulation Detection and Localization
The extraordinary ability of generative models emerges as a new trend in image editing and generating realistic images, posing a serious threat to the trustworthiness of multimedia data and driving the research of image manipulation detection and location (IMDL).
Collaboration of Teachers for Semi-supervised Object Detection
To address these issues, we propose the Collaboration of Teachers Framework (CTF), which consists of multiple pairs of teacher and student models for training.
U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers
Diffusion Transformers (DiTs) introduce the transformer architecture to diffusion tasks for latent-space image generation.
LIPT: Latency-aware Image Processing Transformer
Extensive experiments on multiple image processing tasks (e. g., image super-resolution (SR), JPEG artifact reduction, and image denoising) demonstrate the superiority of LIPT on both latency and PSNR.
Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution
Knowledge distillation (KD) is a promising yet challenging model compression technique that transfers rich learning representations from a well-performing but cumbersome teacher model to a compact student model.
IPT-V2: Efficient Image Processing Transformer using Hierarchical Attentions
Recent advances have demonstrated the powerful capability of transformer architecture in image restoration.
DiJiang: Efficient Large Language Models through Compact Kernelization
In an effort to reduce the computational load of Transformers, research on linear attention has gained significant momentum.
Distilling Semantic Priors from SAM to Efficient Image Restoration Models
SPD leverages a self-distillation manner to distill the fused semantic priors to boost the performance of original IR models.
Vision Superalignment: Weak-to-Strong Generalization for Vision Foundation Models
Recent advancements in large language models have sparked interest in their extraordinary and near-superhuman capabilities, leading researchers to explore methods for evaluating and optimizing these abilities, which is called superalignment.
Image Processing GNN: Breaking Rigidity in Super-Resolution
Alternatively we leverage the flexibility of graphs and propose the Image Processing GNN (IPG) model to break the rigidity that dominates previous SR methods.
Ranked #11 on
Image Super-Resolution
on Urban100 - 4x upscaling
PanGu-$π$: Enhancing Language Model Architectures via Nonlinearity Compensation
We then demonstrate that the proposed approach is significantly effective for enhancing the model nonlinearity through carefully designed ablations; thus, we present a new efficient model architecture for establishing modern, namely, PanGu-$\pi$.
CBQ: Cross-Block Quantization for Large Language Models
Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs.
GenDet: Towards Good Generalizations for AI-Generated Image Detection
The misuse of AI imagery can have harmful societal effects, prompting the creation of detectors to combat issues like the spread of fake news.
Towards Higher Ranks via Adversarial Weight Pruning
To this end, we propose a Rank-based PruninG (RPG) method to maintain the ranks of sparse weights in an adversarial manner.
IFT: Image Fusion Transformer for Ghost-free High Dynamic Range Imaging
By matching similar patches between frames, objects with large motion ranges in dynamic scenes can be aligned, which can effectively alleviate the generation of artifacts.
Data Upcycling Knowledge Distillation for Image Super-Resolution
Knowledge distillation (KD) compresses deep neural networks by transferring task-related knowledge from cumbersome pre-trained teacher models to compact student models.
Ranked #32 on
Image Super-Resolution
on Urban100 - 4x upscaling
Multiscale Positive-Unlabeled Detection of AI-Generated Texts
Recent releases of Large Language Models (LLMs), e. g. ChatGPT, are astonishing at generating human-like texts, but they may impact the authenticity of texts.
VanillaNet: the Power of Minimalism in Deep Learning
In this study, we introduce VanillaNet, a neural network architecture that embraces elegance in design.
RefSR-NeRF: Towards High Fidelity and Super Resolution View Synthesis
We present Reference-guided Super-Resolution Neural Radiance Field (RefSR-NeRF) that extends NeRF to super resolution and photorealistic novel view synthesis.
Toward Accurate Post-Training Quantization for Image Super Resolution
In this paper, we study post-training quantization(PTQ) for image super resolution using only a few unlabeled calibration images.
Brain-inspired Multilayer Perceptron with Spiking Neurons
However, due to the simplicity of their structures, the performance highly depends on the local features communication machenism.
Adder Attention for Vision Transformer
With the new operation, vision transformers constructed using additions can also provide powerful feature representations.
Positive and Unlabeled Federated Learning
We study the problem of learning from positive and unlabeled (PU) data in the federated setting, where each client only labels a little part of their dataset due to the limitation of resources and time.
Federated Learning with Positive and Unlabeled Data
We study the problem of learning from positive and unlabeled (PU) data in the federated setting, where each client only labels a little part of their dataset due to the limitation of resources and time.
Learning Student Networks in the Wild
Experiments on various datasets demonstrate that the student networks learned by the proposed method can achieve comparable performance with those using the original dataset.
Data-Free Knowledge Distillation for Image Super-Resolution
Experiments on various datasets and architectures demonstrate that the proposed method is able to be utilized for effectively learning portable student networks without the original data, e. g., with 0. 16dB PSNR drop on Set5 for x2 super resolution.
Universal Adder Neural Networks
The widely-used convolutions in deep neural networks are exactly cross-correlation to measure the similarity between input feature and convolution filters, which involves massive multiplications between float values.
Winograd Algorithm for AdderNet
Adder neural network (AdderNet) is a new kind of deep model that replaces the original massive multiplications in convolutions by additions while preserving the high performance.
AdderNet and its Minimalist Hardware Design for Energy-Efficient Artificial Intelligence
With a comprehensive comparison on the performance, power consumption, hardware resource consumption and network generalization capability, we conclude the AdderNet is able to surpass all the other competitors including the classical CNN, novel memristor-network, XNOR-Net and the shift-kernel based network, indicating its great potential in future high performance and energy-efficient artificial intelligence applications.
A Survey on Visual Transformer
Transformer, first applied to the field of natural language processing, is a type of deep neural network mainly based on the self-attention mechanism.
Pre-Trained Image Processing Transformer
To maximally excavate the capability of transformer, we present to utilize the well-known ImageNet benchmark for generating a large amount of corrupted image pairs.
Ranked #1 on
Single Image Deraining
on Rain100L
(using extra training data)
AdderSR: Towards Energy Efficient Image Super-Resolution
To this end, we thoroughly analyze the relationship between an adder operation and the identity mapping and insert shortcuts to enhance the performance of SR models using adder networks.
Frequency Domain Compact 3D Convolutional Neural Networks
This paper studies the compression and acceleration of 3-dimensional convolutional neural networks (3D CNNs).
A Semi-Supervised Assessor of Neural Architectures
A graph convolutional neural network is introduced to predict the performance of architectures based on the learned representations and their relation modeled by the graph.
Distilling portable Generative Adversarial Networks for Image Translation
To promote the capability of student generator, we include a student discriminator to measure the distances between real images, and images generated by student and teacher generators.
Widening and Squeezing: Towards Accurate and Efficient QNNs
Quantization neural networks (QNNs) are very attractive to the industry because their extremely cheap calculation and storage overhead, but their performance is still worse than that of networks with full-precision parameters.
AdderNet: Do We Really Need Multiplications in Deep Learning?
The widely-used convolutions in deep neural networks are exactly cross-correlation to measure the similarity between input feature and convolution filters, which involves massive multiplications between float values.
Positive-Unlabeled Compression on the Cloud
In practice, only a small portion of the original training set is required as positive examples and more useful training examples can be obtained from the massive unlabeled data on the cloud through a PU classifier with an attention based multi-scale feature extractor.
Co-Evolutionary Compression for Unpaired Image Translation
Generative adversarial networks (GANs) have been successfully used for considerable computer vision tasks, especially the image-to-image translation.
Data-Free Learning of Student Networks
Learning portable neural networks is very essential for computer vision for the purpose that pre-trained heavy deep models can be well applied on edge devices such as mobile phones and micro sensors.
Learning Student Networks via Feature Embedding
Experiments on benchmark datasets and well-trained networks suggest that the proposed algorithm is superior to state-of-the-art teacher-student learning methods in terms of computational and storage complexity.
