Search Results for author:
Found 19 papers, 12 papers with code
DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
Large language models (LLMs) face a daunting challenge due to the excessive computational and memory requirements of the commonly used Transformer architecture.
PanGu-$π$: Enhancing Language Model Architectures via Nonlinearity Compensation
We then demonstrate that the proposed approach is significantly effective for enhancing the model nonlinearity through carefully designed ablations; thus, we present a new efficient model architecture for establishing modern, namely, PanGu-$\pi$.
UFineBench: Towards Text-based Person Retrieval with Ultra-fine Granularity
Firstly, we construct a new \textbf{dataset} named UFine6926.
LightCLIP: Learning Multi-Level Interaction for Lightweight Vision-Language Models
Moreover, based on the observation that the accuracy of CLIP model does not increase correspondingly as the parameters of text encoder increase, an extra objective of masked language modeling (MLM) is leveraged for maximizing the potential of the shortened text encoder.
Towards Higher Ranks via Adversarial Weight Pruning
To this end, we propose a Rank-based PruninG (RPG) method to maintain the ranks of sparse weights in an adversarial manner.
Data-Free Distillation of Language Model by Text-to-Text Transfer
Data-Free Knowledge Distillation (DFKD) plays a vital role in compressing the model when original training data is unavailable.
NICE: Improving Panoptic Narrative Detection and Segmentation with Cascading Collaborative Learning
To address this, we introduce two cascading modules based on the barycenter of the mask, which are Coordinate Guided Aggregation (CGA) and Barycenter Driven Localization (BDL), responsible for segmentation and detection, respectively.
How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations
Through extensive probing and a new pasting experiment, we further reveal several mechanisms within the trained transformers, such as concrete copying behaviors on both the inputs and the representations, linear ICL capability of the upper layers alone, and a post-ICL representation selection mechanism in a harder mixture setting.
Joint Adversarial and Collaborative Learning for Self-Supervised Action Recognition
Considering the instance-level discriminative ability, contrastive learning methods, including MoCo and SimCLR, have been adapted from the original image representation learning task to solve the self-supervised skeleton-based action recognition task.
FSAR: Federated Skeleton-based Action Recognition with Adaptive Topology Structure and Knowledge Distillation
Existing skeleton-based action recognition methods typically follow a centralized learning paradigm, which can pose privacy concerns when exposing human-related videos.
Contrastive Learning from Spatio-Temporal Mixed Skeleton Sequences for Self-Supervised Skeleton-Based Action Recognition
First, SkeleMix utilizes the topological information of skeleton data to mix two skeleton sequences by randomly combing the cropped skeleton fragments (the trimmed view) with the remaining skeleton sequences (the truncated view).
GraphMLP: A Graph MLP-Like Architecture for 3D Human Pose Estimation
To the best of our knowledge, this is the first MLP-Like architecture for 3D human pose estimation in a single frame and a video sequence.
Ranked #53 on
3D Human Pose Estimation
on Human3.6M
Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognition
In this paper, to make better use of the movement patterns introduced by extreme augmentations, a Contrastive Learning framework utilizing Abundant Information Mining for self-supervised action Representation (AimCLR) is proposed.
Pose-guided Feature Disentangling for Occluded Person Re-identification Based on Transformer
Therefore, we propose a transformer-based Pose-guided Feature Disentangling (PFD) method by utilizing pose information to clearly disentangle semantic components (e. g. human body or joint parts) and selectively match non-occluded parts correspondingly.
Learning Student Networks in the Wild
Experiments on various datasets demonstrate that the student networks learned by the proposed method can achieve comparable performance with those using the original dataset.
Pre-Trained Image Processing Transformer
To maximally excavate the capability of transformer, we present to utilize the well-known ImageNet benchmark for generating a large amount of corrupted image pairs.
Ranked #1 on
Single Image Deraining
on Rain100L
(using extra training data)
On Positive-Unlabeled Classification in GAN
In contrast, it is more reasonable to treat the generated data as unlabeled, which could be positive or negative according to their quality.
Learning from Bad Data via Generation
A worst-case formulation can be developed over this distribution set, and then be interpreted as a generation task in an adversarial manner.
Robust Student Network Learning
In this way, a portable student network with significantly fewer parameters can achieve a considerable accuracy which is comparable to that of teacher network.
