Search Results for author:
Found 10 papers, 3 papers with code
A Survey on Transformer Compression
Model compression methods reduce the memory and computational cost of Transformer, which is a necessary step to implement large language/vision models on practical devices.
PanGu-$π$: Enhancing Language Model Architectures via Nonlinearity Compensation
We then demonstrate that the proposed approach is significantly effective for enhancing the model nonlinearity through carefully designed ablations; thus, we present a new efficient model architecture for establishing modern, namely, PanGu-$\pi$.
GenDet: Towards Good Generalizations for AI-Generated Image Detection
The misuse of AI imagery can have harmful societal effects, prompting the creation of detectors to combat issues like the spread of fake news.
Data-Free Distillation of Language Model by Text-to-Text Transfer
Data-Free Knowledge Distillation (DFKD) plays a vital role in compressing the model when original training data is unavailable.
Less is More: Focus Attention for Efficient DETR
DETR-like models have significantly boosted the performance of detectors and even outperformed classical convolutional models.
Finding One Missing Puzzle of Contextual Word Embedding: Representing Contexts as Manifold
The current understanding of contextual word embedding interprets the representation by associating each token to a vector that is dynamically modulated by the context.
How Well Does Self-Supervised Pre-Training Perform with Streaming ImageNet?
Prior works on self-supervised pre-training focus on the joint training scenario, where massive unlabeled data are assumed to be given as input all at once, and only then is a learner trained.
How Well Does Self-Supervised Pre-Training Perform with Streaming Data?
Prior works on self-supervised pre-training focus on the joint training scenario, where massive unlabeled data are assumed to be given as input all at once, and only then is a learner trained.
DATSING: Data Augmented Time Series Forecasting with Adversarial Domain Adaptation
In this work, we have developed, DATSING, a transfer learning-based framework that effectively leverages cross-domain time series latent representations to augment target domain forecasting.
DensE: An Enhanced Non-commutative Representation for Knowledge Graph Embedding with Adaptive Semantic Hierarchy
This design principle leads to several advantages of our method: (1) For composite relations, the corresponding diagonal relation matrices can be non-commutative, reflecting a predominant scenario in real world applications; (2) Our model preserves the natural interaction between relational operations and entity embeddings; (3) The scaling operation provides the modeling power for the intrinsic semantic hierarchical structure of entities; (4) The enhanced expressiveness of DensE is achieved with high computational efficiency in terms of both parameter size and training time; and (5) Modeling entities in Euclidean space instead of quaternion space keeps the direct geometrical interpretations of relational patterns.
Ranked #7 on
Link Prediction
on WN18
