Search Results for author:
Found 54 papers, 34 papers with code
A Joint Multiple Criteria Model in Transfer Learning for Cross-domain Chinese Word Segmentation
Chinese word segmentation (CWS) is an essential task for Chinese downstream NLP tasks.
Test-Time Training for Generalization under Distribution Shifts
We introduce a general approach, called test-time training, for improving the performance of predictive models when training and test data come from different distributions.
Amphista: Accelerate LLM Inference with Bi-directional Multiple Drafting Heads in a Non-autoregressive Style
Large Language Models (LLMs) inherently use autoregressive decoding, which lacks parallelism in inference and results in significantly slow inference speeds, especially when hardware parallel accelerators and memory bandwidth are not fully utilized.
Explainable Few-shot Knowledge Tracing
Knowledge tracing (KT), aiming to mine students' mastery of knowledge by their exercise records and predict their performance on future test questions, is a critical task in educational assessment.
Wasserstein Dependent Graph Attention Network for Collaborative Filtering with Uncertainty
Collaborative filtering (CF) is an essential technique in recommender systems that provides personalized recommendations by only leveraging user-item interactions.
A Decade's Battle on Dataset Bias: Are We There Yet?
We revisit the "dataset classification" experiment suggested by Torralba and Efros a decade ago, in the new era with large-scale, diverse, and hopefully less biased datasets as well as more capable neural network architectures.
Massive Activations in Large Language Models
We observe an empirical phenomenon in Large Language Models (LLMs) -- very few activations exhibit significantly larger values than others (e. g., 100, 000 times larger).
A Review of Data Mining in Personalized Education: Current Trends and Future Prospects
Personalized education, tailored to individual student needs, leverages educational technology and artificial intelligence (AI) in the digital age to enhance learning effectiveness.
Neural Network Parameter Diffusion
The autoencoder extracts latent representations of a subset of the trained network parameters.
Zero-shot sketch-based remote sensing image retrieval based on multi-level and attention-guided tokenization
To address this gap, our study introduces a novel zero-shot, sketch-based retrieval method for remote sensing images, leveraging multi-level feature extraction, self-attention-guided tokenization and filtering, and cross-modality attention update.
Deconstructing Denoising Diffusion Models for Self-Supervised Learning
In this study, we examine the representation learning abilities of Denoising Diffusion Models (DDM) that were originally purposed for image generation.
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
To understand the roots of these errors, we explore the gap between the visual embedding space of CLIP and vision-only self-supervised learning.
Initializing Models with Larger Ones
Weight selection offers a new approach to leverage the power of pretrained models in resource-constrained settings, and we hope it can be a useful tool for training small models in the large-model era.
ConvNet vs Transformer, Supervised vs CLIP: Beyond ImageNet Accuracy
Modern computer vision offers a great variety of models to practitioners, and selecting a model from multiple options for specific applications can be challenging.
A Coefficient Makes SVRG Effective
Stochastic Variance Reduced Gradient (SVRG), introduced by Johnson & Zhang (2013), is a theoretically compelling optimization method.
Semantic Graph Representation Learning for Handwritten Mathematical Expression Recognition
Then we design a semantic aware module (SAM), which projects the visual and classification feature into semantic space.
A Simple and Effective Pruning Approach for Large Language Models
Motivated by the recent observation of emergent large magnitude features in LLMs, our approach prunes weights with the smallest magnitudes multiplied by the corresponding input activations, on a per-output basis.
One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuning
We present Generalized LoRA (GLoRA), an advanced approach for universal parameter-efficient fine-tuning tasks.
ImageBind: One Embedding Space To Bind Them All
We show that all combinations of paired data are not necessary to train such a joint embedding, and only image-paired data is sufficient to bind the modalities together.
Ranked #2 on
Zero-shot Classification (unified classes)
on LLVIP
A optimization framework for herbal prescription planning based on deep reinforcement learning
We constructed a high-quality benchmark dataset for sequential diagnosis and treatment of diabetes and evaluated PrescDRL against this benchmark.
Dropout Reduces Underfitting
Additionally, we explore a symmetric technique for regularizing overfitting models - late dropout, where dropout is not used in the early iterations and is only activated later in training.
ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders
This co-design of self-supervised learning techniques and architectural improvement results in a new model family called ConvNeXt V2, which significantly improves the performance of pure ConvNets on various recognition benchmarks, including ImageNet classification, COCO detection, and ADE20K segmentation.
Ranked #45 on
Semantic Segmentation
on ADE20K
1st Place Solutions for UG2+ Challenge 2022 ATMOSPHERIC TURBULENCE MITIGATION
In this technical report, we briefly introduce the solution of our team ''summer'' for Atomospheric Turbulence Mitigation in UG$^2$+ Challenge in CVPR 2022.
A ConvNet for the 2020s
The "Roaring 20s" of visual recognition began with the introduction of Vision Transformers (ViTs), which quickly superseded ConvNets as the state-of-the-art image classification model.
Ranked #1 on
Classification
on InDL
Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space
This paper explores the feasibility of finding an optimal sub-model from a vision transformer and introduces a pure vision transformer slimming (ViT-Slim) framework.
MSeg: A Composite Dataset for Multi-domain Semantic Segmentation
We adopt zero-shot cross-dataset transfer as a benchmark to systematically evaluate a model's robustness and show that MSeg training yields substantially more robust models in comparison to training on individual datasets or naive mixing of datasets without the presented contributions.
Ranked #8 on
Semantic Segmentation
on ScanNetV2
Anytime Dense Prediction with Confidence Adaptivity
A cascade of "exits" is attached to the model to make multiple predictions.
Contrastive Learning for Recommender System
To solve this problem, we propose a graph contrastive learning module for a general recommender system that learns the embeddings in a self-supervised manner and reduces the randomness of message dropout.
Regularization Matters in Policy Optimization - An Empirical Study on Continuous Control
In this work, we present the first comprehensive study of regularization techniques with multiple policy optimization algorithms on continuous control tasks.
Un-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning
This drawback hinders the model from learning subtle variance and fine-grained information.
Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning
The edge between these two lines of works has yet been underexplored, and the effectiveness of meta-learning in few-shot learning remains unclear.
Convolutional Networks with Dense Connectivity
Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output.
Leveraging Prior Knowledge for Protein-Protein Interaction Extraction with Memory Network
Automatically extracting Protein-Protein Interactions (PPI) from biomedical literature provides additional support for precision medicine efforts.
Chemical-induced Disease Relation Extraction with Dependency Information and Prior Knowledge
KBs are important resources for biomedical relation extraction.
Knowledge-guided Convolutional Networks for Chemical-Disease Relation Extraction
Results: This paper proposes a novel model called "Knowledge-guided Convolutional Networks (KCN)" to leverage prior knowledge for CDR extraction.
Combining Context and Knowledge Representations for Chemical-Disease Relation Extraction
KBs contain huge amounts of structured information about entities and relationships, therefore plays a pivotal role in chemical-disease relation (CDR) extraction.
Improving Neural Protein-Protein Interaction Extraction with Knowledge Selection
Finally, the selected relation embedding and the context features are concatenated for PPI extraction.
Regularization Matters in Policy Optimization
In this work, we present the first comprehensive study of regularization techniques with multiple policy optimization algorithms on continuous control tasks.
Exploring Simple and Transferable Recognition-Aware Image Processing
Interestingly, the processing model's ability to enhance recognition quality can transfer when evaluated on models of different architectures, recognized categories, tasks and training datasets.
Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
In this paper, we propose Test-Time Training, a general approach for improving the performance of predictive models when training and test data come from different distributions.
Ranked #34 on
Language Modelling
on LAMBADA
Building change detection for remote sensing images
CARLA MAP Leaderboard
+6
Test-Time Training for Out-of-Distribution Generalization
We introduce a general approach, called test-time training, for improving the performance of predictive models when test and training data come from different distributions.
DUT-BIM at MEDIQA 2019: Utilizing Transformer Network and Medical Domain-Specific Contextualized Representations for Question Answering
BioNLP 2019 proposes Question Answering (QA) task, which encourages the use of text mining technology to automatically judge whether a search result is an answer to the medical question.
Few Sample Knowledge Distillation for Efficient Network Compression
Deep neural network compression techniques such as pruning and weight tensor decomposition usually require fine-tuning to recover the prediction accuracy when the compression ratio is high.
Few-shot Object Detection via Feature Reweighting
The feature learner extracts meta features that are generalizable to detect novel object classes, using training data from base classes with sufficient samples.
Ranked #23 on
Few-Shot Object Detection
on MS-COCO (30-shot)
Rethinking the Value of Network Pruning
Our observations are consistent for multiple network architectures, datasets, and tasks, which imply that: 1) training a large, over-parameterized model is often not necessary to obtain an efficient final model, 2) learned "important" weights of the large model are typically not useful for the small pruned model, 3) the pruned architecture itself, rather than a set of inherited "important" weights, is more crucial to the efficiency in the final model, which suggests that in some cases pruning can be useful as an architecture search paradigm.
Knowledge Distillation from Few Samples
Taking the assumption that both "teacher" and "student" have the same feature map sizes at each corresponding block, we add a $1\times 1$ conv-layer at the end of each block in the student-net, and align the block-level outputs between "teacher" and "student" by estimating the parameters of the added layer with limited samples.
Object Detection from Scratch with Deep Supervision
Thus, a better solution to handle these critical problems is to train object detectors from scratch, which motivates our proposed method.
Learning Efficient Convolutional Networks through Network Slimming
For VGGNet, a multi-pass version of network slimming gives a 20x reduction in model size and a 5x reduction in computing operations.
DSOD: Learning Deeply Supervised Object Detectors from Scratch
State-of-the-art object objectors rely heavily on the off-the-shelf networks pre-trained on large-scale classification datasets like ImageNet, which incurs learning bias due to the difference on both the loss functions and the category distributions between classification and detection tasks.
Snapshot Ensembles: Train 1, get M for free
In this paper, we propose a method to obtain the seemingly contradictory goal of ensembling multiple neural networks at no additional training cost.
Densely Connected Convolutional Networks
Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output.
Ranked #1 on
Classification
on XImageNet-12
Deep Networks with Stochastic Depth
With stochastic depth we can increase the depth of residual networks even beyond 1200 layers and still yield meaningful improvements in test error (4. 91% on CIFAR-10).
Ranked #20 on
Image Classification
on SVHN
