Search Results for author:
Found 51 papers, 31 papers with code
Instruction Multi-Constraint Molecular Generation Using a Teacher-Student Large Language Model
While various models and computational tools have been proposed for structure and property analysis of molecules, generating molecules that conform to all desired structures and properties remains a challenge.
depyf: Open the Opaque Box of PyTorch Compiler for Machine Learning Researchers
PyTorch \texttt{2. x} introduces a compiler designed to accelerate deep learning programs.
TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables
Experimentally, TimeXer significantly improves time series forecasting with exogenous variables and achieves consistent state-of-the-art performance in twelve real-world forecasting benchmarks.
TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling
To emphasize temporal correlation modeling, this paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
Transolver: A Fast Transformer Solver for PDEs on General Geometries
Transformers have empowered many milestones across various fields and have recently been applied to solve partial differential equations (PDEs).
Timer: Transformers for Time Series Analysis at Scale
Continuous progresses have been achieved as the emergence of large language models, exhibiting unprecedented ability in few-shot generalization, scalability, and task generality, which is however absent in time series models.
AutoTimes: Autoregressive Time Series Forecasters via Large Language Models
Foundation models of time series have not been fully developed due to the limited availability of large-scale time series and the underexploration of scalable pre-training.
EuLagNet: Eulerian Fluid Prediction with Lagrangian Dynamics
Accurately predicting the future fluid is important to extensive areas, such as meteorology, oceanology and aerodynamics.
HelmFluid: Learning Helmholtz Dynamics for Interpretable Fluid Prediction
Compared with previous velocity estimating methods, HelmFluid is faithfully derived from Helmholtz theorem and ravels out complex fluid dynamics with physically interpretable evidence.
HarmonyDream: Task Harmonization Inside World Models
Model-based reinforcement learning (MBRL) holds the promise of sample-efficient learning by utilizing a world model, which models how the environment works and typically encompasses components for two tasks: observation modeling and reward modeling.
Ranked #4 on
Atari Games 100k
on Atari 100k
Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors
While previous models suffer from complicated series variations induced by changing temporal distribution, we tackle non-stationary time series with modern Koopman theory that fundamentally considers the underlying time-variant dynamics.
SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling
By relating masked modeling to manifold learning, SimMTM proposes to recover masked time points by the weighted aggregation of multiple neighbors outside the manifold, which eases the reconstruction task by assembling ruined but complementary temporal variations from multiple masked series.
CLIPood: Generalizing CLIP to Out-of-Distributions
This paper aims at generalizing CLIP to out-of-distribution test data on downstream tasks.
Solving High-Dimensional PDEs with Latent Spectral Models
A burgeoning paradigm is learning neural operators to approximate the input-output mappings of PDEs.
TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
TimesBlock can discover the multi-periodicity adaptively and extract the complex temporal variations from transformed 2D tensors by a parameter-efficient inception block.
Requirements Engineering for Machine Learning: A Review and Reflection
Today, many industrial processes are undergoing digital transformation, which often requires the integration of well-understood domain models and state-of-the-art machine learning technology in business processes.
Hub-Pathway: Transfer Learning from A Hub of Pre-trained Models
The proposed framework can be trained end-to-end with the target task-specific loss, where it learns to explore better pathway configurations and exploit the knowledge in pre-trained models for each target datum.
Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting
However, their performance can degenerate terribly on non-stationary real-world data in which the joint distribution changes over time.
MetaSets: Meta-Learning on Point Sets for Generalizable Representations
It is a challenging problem due to the substantial geometry shift from simulated to real data, such that most existing 3D models underperform due to overfitting the complete geometries in the source domain.
From Big to Small: Adaptive Learning to Partial-Set Domains
Still, the common requirement of identical class space shared across domains hinders applications of domain adaptation to partial-set domains.
Debiased Self-Training for Semi-Supervised Learning
Yet these datasets are time-consuming and labor-exhaustive to obtain on realistic tasks.
Supported Policy Optimization for Offline Reinforcement Learning
Policy constraint methods to offline reinforcement learning (RL) typically utilize parameterization or regularization that constrains the policy to perform actions within the support set of the behavior policy.
Flowformer: Linearizing Transformers with Conservation Flows
By respectively conserving the incoming flow of sinks for source competition and the outgoing flow of sources for sink allocation, Flow-Attention inherently generates informative attentions without using specific inductive biases.
Ranked #4 on
D4RL
on D4RL
What Makes the Story Forward? Inferring Commonsense Explanations as Prompts for Future Event Generation
To alleviate the knowledge forgetting issue, we design two modules, Im and Gm, for each type of knowledge, which are combined via prompt tuning.
Transferability in Deep Learning: A Survey
The success of deep learning algorithms generally depends on large-scale data, while humans appear to have inherent ability of knowledge transfer, by recognizing and applying relevant knowledge from previous learning experiences when encountering and solving unseen tasks.
Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
(2) The best ranked PTM can either be fine-tuned and deployed if we have no preference for the model's architecture or the target PTM can be tuned by the top $K$ ranked PTMs via a Bayesian procedure that we propose.
Omni-Training: Bridging Pre-Training and Meta-Training for Few-Shot Learning
While pre-training and meta-training can create deep models powerful for few-shot generalization, we find that pre-training and meta-training focuses respectively on cross-domain transferability and cross-task transferability, which restricts their data efficiency in the entangled settings of domain shift and task shift.
X-model: Improving Data Efficiency in Deep Learning with A Minimax Model
To take the power of both worlds, we propose a novel X-model by simultaneously encouraging the invariance to {data stochasticity} and {model stochasticity}.
ModeRNN: Harnessing Spatiotemporal Mode Collapse in Unsupervised Predictive Learning
To this end, we propose ModeRNN, which introduces a novel method to learn structured hidden representations between recurrent states.
Decoupled Adaptation for Cross-Domain Object Detection
Besides, previous methods focused on category adaptation but ignored another important part for object detection, i. e., the adaptation on bounding box regression.
Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy
Unsupervised detection of anomaly points in time series is a challenging problem, which requires the model to derive a distinguishable criterion.
The Report on China-Spain Joint Clinical Testing for Rapid COVID-19 Risk Screening by Eye-region Manifestations
The image-level performance of COVID-19 prescreening model in the China-Spain multicenter study achieved an AUC of 0. 913 (95% CI, 0. 898-0. 927), with a sensitivity of 0. 695 (95% CI, 0. 643-0. 748), a specificity of 0. 904 (95% CI, 0. 891 -0. 919), an accuracy of 0. 875(0. 861-0. 889), and a F1 of 0. 611(0. 568-0. 655).
Zoo-Tuning: Adaptive Transfer from a Zoo of Models
We propose \emph{Zoo-Tuning} to address these challenges, which learns to adaptively transfer the parameters of pretrained models to the target task.
Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
Going beyond Transformers, we design Autoformer as a novel decomposition architecture with an Auto-Correlation mechanism.
Transferable Query Selection for Active Domain Adaptation
Due to the domain shift, the query selection criteria of prior active learning methods may be ineffective to select the most informative target samples for annotation.
Rapid COVID-19 Risk Screening by Eye-region Manifestations
It is still nontrivial to develop a new fast COVID-19 screening method with the easier access and lower cost, due to the technical and cost limitations of the current testing methods in the medical resource-poor districts.
MetaSets：Meta-Learning on Point Sets for Generalizable Representations
It is a challenging problem due to the substantial geometry shift from simulated to real data, such that most existing 3D models underperform due to overfitting the complete geometries in the source domain.
Open Domain Generalization with Domain-Augmented Meta-Learning
Leveraging datasets available to learn a model with high generalization ability to unseen domains is important for computer vision, especially when the unseen domain's annotated data are unavailable.
PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning
This paper models these structures by presenting PredRNN, a new recurrent network, in which a pair of memory cells are explicitly decoupled, operate in nearly independent transition manners, and finally form unified representations of the complex environment.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
Regressive Domain Adaptation for Unsupervised Keypoint Detection
First, based on our observation that the probability density of the output space is sparse, we introduce a spatial probability distribution to describe this sparsity and then use it to guide the learning of the adversarial regressor.
Cycle Self-Training for Domain Adaptation
In the forward step, CST generates target pseudo-labels with a source-trained classifier.
MotionRNN: A Flexible Model for Video Prediction with Spacetime-Varying Motions
With high flexibility, this framework can adapt to a series of models for deterministic spatiotemporal prediction.
Self-Tuning for Data-Efficient Deep Learning
Deep learning has made revolutionary advances to diverse applications in the presence of large-scale labeled datasets.
LogME: Practical Assessment of Pre-trained Models for Transfer Learning
In pursuit of a practical assessment method, we propose to estimate the maximum value of label evidence given features extracted by pre-trained models.
Ranked #3 on
Transferability
on classification benchmark
GAHNE: Graph-Aggregated Heterogeneous Network Embedding
Heterogeneous network embedding aims to embed nodes into low-dimensional vectors which capture rich intrinsic information of heterogeneous networks.
Stochastic Normalization
During training, two branches are stochastically selected to avoid over-depending on some sample statistics, resulting in a strong regularization effect, which we interpret as ``architecture regularization.''
Co-Tuning for Transfer Learning
Fine-tuning pre-trained deep neural networks (DNNs) to a target dataset, also known as transfer learning, is widely used in computer vision and NLP.
Ranked #1 on
Transfer Learning
on COCO70
Learning to Adapt to Evolving Domains
(2) Since the target data arrive online, the agent should also maintain competence on previous target domains, i. e. to adapt without forgetting.
Bi-tuning of Pre-trained Representations
It is common within the deep learning community to first pre-train a deep neural network from a large-scale dataset and then fine-tune the pre-trained model to a specific downstream task.
VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
This paper introduces video domain generalization where most video classification networks degenerate due to the lack of exposure to the target domains of divergent distributions.
Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation
In this paper, a series of experiments based on spectral analysis of the feature representations have been conducted, revealing an unexpected deterioration of the discriminability while learning transferable features adversarially.
