Search Results for author:
Found 99 papers, 35 papers with code
PCEE-BERT: Accelerating BERT Inference via Patient and Confident Early Exiting
In this work, we propose Patient and Confident Early Exiting BERT (PCEE-BERT), an off-the-shelf sample-dependent early exiting method that can work with different PLMs and can also work along with popular model compression methods.
HwTscSU’s Submissions on WAT 2022 Shared Task
In this paper we describe our submission to the shared tasks of the 9th Workshop on Asian Translation (WAT 2022) on NICT–SAP under the team name ”HwTscSU”.
Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey
Due to intensive synchronization of models and sharing of data across GPUs and computing nodes during distributed training and inference processes, communication efficiency becomes the bottleneck for achieving high performance at a large scale.
The Double-Edged Sword of Input Perturbations to Robust Accurate Fairness
Deep neural networks (DNNs) are known to be sensitive to adversarial input perturbations, leading to a reduction in either prediction accuracy or individual fairness.
Identifiable Latent Neural Causal Models
This work establishes a {sufficient} and {necessary} condition characterizing the types of distribution shifts for identifiability in the context of latent additive noise models.
A Causal Inspired Early-Branching Structure for Domain Generalization
By d-separation, we observe that the causal feature can be further characterized by being independent of the domain conditioned on the object, and we propose the following two strategies as complements for the basic framework.
Collaborate to Adapt: Source-Free Graph Domain Adaptation via Bi-directional Adaptation
Unsupervised Graph Domain Adaptation (UGDA) has emerged as a practical solution to transfer knowledge from a label-rich source graph to a completely unlabelled target graph.
BuffGraph: Enhancing Class-Imbalanced Node Classification via Buffer Nodes
Class imbalance in graph-structured data, where minor classes are significantly underrepresented, poses a critical challenge for Graph Neural Networks (GNNs).
LinkNER: Linking Local Named Entity Recognition Models to Large Language Models using Uncertainty
Named Entity Recognition (NER) serves as a fundamental task in natural language understanding, bearing direct implications for web content analysis, search engines, and information retrieval systems.
Distillation Enhanced Generative Retrieval
Generative retrieval is a promising new paradigm in text retrieval that generates identifier strings of relevant passages as the retrieval target.
Revealing Multimodal Contrastive Representation Learning through Latent Partial Causal Models
Multimodal contrastive representation learning methods have proven successful across a range of domains, partly due to their ability to generate meaningful shared representations of complex phenomena.
Rethinking Propagation for Unsupervised Graph Domain Adaptation
Motivated by our empirical analysis, we reevaluate the role of GNNs in graph domain adaptation and uncover the pivotal role of the propagation process in GNNs for adapting to different graph domains.
PolyTOPS: Reconfigurable and Flexible Polyhedral Scheduler
Different scenarios, depending on the target architecture, compilation environment, and application domain, may require different kinds of optimization to best exploit the architecture feature set.
A Video Coding Method Based on Neural Network for CLIC2024
Meanwhile, the deep learning methods propose a convolutional neural network-based loop filter (CNNLF), which is turned on/off based on the rate-distortion optimization at the CTU and frame level.
GBSS:a global building semantic segmentation dataset for large-scale remote sensing building extraction
Semantic segmentation techniques for extracting building footprints from high-resolution remote sensing images have been widely used in many fields such as urban planning.
Coreference Graph Guidance for Mind-Map Generation
Then we employ a coreference graph encoder to mine the potential governing relations between sentences.
CLAP: Isolating Content from Style through Contrastive Learning with Augmented Prompts
To address this limitation, we adopt a causal generative perspective for multimodal data and propose contrastive learning with data augmentation to disentangle content features from the original representations.
Identifiable Latent Polynomial Causal Models Through the Lens of Change
However, this progress rests on the assumption that the causal relationships among latent causal variables adhere strictly to linear Gaussian models.
Interactive Navigation in Environments with Traversable Obstacles Using Large Language and Vision-Language Models
This paper proposes an interactive navigation framework by using large language and vision-language models, allowing robots to navigate in environments with traversable obstacles.
EX-Graph: A Pioneering Dataset Bridging Ethereum and X
While numerous public blockchain datasets are available, their utility is constrained by an exclusive focus on blockchain data.
Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templates
Oobleck enables resilient distributed training of large DNN models with guaranteed fault tolerance.
RADE: Reference-Assisted Dialogue Evaluation for Open-Domain Dialogue
To this end, we propose the Reference-Assisted Dialogue Evaluation (RADE) approach under the multi-task learning framework, which leverages the pre-created utterance as reference other than the gold response to relief the one-to-many problem.
CFDBench: A Large-Scale Benchmark for Machine Learning Methods in Fluid Dynamics
Appropriate modifications were made to apply popular deep neural networks to CFDBench and enable the accommodation of more changing inputs.
Depth analysis of battery performance based on a data-driven approach
Capacity attenuation is one of the most intractable issues in the current of application of the cells.
Factor Graph Neural Networks
In recent years, we have witnessed a surge of Graph Neural Networks (GNNs), most of which can learn powerful representations in an end-to-end fashion with great success in many real-world applications.
Discovering a reaction-diffusion model for Alzheimer's disease by combining PINNs with symbolic regression
Specifically, we integrate physics informed neural networks (PINNs) and symbolic regression to discover a reaction-diffusion type partial differential equation for tau protein misfolding and spreading.
Certified Robustness for Large Language Models with Self-Denoising
This largely falls into the study of certified robust LLMs, i. e., all predictions of LLM are certified to be correct in a local region around the input.
OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models
The scale of large pre-trained models (PTMs) poses significant challenges in adapting to downstream tasks due to the high optimization overhead and storage costs associated with full-parameter fine-tuning.
Uncertainty in Natural Language Processing: Sources, Quantification, and Applications
Therefore, in this survey, we provide a comprehensive review of uncertainty-relevant works in the NLP field.
DataAI-6G: A System Parameters Configurable Channel Dataset for AI-6G Research
A common and accurate dataset is essential for the research of AI communication.
Uncertainty-Aware Unlikelihood Learning Improves Generative Aspect Sentiment Quad Prediction
We further propose marginalized unlikelihood learning to suppress the uncertainty-aware mistake tokens.
E-NER: Evidential Deep Learning for Trustworthy Named Entity Recognition
Most named entity recognition (NER) systems focus on improving model performance, ignoring the need to quantify model uncertainty, which is critical to the reliability of NER systems in open environments.
RobustFair: Adversarial Evaluation through Fairness Confusion Directed Gradient Search
This matrix categorizes predictions as true fair, true biased, false fair, and false biased, and the perturbations guided by it can produce a dual impact on instances and their similar counterparts to either undermine prediction accuracy (robustness) or cause biased predictions (individual fairness).
Parameter-Efficient Cross-lingual Transfer of Vision and Language Models via Translation-based Alignment
Extensive experiments on XTD and Multi30K datasets, covering 11 languages under zero-shot, few-shot, and full-dataset learning scenarios, show that our framework significantly reduces the multilingual disparities among languages and improves cross-lingual transfer results, especially in low-resource scenarios, while only keeping and fine-tuning an extremely small number of parameters compared to the full model (e. g., Our framework only requires 0. 16\% additional parameters of a full-model for each language in the few-shot learning scenario).
Tool Learning with Foundation Models
Considering the lack of a systematic tool learning evaluation in prior works, we experiment with 18 representative tools and show the potential of current foundation models in skillfully utilizing tools.
A robust design of time-varying internal model principle-based control for ultra-precision tracking in a direct-drive servo stage
By means of the ESO feedback, the plant model is kept as nominal, and hence the structural robustness is achieved for the time-varying internal model.
BERT4ETH: A Pre-trained Transformer for Ethereum Fraud Detection
As various forms of fraud proliferate on Ethereum, it is imperative to safeguard against these malicious activities to protect susceptible users from being victimized.
Multi-pooling 3D Convolutional Neural Network for fMRI Classification of Visual Brain States
Neural decoding of visual object classification via functional magnetic resonance imaging (fMRI) data is challenging and is vital to understand underlying brain mechanisms.
Slapo: A Schedule Language for Progressive Optimization of Large Deep Learning Model Training
Specifically, Slapo works on a PyTorch model and uses a set of schedule primitives to convert the model for common model training optimizations such as high-performance kernels, effective 3D parallelism, and efficient activation checkpointing.
Cost-minimization predictive energy management of a postal-delivery fuel cell electric vehicle with intelligent battery State-of-Charge Planner
Fuel cell electric vehicles have earned substantial attentions in recent decades due to their high-efficiency and zero-emission features, while the high operating costs remain the major barrier towards their large-scale commercialization.
Sparse Structure Search for Delta Tuning
Generally, DT methods exquisitely design delta modules (DT modules) which could be applied to arbitrary fine-grained positions inside PTMs.
Uncertainty Sentence Sampling by Virtual Adversarial Perturbation
Active learning for sentence understanding attempts to reduce the annotation cost by identifying the most informative examples.
How to Define the Propagation Environment Semantics and Its Application in Scatterer-Based Beam Prediction
Inspired by task-oriented semantic communication and machine learning (ML) powered environment-channel mapping methods, this work aims to provide a new view of the environment from the semantic level, which defines the propagation environment semantics (PES) as a limited set of propagation environment semantic symbols (PESS) for diverse application tasks.
Identifiable Latent Causal Content for Domain Adaptation under Latent Covariate Shift
Within this new paradigm, we present an intricate causal generative model by introducing latent noises across domains, along with a latent content variable and a latent style variable to achieve more nuanced rendering of observational data.
Identifying Weight-Variant Latent Causal Models
The task of causal representation learning aims to uncover latent higher-level causal representations that affect lower-level observations.
Truncated Matrix Power Iteration for Differentiable DAG Learning
Recovering underlying Directed Acyclic Graph (DAG) structures from observational data is highly challenging due to the combinatorial nature of the DAG-constrained optimization problem.
Sparse Structure Search for Parameter-Efficient Tuning
The searched structures preserve more than 99\% fine-tuning performance with 0. 01\% trainable parameters.
RMT-Net: Reject-aware Multi-Task Network for Modeling Missing-not-at-random Data in Financial Credit Scoring
Accordingly, we for the first time propose to model the biased credit scoring data with Multi-Task Learning (MTL).
NTIRE 2022 Challenge on High Dynamic Range Imaging: Methods and Results
The challenge is composed of two tracks with an emphasis on fidelity and complexity constraints: In Track 1, participants are asked to optimize objective fidelity scores while imposing a low-complexity constraint (i. e. solutions can not exceed a given number of operations).
MolMiner: You only look once for chemical structure recognition
Due to a backlog of decades and an increasing amount of these printed literature, there is a high demand for the translation of printed depictions into machine-readable formats, which is known as Optical Chemical Structure Recognition (OCSR).
Adversarial Training-Aided Time-Varying Channel Prediction for TDD/FDD Systems
In this paper, a time-varying channel prediction method based on conditional generative adversarial network (CPcGAN) is proposed for time division duplexing/frequency division duplexing (TDD/FDD) systems.
Sequence-Based Target Coin Prediction for Cryptocurrency Pump-and-Dump
With the proliferation of pump-and-dump schemes (P&Ds) in the cryptocurrency market, it becomes imperative to detect such fraudulent activities in advance to alert potentially susceptible investors.
Enhanced Contour Tracking: a Time-Varying Internal Model Principle-Based Approach
The proposed TV-IMCC is twofold, including an extended position domain framework with master-slave structures for contour regulation, and a time-varying internal model principle-based controller for each axial tracking precision improvement.
Systems Biology: Identifiability analysis and parameter identification via systems-biology informed neural networks
The dynamics of systems biological processes are usually modeled by a system of ordinary differential equations (ODEs) with many unknown parameters that need to be inferred from noisy and sparse measurements.
SympOCnet: Solving optimal control problems with applications to high-dimensional multi-agent path planning problems
Solving high-dimensional optimal control problems in real-time is an important but challenging problem, with applications to multi-agent path planning problems, which have drawn increased attention given the growing popularity of drones in recent years.
MyoPS: A Benchmark of Myocardial Pathology Segmentation Combining Three-Sequence Cardiac Magnetic Resonance Images
Assessment of myocardial viability is essential in diagnosis and treatment management of patients suffering from myocardial infarction, and classification of pathology on myocardium is the key to this assessment.
GFINNs: GENERIC Formalism Informed Neural Networks for Deterministic and Stochastic Dynamical Systems
We propose the GENERIC formalism informed neural networks (GFINNs) that obey the symmetric degeneracy conditions of the GENERIC formalism.
Adaptive Optimizers with Sparse Group Lasso for Neural Networks in CTR Prediction
We develop a novel framework that adds the regularizers of the sparse group lasso to a family of adaptive optimizers in deep learning, such as Momentum, Adagrad, Adam, AMSGrad, AdaHessian, and create a new class of optimizers, which are named Group Momentum, Group Adagrad, Group Adam, Group AMSGrad and Group AdaHessian, etc., accordingly.
Uncertainty-Guided Mixup for Semi-Supervised Domain Adaptation without Source Data
Present domain adaptation methods usually perform explicit representation alignment by simultaneously accessing the source data and target data.
Semi-Supervised Hypothesis Transfer for Source-Free Domain Adaptation
Domain Adaptation has been widely used to deal with the distribution shift in vision, language, multimedia etc.
Automatic Pulmonary Artery-Vein Separation in CT Images using Twin-Pipe Network and Topology Reconstruction
We propose a novel method for automatic separation of pulmonary arteries and veins from chest CT images.
Adaptive Optimizers with Sparse Group Lasso
We develop a novel framework that adds the regularizers to a family of adaptive optimizers in deep learning, such as MOMENTUM, ADAGRAD, ADAM, AMSGRAD, ADAHESSIAN, and create a new class of optimizers, which are named GROUP MOMENTUM, GROUP ADAGRAD, GROUP ADAM, GROUP AMSGRAD and GROUP ADAHESSIAN, etc., accordingly.
Market Impact in Trader-Agents: Adding Multi-Level Order-Flow Imbalance-Sensitivity to Automated Trading Systems
We demonstrate that the new imbalance-sensitive trader-agents introduced here do exhibit market impact effects, and hence are better-suited to operating in markets where impact is a factor of concern or interest, but do not suffer the weaknesses of the methods used by Church & Cliff.
Learning Poisson systems and trajectories of autonomous systems via Poisson neural networks
We propose the Poisson neural networks (PNNs) to learn Poisson systems and trajectories of autonomous systems from data.
Cyclic Label Propagation for Graph Semi-supervised Learning
To overcome these limitations, we introduce a novel framework for graph semi-supervised learning termed as Cyclic Label Propagation (CycProp for abbreviation), which integrates GNNs into the process of label propagation in a cyclic and mutually reinforcing manner to exploit the advantages of both GNNs and LPA.
Deep Reinforcement Learning of Transition States
Combining reinforcement learning (RL) and molecular dynamics (MD) simulations, we propose a machine-learning approach (RL$^\ddag$) to automatically unravel chemical reaction mechanisms.
CL-MAPF: Multi-Agent Path Finding for Car-Like Robots with Kinematic and Spatiotemporal Constraints
In this paper, we give a mathematical formalization of Multi-Agent Path Finding for Car-Like robots (CL-MAPF) problem.
Robotics Multiagent Systems
Brain Tumor Segmentation Network Using Attention-based Fusion and Spatial Relationship Constraint
Considering that multi-modal MR images can reflect different tumor biological properties, we develop a novel multi-modal tumor segmentation network (MMTSN) to robustly segment brain tumors based on multi-modal MR images.
Kagome quantum anomalous Hall effect with high Chern number and large band gap
For the bilayer compound Co6Sn5Se4, it becomes a half-metal, with a relatively flat plateau in its anomalous Hall conductivity corresponding to |C| = 3 near the Fermi level.
Materials Science
Multi-Modality Pathology Segmentation Framework: Application to Cardiac Magnetic Resonance Images
The PRSN is designed to segment pathological region based on the result of ASSN, in which a fusion block based on channel attention is proposed to better aggregate multi-modality information from multi-modality CMR images.
Distributed Linguistic Representations in Decision Making: Taxonomy, Key Elements and Applications, and Challenges in Data Science and Explainable Artificial Intelligence
Distributed linguistic representations are powerful tools for modelling the uncertainty and complexity of preference information in linguistic decision making.
FADACS: A Few-shot Adversarial Domain Adaptation Architecture for Context-Aware Parking Availability Sensing
We quantify the strength of our proposed framework in different cases and compare it to the existing data-driven approaches.
Is Network the Bottleneck of Distributed Training?
As such, we advocate that the real challenge of distributed training is for the network community to develop high-performance network transport to fully utilize the network capacity and achieve linear scale-out.
Time-Stretched Femtosecond Lidar Using Microwave Photonic Signal Processing
After that, the frequency variation of the microwave pulse is uploaded to the first order sidebands.
A Perspective on Deep Learning for Molecular Modeling and Simulations
Deep learning is transforming many areas in science, and it has great potential in modeling molecular systems.
Adaptive-Step Graph Meta-Learner for Few-Shot Graph Classification
The shared sub-structures between training classes and test classes are essential in few-shot graph classification.
Multiplicative Gaussian Particle Filter
We propose a new sampling-based approach for approximate inference in filtering problems.
SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems
We propose new symplectic networks (SympNets) for identifying Hamiltonian systems from data based on a composition of linear, activation and gradient modules.
KerGM: Kernelized Graph Matching
Graph matching plays a central role in such fields as computer vision, pattern recognition, and bioinformatics.
Scalable Global Alignment Graph Kernel Using Random Features: From Node Embedding to Graph Embedding
In particular, RGE is shown to achieve \emph{(quasi-)linear scalability} with respect to the number and the size of the graphs.
Visual Relationship Detection with Low Rank Non-Negative Tensor Decomposition
We address the problem of Visual Relationship Detection (VRD) which aims to describe the relationships between pairs of objects in the form of triplets of (subject, predicate, object).
Hierarchical Graph Pooling with Structure Learning
HGP-SL incorporates graph pooling and structure learning into a unified module to generate hierarchical representations of graphs.
Ranked #1 on
Graph Classification
on PROTEINS
Deep Graphical Feature Learning for the Feature Matching Problem
Rich local representation is a key part of efficient feature matching methods.
Learning Clustered Representation for Complex Free Energy Landscapes
In this paper we first analyzed the inductive bias underlying the data scattered across complex free energy landscapes (FEL), and exploited it to train deep neural networks which yield reduced and clustered representation for the FEL.
Factor Graph Neural Network
Most of the successful deep neural network architectures are structured, often consisting of elements like convolutional neural networks and gated recurrent neural networks.
phq: a Fortran code to compute phonon quasiparticle properties and dispersions
We here introduce a Fortran code that computes anharmonic free energy of solids from first-principles based on our phonon quasiparticle approach.
Materials Science
RetGK: Graph Kernels based on Return Probabilities of Random Walks
Graph-structured data arise in wide applications, such as computer vision, bioinformatics, and social networks.
Aligning Infinite-Dimensional Covariance Matrices in Reproducing Kernel Hilbert Spaces for Domain Adaptation
Domain shift, which occurs when there is a mismatch between the distributions of training (source) and testing (target) datasets, usually results in poor performance of the trained model on the target domain.
Convolutional Sequence to Sequence Model for Human Dynamics
Human motion modeling is a classic problem in computer vision and graphics.
OmicsMapNet: Transforming omics data to take advantage of Deep Convolutional Neural Network for discovery
We developed OmicsMapNet approach to take advantage of existing deep leaning frameworks to analyze high-dimensional omics data as 2-dimensional images.
Learning Deep Gradient Descent Optimization for Image Deconvolution
Extensive experiments on synthetic benchmarks and challenging real-world images demonstrate that the proposed deep optimization method is effective and robust to produce favorable results as well as practical for real-world image deblurring applications.
Depth and Image Restoration From Light Field in a Scattering Medium
First, we present a new single image restoration algorithm which removes backscatter and attenuation from images better than existing methods, and apply it to each view in the light field.
Pairwise Matching Through Max-Weight Bipartite Belief Propagation
Feature matching is a key problem in computer vision and pattern recognition.
Joint Probabilistic Matching Using m-Best Solutions
Matching between two sets of objects is typically approached by finding the object pairs that collectively maximize the joint matching score.
Joint Probabilistic Data Association Revisited
In this paper, we revisit the joint probabilistic data association (JPDA) technique and propose a novel solution based on recent developments in finding the m-best solutions to an integer linear program.
Learning Graph Structure for Multi-Label Image Classification via Clique Generation
Exploiting label dependency for multi-label image classification can significantly improve classification performance.
Managing Multi-Granular Linguistic Distribution Assessments in Large-Scale Multi-Attribute Group Decision Making
Linguistic large-scale group decision making (LGDM) problems are more and more common nowadays.
Constraint Reduction using Marginal Polytope Diagrams for MAP LP Relaxations
We show that using Marginal Polytope Diagrams allows the number of constraints to be reduced without loosening the LP relaxations.
A feasible roadmap for unsupervised deconvolution of two-source mixed gene expressions
Tissue heterogeneity is a major confounding factor in studying individual populations that cannot be resolved directly by global profiling.
