Search Results for author:
Found 122 papers, 39 papers with code
When does Further Pre-training MLM Help? An Empirical Study on Task-Oriented Dialog Pre-training
Further pre-training language models on in-domain data (domain-adaptive pre-training, DAPT) or task-relevant data (task-adaptive pre-training, TAPT) before fine-tuning has been shown to improve downstream tasks’ performances.
Improving Query Graph Generation for Complex Question Answering over Knowledge Base
Most of the existing Knowledge-based Question Answering (KBQA) methods first learn to map the given question to a query graph, and then convert the graph to an executable query to find the answer.
A Comprehensive Study on Post-Training Quantization for Large Language Models
Post-training quantization (\ptq) had been recently shown as a compromising method to reduce memory consumption and/or compute cost for large language models.
MGA: Medical generalist agent through text-guided knowledge transformation
To this end, we propose a novel medical generalist agent, MGA, that can address three kinds of common clinical tasks via clinical reports knowledge transformation.
Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Cases
Improving the deployment efficiency of transformer-based language models has been challenging given their high computation and memory cost.
SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction
SuperScaler is a system that facilitates the design and generation of highly flexible parallelization plans.
DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing
Recent advances on deep learning models come at the price of formidable training cost.
Iterative Data Refinement for Self-Supervised MR Image Reconstruction
Then, we design an effective self-supervised training data refinement method to reduce this data bias.
Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformers
Large-scale transformer models have become the de-facto architectures for various machine learning applications, e. g., CV and NLP.
Uncertainty-Aware Multi-Parametric Magnetic Resonance Image Information Fusion for 3D Object Segmentation
Multi-parametric magnetic resonance (MR) imaging is an indispensable tool in the clinic.
Semi-Supervised and Self-Supervised Collaborative Learning for Prostate 3D MR Image Segmentation
The proposed framework is very useful in clinical applications when training data with dense annotations are difficult to obtain.
DIGEST: Deeply supervIsed knowledGE tranSfer neTwork learning for brain tumor segmentation with incomplete multi-modal MRI scans
In this work, we propose a Deeply supervIsed knowledGE tranSfer neTwork (DIGEST), which achieves accurate brain tumor segmentation under different modality-missing scenarios.
Adaptive PromptNet For Auxiliary Glioma Diagnosis without Contrast-Enhanced MRI
Multi-contrast magnetic resonance imaging (MRI)-based automatic auxiliary glioma diagnosis plays an important role in the clinic.
Reversed Image Signal Processing and RAW Reconstruction. AIM 2022 Challenge Report
Cameras capture sensor RAW images and transform them into pleasant RGB images, suitable for the human eyes, using their integrated Image Signal Processor (ISP).
From WSI-level to Patch-level: Structure Prior Guided Binuclear Cell Fine-grained Detection
The coarse detection network is a multi-task detection framework based on circular bounding boxes for cells detection, and central key points for nucleus detection.
SelfCoLearn: Self-supervised collaborative learning for accelerating dynamic MR imaging
Lately, deep learning has been extensively investigated for accelerating dynamic magnetic resonance (MR) imaging, with encouraging progresses achieved.
BiFeat: Supercharge GNN Training via Graph Feature Quantization
We identify the main accuracy impact factors in graph feature quantization and theoretically prove that BiFeat training converges to a network where the loss is within $\epsilon$ of the optimal loss of uncompressed network.
Rethinking Query-Key Pairwise Interactions in Vision Transformers
Vision Transformers have achieved state-of-the-art performance in many visual tasks.
DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
DeepSpeed Inference reduces latency by up to 7. 3X over the state-of-the-art for latency-oriented scenarios and increases throughput by over 1. 5x for throughput-oriented scenarios.
SJ-HD^2R: Selective Joint High Dynamic Range and Denoising Imaging for Dynamic Scenes
To further leverage these two paradigms, we propose a selective and joint HDR and denoising (SJ-HD$^2$R) imaging framework, utilizing scenario-specific priors to conduct the path selection with an accuracy of more than 93. 3$\%$.
Rethinking the optimization process for self-supervised model-driven MRI reconstruction
Recovering high-quality images from undersampled measurements is critical for accelerated MRI reconstruction.
PARCEL: Physics-based Unsupervised Contrastive Representation Learning for Multi-coil MR Imaging
However, in the absence of high-quality, fully sampled datasets for training, the performance of these methods is limited.
Expert Knowledge-guided Geometric Representation Learning for Magnetic Resonance Imaging-based Glioma Grading
Geometric manifolds of hand-crafted features and learned features are constructed to mine the implicit relationship between deep learning and radiomics, and therefore to dig mutual consent and essential representation for the glioma grades.
Radiomic biomarker extracted from PI-RADS 3 patients support more eìcient and robust prostate cancer diagnosis: a multi-center study
Prostate Imaging Reporting and Data System (PI-RADS) based on multi-parametric MRI classi\^ees patients into 5 categories (PI-RADS 1-5) for routine clinical diagnosis guidance.
Finding One Missing Puzzle of Contextual Word Embedding: Representing Contexts as Manifold
The current understanding of contextual word embedding interprets the representation by associating each token to a vector that is dynamically modulated by the context.
Demystifying Hyperparameter Optimization in Federated Learning
Such data heterogeneity and privacy requirements bring unique challenges for learning hyperparameter optimization as the training dynamics change across clients even within the same training round and they are difficult to measure due to privacy constraints.
Self-Supervised Learning for MRI Reconstruction with a Parallel Network Training Framework
Specifically, during model optimization, two subsets are constructed by randomly selecting part of k-space data from the undersampled data and then fed into two parallel reconstruction networks to perform information recovery.
A Novel Hybrid Convolutional Neural Network for Accurate Organ Segmentation in 3D Head and Neck CT Images
An essential step of RT planning is the accurate segmentation of various organs-at-risks (OARs) in HaN CT images.
Group Shift Pointwise Convolution for Volumetric Medical Image Segmentation
Recent studies have witnessed the effectiveness of 3D convolutions on segmenting volumetric medical images.
VIL-100: A New Dataset and A Baseline Model for Video Instance Lane Detection
Lane detection plays a key role in autonomous driving.
Blind Image Quality Assessment for MRI with A Deep Three-dimensional content-adaptive Hyper-Network
However, the performance of these methods is limited due to the utilization of simple content-non-adaptive network parameters and the waste of the important 3D spatial information of the medical images.
Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report
A detailed description of all models developed in the challenge is provided in this paper.
NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results
This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results.
LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding
text, table, image) and propose a novel layout-aware multimodal hierarchical framework, LAMPreT, to model the blocks and the whole document.
Supervised Learning in the Presence of Noise: Application in ICD-10 Code Classification
We first identify ICD-10 codes that human coders tend to misuse or confuse, based on the codes' locations in the ICD-10 hierarchy, the types of the codes, and baseline classifier's prediction behaviors; we then develop a novel training strategy that accounts for such noise.
Ion-beam-assisted characterization of quinoline-insoluble particles in nuclear graphite
A much higher concentration of QI particles in NBG-18 than IG-110 was characterized and is suggested to be responsible for the smaller maximum dimensional shrinkage of NBG-18 than IG-110 during irradiation.
Applied Physics
Efficient Multi-Stage Video Denoising with Recurrent Spatio-Temporal Fusion
Then, a denoising stage removes the noise in the fused frame.
Ranked #3 on
Video Denoising
on CRVD
Measurement of the absolute branching fractions for purely leptonic $D_s^+$ decays
Constraining our measurement to the Standard Model expectation of lepton universality ($R=9. 75$), we find the more precise results $\cal B(D_s^+\to \tau^+\nu_\tau) = (5. 22\pm0. 10\pm 0. 14)\times10^{-2}$ and $A_{\it CP}(\tau^\pm\nu_\tau) = (-0. 1\pm1. 9\pm1. 0)\%$.
High Energy Physics - Experiment
Cross section measurement of $e^+e^- \to p\bar{p}η$ and $e^+e^- \to p\bar{p}ω$ at center-of-mass energies between 3.773 GeV and 4.6 GeV
Based on $14. 7~\textrm{fb}^{-1}$ of $e^+e^-$ annihilation data collected with the BESIII detector at the BEPCII collider at 17 different center-of-mass energies between $3. 7730~\textrm{GeV}$ and $4. 5995~\textrm{GeV}$, Born cross sections of the two processes $e^+e^- \to p\bar{p}\eta$ and $e^+e^- \to p\bar{p}\omega$ are measured for the first time.
High Energy Physics - Experiment
Hyperspectral Image Denoising With Realistic Data
On the other hand, we propose an accurate HSI noise model which matches the distribution of real data well and can be employed to synthesize realistic dataset.
Robustness Testing of Language Understanding in Task-Oriented Dialog
Most language understanding models in task-oriented dialog systems are trained on a small amount of annotated training data, and evaluated in a small set from the same distribution.
Measurements of the center-of-mass energies of $e^{+}e^{-}$ collisions at BESIII
During the 2016-17 and 2018-19 running periods, the BESIII experiment collected 7. 5~fb$^{-1}$ of $e^+e^-$ collision data at center-of-mass energies ranging from 4. 13 to 4. 44 GeV.
High Energy Physics - Experiment
Storms and the Depletion of Ammonia in Jupiter: II. Explaining the Juno Observations
In parallel, the Equatorial Zone is peculiar for its absence of lightning, which is otherwise prevalent most everywhere else on the planet.
Earth and Planetary Astrophysics Solar and Stellar Astrophysics
The water abundance in Jupiter's equatorial zone
Oxygen is the most common element after hydrogen and helium in Jupiter's atmosphere, and may have been the primary condensable (as water ice) in the protoplanetary disk.
Earth and Planetary Astrophysics
Annotation-efficient deep learning for automatic medical image segmentation
Automatic medical image segmentation plays a critical role in scientific research and medical care.
Search for the reaction $e^{+}e^{-} \rightarrow π^{+}π^{-} χ_{cJ}$ and a charmonium-like structure decaying to $χ_{cJ}π^{\pm}$ between 4.18 and 4.60 GeV
We search for the process $e^{+}e^{-}\rightarrow \pi ^{+}\pi ^{-} \chi_{cJ}$ ($J=0, 1, 2$) and for a charged charmonium-like state in the $\pi ^{\pm} \chi_{cJ}$ subsystem.
High Energy Physics - Experiment
PaGraph: Scaling GNN Training on Large Graphs via Computation-aware Caching and Partitioning
Emerging graph neural networks (GNNs) have extended the successes of deep learning techniques against datasets like images and texts to more complex graph-structured data.
Leveraging Semantic and Lexical Matching to Improve the Recall of Document Retrieval Systems: A Hybrid Approach
A hybrid approach, which leverages both semantic (deep neural network-based) and lexical (keyword matching-based) retrieval models, is proposed.
Measuring galaxy abundance and clustering at high redshift from incomplete spectroscopic data: Tests on mock catalogs
We also quantify improvements expected in the PFS-like galaxy survey relative to zCOSMOS and VIPERS surveys.
Astrophysics of Galaxies
Simultaneously-Collected Multimodal Lying Pose Dataset: Towards In-Bed Human Pose Monitoring under Adverse Vision Conditions
Computer vision (CV) has achieved great success in interpreting semantic meanings from images, yet CV algorithms can be brittle for tasks with adverse vision conditions and the ones suffering from data/label pair limitation.
A coarse-to-fine framework for unsupervised multi-contrast MR image deformable registration with dual consistency constraint
Multi-contrast magnetic resonance (MR) image registration is useful in the clinic to achieve fast and accurate imaging-based disease diagnosis and treatment planning.
Model independent determination of the spin of the $Ω^{-}$ and its polarization alignment in $ψ(3686)\rightarrowΩ^{-}\barΩ^{+}$
We present an analysis of the process $\psi(3686) \to \Omega^- \bar{\Omega}^+$ ($\Omega^-\to K^-\Lambda$, $\bar{\Omega}^+\to K^+\bar{\Lambda}$, $\Lambda\to p\pi^-$, $\bar{\Lambda}\to \bar{p}\pi^+$) based on a data set of $448\times 10^6$ $\psi(3686)$ decays collected with the BESIII detector at the BEPCII electron-positron collider.
High Energy Physics - Experiment
A Complex KBQA System using Multiple Reasoning Paths
Multi-hop knowledge based question answering (KBQA) is a complex task for natural language understanding.
Towards in-store multi-person tracking using head detection and track heatmaps
In addition, we describe an illustrative example of the use of this dataset for tracking participants based on a head tracking model in an effort to minimize errors due to occlusion.
Classification Trees for Imbalanced and Sparse Data: Surface-to-Volume Regularization
When data are limited in one or more of the classes, the estimated decision boundaries are often irregularly shaped due to the limited sample size, leading to poor generalization error.
Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching
In order to better capture sentence level semantic relations within a document, we pre-train the model with a novel masked sentence block language modeling task in addition to the masked word language modeling task used by BERT.
Hierarchically Fair Federated Learning
When the federated learning is adopted among competitive agents with siloed datasets, agents are self-interested and participate only if they are fairly rewarded.
Downstream Model Design of Pre-trained Language Model for Relation Extraction Task
We believe that PLMs can also be used to solve the relation extraction problem, but it is necessary to establish a specially designed downstream task model or even loss function for dealing with complicated relations.
Ranked #3 on
Relation Extraction
on SemEval-2010 Task 8
Incorporating Expert Prior in Bayesian Optimisation via Space Warping
In this paper, we represent the prior knowledge about the function optimum through a prior distribution.
Relating the structure of dark matter halos to their assembly and environment
Using decision trees built with the random ensemble method, we find that the correlation between halo concentration and assembly history is tight: more than 60% of the variance in halo concentration can be explained by assembly history alone.
Astrophysics of Galaxies Cosmology and Nongalactic Astrophysics
Incorporating Expert Prior Knowledge into Experimental Design via Posterior Sampling
Again, it is unknown how to incorporate the expert prior knowledge about the global optimum into Bayesian optimization process.
DLSpec: A Deep Learning Task Exchange Specification
Deep Learning (DL) innovations are being introduced at a rapid pace.
MLModelScope: A Distributed Platform for Model Evaluation and Benchmarking at Scale
Machine Learning (ML) and Deep Learning (DL) innovations are being introduced at such a rapid pace that researchers are hard-pressed to analyze and study them.
Non-rigid Registration Method between 3D CT Liver Data and 2D Ultrasonic Images based on Demons Model
The non-rigid registration between CT data and ultrasonic images of liver can facilitate the diagnosis and treatment, which has been widely studied in recent years.
The Design and Implementation of a Scalable DL Benchmarking Platform
MLModelScope defines abstractions for frameworks and supports board range of DL models and evaluation scenarios.
DLBricks: Composable Benchmark Generation to Reduce Deep Learning Benchmarking Effort on CPUs (Extended)
We show that DLBricks provides an accurate performance estimate for the DL models and reduces the benchmarking time across systems (e. g. within $95\%$ accuracy and up to $4. 4\times$ benchmarking time speedup on Amazon EC2 c5. xlarge).
Benanza: Automatic $μ$Benchmark Generation to Compute "Lower-bound" Latency and Inform Optimizations of Deep Learning Models on GPUs
An important venue for such improvement is to profile the execution of these models and characterize their performance to identify possible optimization opportunities.
Self-Adaptive Network Pruning
Our method introduces a general Saliency-and-Pruning Module (SPM) for each convolutional layer, which learns to predict saliency scores and applies pruning for each channel.
MLModelScope: A Distributed Platform for ML Model Evaluation and Benchmarking at Scale
Machine Learning (ML) and Deep Learning (DL) innovations are being introduced at such a rapid pace that researchers are hard-pressed to analyze and study them.
AI Matrix: A Deep Learning Benchmark for Alibaba Data Centers
Alibaba has China's largest e-commerce platform.
Interpretable Biomanufacturing Process Risk and Sensitivity Analyses for Quality-by-Design and Stability Control
While biomanufacturing plays a significant role in supporting the economy and ensuring public health, it faces critical challenges, including complexity, high variability, lengthy lead time, and very limited process data, especially for personalized new cell and gene biotherapeutics.
XSP: Across-Stack Profiling and Analysis of Machine Learning Models on GPUs
Such an endeavor is challenging as the characteristics of an ML model depend on the interplay between the model, framework, system libraries, and the hardware (or the HW/SW stack).
Delving Deep Into Hybrid Annotations for 3D Human Recovery in the Wild
Specifically, we focus on the challenging task of in-the-wild 3D human recovery from single images when paired 3D annotations are not fully available.
Model-based Convolutional De-Aliasing Network Learning for Parallel MR Imaging
Three main contributions have been made: a de-aliasing reconstruction model was proposed to accelerate parallel MR imaging with deep learning exploring both spatial redundancy and multi-coil correlations; a split Bregman iteration algorithm was developed to solve the model efficiently; and unlike most existing parallel imaging methods which rely on the accuracy of the estimated multi-coil sensitivity, the proposed method can perform parallel reconstruction from undersampled data without explicit sensitivity calculation.
Learning Cross-Modal Deep Representations for Multi-Modal MR Image Segmentation
From the different modalities, one modality that contributes most to the results is selected as the master modality, which supervises the information selection of the other assistant modalities.
One-shot Face Reenactment
However, in real-world scenario end-users often only have one target face at hand, rendering existing methods inapplicable.
Accelerating Experimental Design by Incorporating Experimenter Hunches
In this paper, we consider per-variable monotonic trend in the underlying property that results in a unimodal trend in those variables for a target value optimization.
X-Net: Brain Stroke Lesion Segmentation Based on Depthwise Separable Convolution and Long-range Dependencies
Recently, approaches based on deep learning and methods for contextual information extraction have served in many image segmentation tasks.
CLCI-Net: Cross-Level fusion and Context Inference Networks for Lesion Segmentation of Chronic Stroke
To address these challenges, this paper proposes a Cross-Level fusion and Context Inference Network (CLCI-Net) for the chronic stroke lesion segmentation from T1-weighted MR images.