Search Results for author:
Found 98 papers, 32 papers with code
Terraced Compression Method with Automated Threshold Selection for Multidimensional Image Clustering of Heterogeneous Bodies
The results illustrate the method's efficacy in detecting and identifying various heterogeneous body types, depths, and thicknesses.
Separate, Dynamic and Differentiable (SMART) Pruner for Block/Output Channel Pruning on Computer Vision Tasks
In our experiments, the SMART pruner consistently demonstrated its superiority over existing pruning methods across a wide range of tasks and models on block and output channel pruning.
FineDiffusion: Scaling up Diffusion Models for Fine-grained Image Generation with 10,000 Classes
The class-conditional image generation based on diffusion models is renowned for generating high-quality and diverse images.
FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Models
In the decoder, we further modulate features from the dual streams based on a given content image and the corresponding style text prompt for precise style transfer.
Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation
Additionally, Parrot employs a joint optimization approach for the T2I model and the prompt expansion network, facilitating the generation of quality-aware text prompts, thus further enhancing the final image quality.
Predicting Infant Brain Connectivity with Federated Multi-Trajectory GNNs using Scarce Data
The three key innovations of FedGmTE-Net++ are: (i) presenting the first federated learning framework specifically designed for brain multi-trajectory evolution prediction in a data-scarce environment, (ii) incorporating an auxiliary regularizer in the local objective function to exploit all the longitudinal brain connectivity within the evolution trajectory and maximize data utilization, (iii) introducing a two-step imputation process, comprising a preliminary KNN-based precompletion followed by an imputation refinement step that employs regressors to improve similarity scores and refine imputations.
Rich Human Feedback for Text-to-Image Generation
We show that the predicted rich human feedback can be leveraged to improve image generation, for example, by selecting high-quality training data to finetune and improve the generative models, or by creating masks with predicted heatmaps to inpaint the problematic regions.
UniAR: Unifying Human Attention and Response Prediction on Visual Content
Such a model would enable predicting subjective feedback such as overall satisfaction or aesthetic quality ratings, along with the underlying human attention or interaction heatmaps and viewing order, enabling designers and content-creation models to optimize their creation for human-centric improvements.
Holistic Evaluation of GPT-4V for Biomedical Imaging
GPT-4V represents a breakthrough in artificial general intelligence (AGI) for computer vision, with applications in the biomedical domain.
AUC-mixup: Deep AUC Maximization with Mixup
While deep AUC maximization (DAM) has shown remarkable success on imbalanced medical tasks, e. g., chest X-rays classification and skin lesions classification, it could suffer from severe overfitting when applied to small datasets due to its aggressive nature of pushing prediction scores of positive data away from that of negative data.
The Road to On-board Change Detection: A Lightweight Patch-Level Change Detection Network via Exploring the Potential of Pruning and Pooling
In addition, our method reduces more than 60% memory costs of the subsequent pixel-level CD processing stage.
Self-supervised Fetal MRI 3D Reconstruction Based on Radiation Diffusion Generation Model
Finally, the experimental results on real-world fetal brain MRI stacks demonstrate the state-of-the-art performance of our method.
A Zero-Shot Language Agent for Computer Control with Structured Reflection
To perform a task, recent works often require a model to learn from trace examples of the task via either supervised learning or few/many-shot prompting.
Automatic Macro Mining from Interaction Traces at Scale
Macros are building block tasks of our everyday smartphone activity (e. g., "login", or "booking a flight").
PrototypeFormer: Learning to Explore Prototype Relationships for Few-shot Image Classification
In particular, our method achieves 97. 07% and 90. 88% on 5-way 5-shot and 5-way 1-shot tasks of miniImageNet, which surpasses the state-of-the-art results with accuracy of 7. 27% and 8. 72%, respectively.
Cartoondiff: Training-free Cartoon Image Generation with Diffusion Transformer Models
Image cartoonization has attracted significant interest in the field of image generation.
Early Autism Diagnosis based on Path Signature and Siamese Unsupervised Feature Compressor
Specifically, we propose a Siamese verification framework to extend the scarce data, and an unsupervised compressor to alleviate data imbalance by extracting key features.
Improving Federated Aggregation with Deep Unfolding Networks
By incorporating unbiased weights into the model, the proposed approach effectively addresses quality-aware aggregation under the heterogeneity of the participating clients and the FL environment.
Quantum Federated Learning: Analysis, Design and Implementation Challenges
Quantum Federated Learning (QFL) has gained significant attention due to quantum computing and machine learning advancements.
Decentralized Quantum Federated Learning for Metaverse: Analysis, Design and Implementation
To this end, we develop a decentralized and trustworthy quantum federated learning (QFL) framework.
Artificial General Intelligence for Medical Imaging
In this review, we explore the potential applications of Artificial General Intelligence (AGI) models in healthcare, focusing on foundational Large Language Models (LLMs), Large Vision Models, and Large Multimodal Models.
LibAUC: A Deep Learning Library for X-Risk Optimization
This paper introduces the award-winning deep learning (DL) library called LibAUC for implementing state-of-the-art algorithms towards optimizing a family of risk functions named X-risks.
PaLI-X: On Scaling up a Multilingual Vision and Language Model
We present the training recipe and results of scaling up PaLI-X, a multilingual vision and language model, both in terms of size of the components and the breadth of its training task mixture.
Ranked #1 on
Fine-Grained Image Recognition
on OVEN
Learning a Single Convolutional Layer Model for Low Light Image Enhancement
Low-light image enhancement (LLIE) aims to improve the illuminance of images due to insufficient light exposure.
Do We Need an Encoder-Decoder to Model Dynamical Systems on Networks?
As deep learning gains popularity in modelling dynamical systems, we expose an underappreciated misunderstanding relevant to modelling dynamics on networks.
Secure Communication Model For Quantum Federated Learning: A Post Quantum Cryptography (PQC) Framework
We design a model of Post Quantum Cryptography (PQC) Quantum Federated Learning (QFL).
Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task
To this end, in this study, we evaluate the performance of ChatGPT/GPT-4 on a radiology NLI task and compare it to other models fine-tuned specifically on task-related data samples.
IterativePFN: True Iterative Point Cloud Filtering
Consequently, a fundamental 3D vision task is the removal of noise, known as point cloud filtering or denoising.
COMIC: An Unsupervised Change Detection Method for Heterogeneous Remote Sensing Images Based on Copula Mixtures and Cycle-Consistent Adversarial Networks
For this problem, an unsupervised change detection method has been proposed recently based on the image translation technique of Cycle-Consistent Adversarial Networks (CycleGANs), where one image is translated from its original modality to the modality of the other image so that the difference map can be obtained by performing arithmetical subtraction.
Deep neural operator for learning transient response of interpenetrating phase composites subject to dynamic loading
After an offline training, the DNO model can act as surrogate of physics-based FEA to predict the transient mechanical response in terms of reaction force and stress distribution of the IPCs to various strain loads in one second at an accuracy of 98%.
NN-Copula-CD: A Copula-Guided Interpretable Neural Network for Change Detection in Heterogeneous Remote Sensing Images
Change detection (CD) in heterogeneous remote sensing images is a practical and challenging issue for real-life emergencies.
Ins-ATP: Deep Estimation of ATP for Organoid Based on High Throughput Microscopic Images
Performing ATP bioluminescence causes cell lysis of organoids, so it is impossible to observe organoids' long-term viability changes after medication continually.
Post Quantum Secure Blockchain-based Federated Learning for Mobile Edge Computing
We aim to employ Federated Learning (FL) and prominent features of blockchain into MEC architecture such as connected autonomous vehicles to enable complete decentralization, immutability, and rewarding mechanisms simultaneously.
$\rm A^2Q$: Aggregation-Aware Quantization for Graph Neural Networks
Through an in-depth analysis of the topology of GNNs, we observe that the topology of the graph leads to significant differences between nodes, and most of the nodes in a graph appear to have a small aggregation value.
PLay: Parametrically Conditioned Layout Generation using Latent Diffusion
Layout design is an important task in various design fields, including user interface, document, and graphic design.
Detecting Change Intervals with Isolation Distributional Kernel
Detecting abrupt changes in data distribution is one of the most significant tasks in streaming data analysis.
3DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models
Text-guided diffusion models have shown superior performance in image/video generation and editing.
MCTNet: A Multi-Scale CNN-Transformer Network for Change Detection in Optical Remote Sensing Images
For the task of change detection (CD) in remote sensing images, deep convolution neural networks (CNNs)-based methods have recently aggregated transformer modules to improve the capability of global feature extraction.
Spotlight: Mobile UI Understanding using Vision-Language Models with a Focus
Specifically, we enhance a vision-language model that only takes the screenshot of the UI and a region of interest on the screen -- the focus -- as the input.
MUG: Interactive Multimodal Grounding on User Interfaces
To investigate the problem, we create a new dataset that consists of 77, 820 sequences of human user-agent interaction on mobile interfaces in which 20% involves multiple rounds of interactions.
Enabling Conversational Interaction with Mobile UI using Large Language Models
This paper investigates the feasibility of enabling versatile conversational interactions with mobile UIs using a single LLM.
Contrastive Learning for Joint Normal Estimation and Point Cloud Filtering
Existing methods usually perform normal estimation and filtering separately and often show sensitivity to noise and/or inability to preserve sharp geometric features such as corners and edges.
Longitudinal Prediction of Postnatal Brain Magnetic Resonance Images via a Metamorphic Generative Adversarial Network
In this paper, we propose a deep learning framework to predict missing scans from acquired scans, catering to longitudinal infant studies.
PalQuant: Accelerating High-precision Networks on Low-precision Accelerators
In this paper, we propose the PArallel Low-precision Quantization (PalQuant) method that approximates high-precision computations via learning parallel low-precision representations from scratch.
Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization
The key issue is to track and estimate a sequence of $\mathbf g(\mathbf{w})=(g_1(\mathbf{w}), \ldots, g_m(\mathbf{w}))$ across iterations, where $\mathbf g(\mathbf{w})$ has $m$ blocks and it is only allowed to probe $\mathcal{O}(1)$ blocks to attain their stochastic values and Jacobians.
DTG-SSOD: Dense Teacher Guidance for Semi-Supervised Object Detection
Specifically, we propose the Inverse NMS Clustering (INC) and Rank Matching (RM) to instantiate the dense supervision, without the widely used, conventional sparse pseudo labels.
FAIR-BFL: Flexible and Incentive Redesign for Blockchain-based Federated Learning
These impending challenges in the design philosophy of FL call for blockchain-based federated learning (BFL) due to the benefits of coupling FL and blockchain (e. g., democracy, incentive, and immutability).
Ranked #1 on
Malicious Detection
on MNIST
SemMAE: Semantic-Guided Masking for Learning Masked Autoencoders
In this paper, we explore a potential visual analogue of words, i. e., semantic parts, and we integrate semantic information into the training process of MAE by proposing a Semantic-Guided Masking strategy.
Representing Brain Anatomical Regularity and Variability by Few-Shot Embedding
Our experimental results show that: 1) the learned embedding vectors can quantitatively encode the commonality and individuality of cortical folding patterns; 2) with the embeddings we can robustly infer the complicated many-to-many anatomical correspondences among different brains and 3) our model can be successfully transferred to new populations with very limited training samples.
Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis
We use a deep learning based approach to predict whether a selected element in a mobile UI screenshot will be perceived by users as tappable, based on pixels only instead of view hierarchies required by previous work.
PseCo: Pseudo Labeling and Consistency Training for Semi-Supervised Object Detection
Specifically, for pseudo labeling, existing works only focus on the classification score yet fail to guarantee the localization precision of pseudo boxes; For consistency training, the widely adopted random-resize training only considers the label-level consistency but misses the feature-level one, which also plays an important role in ensuring the scale invariance.
A Multi-size Kernel based Adaptive Convolutional Neural Network for Bearing Fault Diagnosis
Ball mixing is a ball bearing production quality problem that is difficult to identify using traditional frequency domain analysis methods since it requires high frequency resolutions of the measurement signals and results in a long analyzing time.
When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee
In this paper, we propose systematic and efficient gradient-based methods for both one-way and two-way partial AUC (pAUC) maximization that are applicable to deep learning.
Adversarial Examples for Good: Adversarial Examples Guided Imbalanced Learning
Adversarial examples are inputs for machine learning models that have been designed by attackers to cause the model to make mistakes.
Learning to Denoise Raw Mobile UI Layouts for Improving Datasets at Scale
The layout of a mobile screen is a critical data source for UI design research and semantic understanding of the screen.
SimViT: Exploring a Simple Vision Transformer with sliding windows
Although vision Transformers have achieved excellent performance as backbone models in many vision tasks, most of them intend to capture global relations of all tokens in an image or a window, which disrupts the inherent spatial and local correlations between patches in 2D structure.
VUT: Versatile UI Transformer for Multi-Modal Multi-Task User Interface Modeling
Our model consists of a multimodal Transformer encoder that jointly encodes UI images and structures, and performs UI object detection when the UI structures are absent in the input.
Knowledge Distillation for Object Detection via Rank Mimicking and Prediction-guided Feature Imitation
Based on the two observations, we propose Rank Mimicking (RM) and Prediction-guided Feature Imitation (PFI) for distilling one-stage detectors, respectively.
Creating User Interface Mock-ups from High-Level Text Descriptions with Deep-Learning Models
The design process of user interfaces (UIs) often begins with articulating high-level design goals.
A Few-shot Learning Graph Multi-Trajectory Evolution Network for Forecasting Multimodal Baby Connectivity Development from a Baseline Timepoint
To the best of our knowledge, this is the first teacher-student architecture tailored for brain graph multi-trajectory growth prediction that is based on few-shot learning and generalized to graph neural networks (GNNs).
VUT: Versatile UI Transformer for Multimodal Multi-Task User Interface Modeling
Our model consists of a multimodal Transformer encoder that jointly encodes UI images and structures, and performs UI object detection when the UI structures are absent in the input.
Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning
Mobile User Interface Summarization generates succinct language descriptions of mobile screens for conveying important contents and functionalities of the screen, which can be useful for many language-based application scenarios.
HelpViz: Automatic Generation of Contextual Visual MobileTutorials from Text-Based Instructions
We present HelpViz, a tool for generating contextual visual mobile tutorials from text-based instructions that are abundant on the web.
Hilbert Sinkhorn Divergence for Optimal Transport
Sinkhorn divergence has become a very popular metric to compare probability distributions in optimal transport.
Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding
Attentional mechanisms are order-invariant.
Gate-Tunable Optical Extinction of Graphene Nanoribbon Nanoclusters
We investigate the optical response of graphene nanoribbons (GNRs) using the broadband nonlinear generation and detection capabilities of nanoscale junctions created at the LaAlO$_3$/SrTiO$_3$ interface.
Mesoscale and Nanoscale Physics Optics
Hardware Acceleration of Fully Quantized BERT for Efficient Natural Language Processing
BERT is the most recent Transformer-based model that achieves state-of-the-art performance in various NLP tasks.
Spacewalker: Rapid UI Design Exploration Using Lightweight Markup Enhancement and Crowd Genetic Programming
Based on our experiments, Spacewalker allows designers to effectively search a large design space of a UI, using the language they are familiar with, and improve their design rapidly at a minimal cost.
Pressure-induced Superconductivity in dual-topological semimetal Pt2HgSe3
Recently monolayer jacutingaite (Pt2HgSe3), a naturally occurring exfoliable mineral, discovered in Brazil in 2008, has been theoretically predicted as a candidate quantum spin Hall system with a 0. 5 eV band gap, while the bulk form is one of only a few known dual-topological insulators which may host different surface states protected by symmetries.
Band Gap
Superconductivity
Materials Science
Dynamic Dual Gating Neural Networks
In particular, dynamic dual gating can provide 59. 7% saving in computing of ResNet50 with 76. 41% top-1 accuracy on ImageNet, which has advanced the state-of-the-art.
Decentralized Federated Learning via Mutual Knowledge Transfer
In this paper, we investigate the problem of decentralized federated learning (DFL) in Internet of things (IoT) systems, where a number of IoT clients train models collectively for a common task without sharing their private training data in the absence of a central server.
The Impact of Isolation Kernel on Agglomerative Hierarchical Clustering Algorithms
In this paper, we identify the root cause of this issue and show that the use of a data-dependent kernel (instead of distance or existing kernel) provides an effective means to address it.
Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements
We propose widget captioning, a novel task for automatically generating language descriptions for UI elements from multimodal input including both the image and the structural representations of user interfaces.
Deep Modeling of Growth Trajectories for Longitudinal Prediction of Missing Infant Cortical Surfaces
Charting cortical growth trajectories is of paramount importance for understanding brain development.
Deep Fiducial Inference
Since the mid-2000s, there has been a resurrection of interest in modern modifications of fiducial inference.
Multi-Site Infant Brain Segmentation Algorithms: The iSeg-2019 Challenge
Deep learning-based methods have achieved state-of-the-art performance; however, one of major limitations is that the learning-based methods may suffer from the multi-site issue, that is, the models trained on a dataset from one site may not be applicable to the datasets acquired from other sites with different imaging protocols/scanners.
An Auto-Context Deformable Registration Network for Infant Brain MRI
Deformable image registration is fundamental to longitudinal and population analysis.
Change Detection in Heterogeneous Optical and SAR Remote Sensing Images via Deep Homogeneous Feature Fusion
To solve this problem, this paper presents a new homogeneous transformation model termed deep homogeneous feature fusion (DHFF) based on image style transfer (IST).
The Medical Scribe: Corpus Development and Model Performance Analyses
We used this annotation scheme to label a corpus of about 6k clinical encounters.
TopoAna: A generic tool for the event type analysis of inclusive Monte-Carlo samples in high energy physics experiments
To help analysts obtain the physics process information from the truth information of the samples, we develop a physics process analysis program, TopoAna, with C++, ROOT, and LaTeX.
High Energy Physics - Experiment
Performance of regression models as a function of experiment noise
A challenge in developing machine learning regression models is that it is difficult to know whether maximal performance has been reached on a particular dataset, or whether further model improvement is possible.
Human-centric Metric for Accelerating Pathology Reports Annotation
However, manual classification for a huge number of reports on multiple tasks is labor-intensive.
A System-Level Solution for Low-Power Object Detection
As a case study, we evaluate our object detection system on a real-world surveillance video with input size of 512x512, and it turns out that the system can achieve an inference speed of 18 fps at the cost of 6. 9W (with display) with an mAP of 66. 4 verified on the PASCAL VOC 2012 dataset.
Learning to Infer Entities, Properties and their Relations from Clinical Conversations
Recently we proposed the Span Attribute Tagging (SAT) Model (Du et al., 2019) to infer clinical entities (e. g., symptoms) and their properties (e. g., duration).
Audio De-identification - a New Entity Recognition Task
To this end, we define the task of audio de-ID, in which audio spans with entity mentions should be detected.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Bidirectional RNN-based Few-shot Training for Detecting Multi-stage Attack
"Feint Attack" is simulated by the real attack inserted in the normal causal attack chain, and the addition of the real attack destroys the causal relationship of the original attack chain.
FRNET: Flattened Residual Network for Infant MRI Skull Stripping
Skull stripping for brain MR images is a basic segmentation task.
Spherical U-Net on Cortical Surfaces: Methods and Applications
In this paper, by leveraging the regular and consistent geometric structure of the resampled cortical surface mapped onto the spherical space, we propose a novel convolution filter analogous to the standard convolution on the image grid.
Audio De-identification: A New Entity Recognition Task
To this end, we define the task of audio de-ID, in which audio spans with entity mentions should be detected.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Deep Morphological Simplification Network (MS-Net) for Guided Registration of Brain Magnetic Resonance Images
Thus, the two trajectories can act as a bridge to link the fixed and the moving images, and guide their registration.
Training Binary Weight Networks via Semi-Binary Decomposition
In this paper, we propose a novel semi-binary decomposition method which decomposes a matrix into two binary matrices and a diagonal matrix.
Gradient Band-based Adversarial Training for Generalized Attack Immunity of A3C Path Finding
In addition, we propose Gradient Band-based Adversarial Training, which trained with a single randomly choose dominant adversarial example without taking any modification, to realize the "1:N" attack immunity for generalized dominant adversarial examples.
Recent Advances in Efficient Computation of Deep Convolutional Neural Networks
As for hardware implementation of deep neural networks, a batch of accelerators based on FPGA/ASIC have been proposed in recent years.
Noise Reduction Methods for Distantly Supervised Biomedical Relation Extraction
Distant supervision has been applied to automatically generate labeled data for biomedical relation extraction.
Scale-constrained Unsupervised Evaluation Method for Multi-scale Image Segmentation
Unsupervised evaluation of segmentation quality is a crucial step in image segmentation applications.
Efficient Regularized Regression for Variable Selection with L0 Penalty
Therefore, it is natural to expect that L0 regularized regression performs better than LASSO.
