Search Results for author:
Found 33 papers, 15 papers with code
SSAD: Self-supervised Auxiliary Detection Framework for Panoramic X-ray based Dental Disease Diagnosis
To enhance the encoder's ability to capture fine-grained features, we incorporate the image encoder of SAM to construct a texture consistency (TC) loss, which extracts image embedding from the input and output of reconstruction branch, and then enforces both embedding into the same feature space.
Scalable Ensembling For Mitigating Reward Overoptimisation
Reinforcement Learning from Human Feedback (RLHF) has enabled significant advancements within language modeling for powerful, instruction-following models.
Linguistic Calibration of Long-Form Generations
Our results demonstrate that long-form generations may be calibrated end-to-end by constructing an objective in the space of the predictions that users make in downstream decision-making.
SCPMan: Shape Context and Prior Constrained Multi-scale Attention Network for Pancreatic Segmentation
Specifically, we proposed a Multi-scale Feature Extraction Module (MFE) and a Mixed-scale Attention Integration Module (MAI) to address unclear pancreas boundaries.
TCSloT: Text Guided 3D Context and Slope Aware Triple Network for Dental Implant Position Prediction
In this paper, we design a Text Guided 3D Context and Slope Aware Triple Network (TCSloT) which enables the perception of contextual information from multiple adjacent slices and awareness of variation of implant slopes.
TCEIP: Text Condition Embedded Regression Network for Dental Implant Position Prediction
When deep neural network has been proposed to assist the dentist in designing the location of dental implant, most of them are targeting simple cases where only one missing tooth is available.
AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
As a demonstration of the research possible in AlpacaFarm, we find that methods that use a reward model can substantially improve over supervised fine-tuning and that our reference PPO implementation leads to a +10% improvement in win-rate against Davinci003.
Two-Stream Regression Network for Dental Implant Position Prediction
In this paper, we develop a two-stream implant position regression framework (TSIPR), which consists of an implant region detector (IRD) and a multi-scale patch embedding regression network (MSPENet), to address this issue.
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
Recent advances in instruction-following large language models (LLMs) have led to dramatic improvements in a range of NLP tasks.
Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping
To reduce the compute time overhead of private learning, we show that \emph{per-layer clipping}, where the gradient of each neural network layer is clipped separately, allows clipping to be performed in conjunction with backpropagation in differentially private optimization.
Holistic Evaluation of Language Models
We present Holistic Evaluation of Language Models (HELM) to improve the transparency of language models.
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe
Privacy concerns have attracted increasing attention in data-driven products due to the tendency of machine learning models to memorize sensitive training data.
A Closer Look at the Calibration of Differentially Private Learners
Across 7 tasks, temperature scaling and Platt scaling with DP-SGD result in an average 3. 1-fold reduction in the in-domain expected calibration error and only incur at most a minor percent drop in accuracy.
Sample hardness based gradient loss for long-tailed cervical cell detection
Due to the difficulty of cancer samples collection and annotation, cervical cancer datasets usually exhibit a long-tailed data distribution.
When Does Differentially Private Learning Not Suffer in High Dimensions?
Large pretrained models can be privately fine-tuned to achieve performance approaching that of non-private models.
Large Language Models Can Be Strong Differentially Private Learners
Differentially Private (DP) learning has seen limited success for building large deep learning models of text, and straightforward attempts at applying Differentially Private Stochastic Gradient Descent (DP-SGD) to NLP tasks have resulted in large performance drops and high computational overhead.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Learning from Pseudo Lesion: A Self-supervised Framework for COVID-19 Diagnosis
The pairs of normal and pseudo COVID-19 images were then used to train an encoder-decoder architecture based U-Net for image restoration, which does not require any labelled data.
Learning to Extend Program Graphs to Work-in-Progress Code
Source code spends most of its time in a broken or incomplete state during software development.
Efficient and Accurate Gradients for Neural SDEs
This reduces computational cost (giving up to a $1. 87\times$ speedup) and removes the numerical truncation errors associated with gradient penalty.
Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations
We perform scalable approximate inference in continuous-depth Bayesian neural networks.
Neural SDEs as Infinite-Dimensional GANs
Stochastic differential equations (SDEs) are a staple of mathematical modelling of temporal dynamics.
Neural SDEs Made Easy: SDEs are Infinite-Dimensional GANs
Several authors have introduced \emph{Neural Stochastic Differential Equations} (Neural SDEs), often involving complex theory with various limitations.
When Does Preconditioning Help or Hurt Generalization?
While second order optimizers such as natural gradient descent (NGD) often speed up optimization, their effect on generalization has been called into question.
Scalable Gradients for Stochastic Differential Equations
The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations.
Ranked #1 on
Video Prediction
on CMU Mocap-2
Scalable Gradients and Variational Inference for Stochastic Differential Equations
We derive reverse-mode (or adjoint) automatic differentiation for solutions of stochastic differential equations (SDEs), allowing time-efficient and constant-memory computation of pathwise gradients, a continuous-time analogue of the reparameterization trick.
Stochastic Runge-Kutta Accelerates Langevin Monte Carlo and Beyond
In this paper, we establish the convergence rate of sampling algorithms obtained by discretizing smooth It\^o diffusions exhibiting fast Wasserstein-$2$ contraction, based on local deviation properties of the integration scheme.
A Novel Method for the Absolute Pose Problem with Pairwise Constraints
In this paper, we consider pairwise constraints and propose a globally optimal algorithm for solving the absolute pose estimation problem.
Fast and Globally Optimal Rigid Registration of 3D Point Sets by Transformation Decomposition
However, the existing global methods are slow for two main reasons: the computational complexity of BnB is exponential to the problem dimensionality (which is six for 3D rigid registration), and the bound evaluation used in BnB is inefficient.
The idemetric property: when most distances are (almost) the same
We introduce the \emph{idemetric} property, which formalises the idea that most nodes in a graph have similar distances between them, and which turns out to be quite standard amongst small-world network models.
Social and Information Networks Discrete Mathematics
Isolating Sources of Disentanglement in Variational Autoencoders
We decompose the evidence lower bound to show the existence of a term measuring the total correlation between latent variables.
Inference Suboptimality in Variational Autoencoders
Furthermore, we show that the parameters used to increase the expressiveness of the approximation play a role in generalizing inference rather than simply improving the complexity of the approximation.
