Search Results for author:
Found 70 papers, 18 papers with code
paht_nlp @ MEDIQA 2021: Multi-grained Query Focused Multi-Answer Summarization
First a RoBERTa model is first applied to give a local ranking of the candidate sentences.
PCEE-BERT: Accelerating BERT Inference via Patient and Confident Early Exiting
In this work, we propose Patient and Confident Early Exiting BERT (PCEE-BERT), an off-the-shelf sample-dependent early exiting method that can work with different PLMs and can also work along with popular model compression methods.
GAML-BERT: Improving BERT Early Exiting by Gradient Aligned Mutual Learning
From this observation, we use mutual learning to improve BERT’s early exiting performances, that is, we ask each exit of a multi-exit BERT to distill knowledge from each other.
Continually Detection, Rapidly React: Unseen Rumors Detection Based on Continual Prompt-Tuning
Since open social platforms allow for a large and continuous flow of unverified information, rumors can emerge unexpectedly and spread quickly.
ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models
Parameter-efficient fine-tuning (PEFT) is widely studied for its effectiveness and efficiency in the era of large language models.
Text2MDT: Extracting Medical Decision Trees from Medical Texts
In this work, we propose a novel task, Text2MDT, to explore the automatic extraction of MDTs from medical texts such as medical guidelines and textbooks.
Efficient Multi-domain Text Recognition Deep Neural Network Parameterization with Residual Adapters
Recent advancements in deep neural networks have markedly enhanced the performance of computer vision tasks, yet the specialized nature of these networks often necessitates extensive data and high computational power.
Overview of the PromptCBLUE Shared Task in CHIP2023
Many teams from both the industry and academia participated in the shared tasks, and the top teams achieved amazing test results.
Efficient LLM inference solution on Intel GPU
A customized Scaled-Dot-Product-Attention kernel is designed to match our fusion policy based on the segment KV cache solution.
Improving Prompt Tuning with Learned Prompting Layers
Prompt tuning prepends a soft prompt to the input embeddings or hidden states and only optimizes the prompt to adapt pretrained models (PTMs) to downstream tasks.
PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domain
Biomedical language understanding benchmarks are the driving forces for artificial intelligence applications with large language model (LLM) back-ends.
UltraFeedback: Boosting Language Models with High-quality Feedback
However, the scarcity of diverse, naturalistic datasets of human preferences on LLM outputs at scale poses a great challenge to RLHF as well as feedback learning research within the open-source community.
Integrating Different Informations for Portfolio Selection
Following the idea of Bayesian learning via Gaussian mixture model, we organically combine the backward-looking information contained in the historical data and the forward-looking information implied by the market portfolio, which is affected by heterogeneous expectations and noisy trading behavior.
Statistical Guarantees of Group-Invariant GANs
Group-invariant generative adversarial networks (GANs) are a type of GANs in which the generators and discriminators are hardwired with group symmetries.
F-PABEE: Flexible-patience-based Early Exiting for Single-label and Multi-label text Classification Tasks
Computational complexity and overthinking problems have become the bottlenecks for pre-training language models (PLMs) with millions or even trillions of parameters.
Multi-Label Classification
Multi Label Text Classification
+2
Unified Demonstration Retriever for In-Context Learning
To train UDR, we cast various tasks' training signals into a unified list-wise ranking formulation by language model's feedback.
SegPrompt: Using Segmentation Map as a Better Prompt to Finetune Deep Models for Kidney Stone Classification
However, the shortage of annotated training data poses a severe problem in improving the performance and generalization ability of the trained model.
Sample Complexity of Probability Divergences under Group Symmetry
We rigorously quantify the improvement in the sample complexity of variational divergence estimations for group-invariant distributions.
Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translation
In this paper, we propose a simple but effective method called "Candidate Soups," which can obtain high-quality translations while maintaining the inference speed of NAT models.
Improving Micro-video Recommendation by Controlling Position Bias
Although the micro-video recommendation can be naturally treated as the sequential recommendation, the previous sequential recommendation models do not fully consider the characteristics of micro-video apps, and in their inductive biases, the role of positions is not in accord with the reality in the micro-video scenario.
Remote Medication Status Prediction for Individuals with Parkinson's Disease using Time-series Data from Smartphones
Medication for neurological diseases such as the Parkinson's disease usually happens remotely away from hospitals.
Explored An Effective Methodology for Fine-Grained Snake Recognition
Then, in order to take full advantage of unlabeled datasets, we use self-supervised learning and supervised learning joint training to provide pre-trained model.
MAGIC: Microlensing Analysis Guided by Intelligent Computation
The key feature of MAGIC is the introduction of a neural controlled differential equation, which provides the capability to handle light curves with irregular sampling and large data gaps.
Deep Federated Anomaly Detection for Multivariate Time Series Data
Specifically, we first design an Exemplar-based Deep Neural network (ExDNN) to learn local time series representations based on their compatibility with an exemplar module which consists of hidden parameters learned to capture varieties of normal patterns on each edge device.
Localized Adversarial Domain Generalization
Adversarial domain generalization is a popular approach to DG, but conventional approaches (1) struggle to sufficiently align features so that local neighborhoods are mixed across domains; and (2) can suffer from feature space over collapse which can threaten generalization performance.
Continuous Detection, Rapidly React: Unseen Rumors Detection based on Continual Prompt-Tuning
Since open social platforms allow for a large and continuous flow of unverified information, rumors can emerge unexpectedly and spread quickly.
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation
Previous works usually adopt heuristic metrics such as the entropy of internal outputs to measure instance difficulty, which suffers from generalization and threshold-tuning.
Structure-preserving GANs
Generative adversarial networks (GANs), a class of distribution-learning methods based on a two-player game between a generator and a discriminator, can generally be formulated as a minmax problem based on the variational representation of a divergence between the unknown and the generated distributions.
AlphaFold Accelerates Artificial Intelligence Powered Drug Discovery: Efficient Discovery of a Novel Cyclin-dependent Kinase 20 (CDK20) Small Molecule Inhibitor
The AlphaFold computer program predicted protein structures for the whole human genome, which has been considered as a remarkable breakthrough both in artificial intelligence (AI) application and structural biology.
Deformation Robust Roto-Scale-Translation Equivariant CNNs
Incorporating group symmetry directly into the learning process has proved to be an effective guideline for model design.
Route Optimization via Environment-Aware Deep Network and Reinforcement Learning
Vehicle mobility optimization in urban areas is a long-standing problem in smart city and spatial data analysis.
Learning to Aggregate and Refine Noisy Labels for Visual Sentiment Analysis
The learned prototypes and their labels can be regarded as denoising features and labels for the local regions and can guide the training process to prevent the model from overfitting the noisy cases.
Federated Learning of Molecular Properties with Graph Neural Networks in a Heterogeneous Setting
FLIT(+) can align the local training across heterogeneous clients by improving the performance for uncertain samples.
Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization
We propose to remove the bias information misused by the target task with a cross-sample adversarial debiasing (CSAD) method.
MVP-BERT: Multi-Vocab Pre-training for Chinese BERT
Despite the development of pre-trained language models (PLMs) significantly raise the performances of various Chinese natural language processing (NLP) tasks, the vocabulary (vocab) for these Chinese PLMs remains to be the one provided by Google Chinese BERT (CITATION), which is based on Chinese characters (chars).
LeeBERT: Learned Early Exit for BERT with cross-level optimization
In this work, to improve efficiency without performance drop, we propose a novel training scheme called Learned Early Exit for BERT (LeeBERT).
Discovering Better Model Architectures for Medical Query Understanding
In developing an online question-answering system for the medical domains, natural language inference (NLI) models play a central role in question matching and intention detection.
Temperature dependent coherence properties of NV ensemble in diamond up to 600K
Nitrogen-vacancy (NV) center in diamond is an ideal candidate for quantum sensors because of its excellent optical and coherence property.
Quantum Physics Mesoscale and Nanoscale Physics
The 'COVID' Crash of the 2020 U.S. Stock Market
We employed the log-periodic power law singularity (LPPLS) methodology to systematically investigate the 2020 stock market crash in the U. S. equities sectors with different levels of total market capitalizations through four major U. S. stock market indexes, including the Wilshire 5000 Total Market index, the S&P 500 index, the S&P MidCap 400 index, and the Russell 2000 index, representing the stocks overall, the large capitalization stocks, the middle capitalization stocks and the small capitalization stocks, respectively.
Lex-BERT: Enhancing BERT based NER with lexicons
In this work, we represent Lex-BERT, which incorporates the lexicon information into Chinese BERT for named entity recognition (NER) tasks in a natural manner.
The 2020 Global Stock Market Crash: Endogenous or Exogenous?
Starting on February 20, 2020, the global stock markets began to suffer the worst decline since the Great Recession in 2008, and the COVID-19 has been widely blamed on the stock market crashes.
CMV-BERT: Contrastive multi-vocab pretraining of BERT
In this work, we represent CMV-BERT, which improves the pretraining of a language model via two ingredients: (a) contrastive learning, which is well studied in the area of computer vision; (b) multiple vocabularies, one of which is fine-grained and the other is coarse-grained.
MVP-BERT: Redesigning Vocabularies for Chinese BERT and Multi-Vocab Pretraining
Despite the development of pre-trained language models (PLMs) significantly raise the performances of various Chinese natural language processing (NLP) tasks, the vocabulary for these Chinese PLMs remain to be the one provided by Google Chinese Bert \cite{devlin2018bert}, which is based on Chinese characters.
Precision-Recall Curve (PRC) Classification Trees
Our algorithm, named as the "Precision-Recall Curve classification tree", or simply the "PRC classification tree" modifies two crucial stages in tree building.
Predicting Parkinson's Disease with Multimodal Irregularly Collected Longitudinal Smartphone Data
Parkinsons Disease is a neurological disorder and prevalent in elderly people.
AutoRC: Improving BERT Based Relation Classification Models via Architecture Search
Ablation study demonstrates the necessity of our search space design and the effectiveness of our search method.
AutoTrans: Automating Transformer Design via Reinforced Architecture Search
Though the transformer architectures have shown dominance in many natural language understanding tasks, there are still unsolved issues for the training of transformer models, especially the need for a principled way of warm-up which has shown importance for stable training of a transformer, as well as whether the task at hand prefer to scale the attention product or not.
Personalized Fashion Recommendation from Personal Social Media Data: An Item-to-Set Metric Learning Approach
In this work, we study the problem of personalized fashion recommendation from social media data, i. e. recommending new outfits to social media users that fit their fashion preferences.
Alleviating the Incompatibility between Cross Entropy Loss and Episode Training for Few-shot Skin Disease Classification
Inspired by the recent success of Few-Shot Learning (FSL) in natural image classification, we propose to apply FSL to skin disease identification to address the extreme scarcity of training sample problem.
Weighted Aggregating Stochastic Gradient Descent for Parallel Deep Learning
Following a theoretical analysis on the characteristics of the new objective function, WASGD introduces a decentralized weighted aggregating scheme based on the performance of local workers.
Unifying Specialist Image Embedding into Universal Image Embedding
Therefore, we propose to distill the knowledge in multiple specialists into a universal embedding to solve this problem.
Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks
We have experimented both (a) improving the fine-tuning of pre-trained language models on a task with a small dataset size, by leveraging datasets of similar tasks; and (b) incorporating the distributional representations of a KG onto the representations of pre-trained language models, via simply concatenation or multi-head attention.
Ranked #17 on
Common Sense Reasoning
on ReCoRD
Scale-Equivariant Neural Networks with Decomposed Convolutional Filters
Encoding the input scale information explicitly into the representation learned by a convolutional neural network (CNN) is beneficial for many vision tasks especially when dealing with multiscale input signals.
Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters
Encoding the scale information explicitly into the representation learned by a convolutional neural network (CNN) is beneficial for many computer vision tasks especially when dealing with multiscale inputs.
PANLP at MEDIQA 2019: Pre-trained Language Models, Transfer Learning and Knowledge Distillation
Transfer learning from the NLI task to the RQE task is also experimented, which proves to be useful in improving the results of fine-tuning MT-DNN large.
Extension of Rough Set Based on Positive Transitive Relation
The new model holds the merit of the existing rough set extension models while avoids their limitations of discarding transitivity or symmetry.
Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization
We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation.
Circular dichroism in high-order harmonic generation: Heralding topological phases and transitions in Chern insulators
Topological materials are of interest to both fundamental science and advanced technologies, because topological states are robust with respect to perturbations and dissipation.
Mesoscale and Nanoscale Physics Quantum Physics
Stop memorizing: A data-dependent regularization framework for intrinsic pattern learning
Deep neural networks (DNNs) typically have enough capacity to fit random data by brute force even when conventional data-dependent regularizations focusing on the geometry of the features are imposed.
Quantum Algorithm Implementations for Beginners
As quantum computers become available to the general public, the need has arisen to train a cohort of quantum programmers, many of whom have been developing classical computer programs for most of their careers.
Emerging Technologies Quantum Physics
Deep Neural Nets with Interpolating Function as Output Activation
We replace the output layer of deep neural nets, typically the softmax function, by a novel interpolating function.
Multi-appearance Segmentation and Extended 0-1 Program for Dense Small Object Tracking
Aiming to address the fast multi-object tracking for dense small object in the cluster background, we review track orientated multi-hypothesis tracking(TOMHT) with consideration of batch optimization.
LDMNet: Low Dimensional Manifold Regularized Neural Networks
To resolve this, we propose a new framework, the Low-Dimensional-Manifold-regularized neural Network (LDMNet), which incorporates a feature regularization method that focuses on the geometry of both the input data and the output features.
A convergence framework for inexact nonconvex and nonsmooth algorithms and its applications to several iterations
In fact, a lot of classical inexact nonconvex and nonsmooth algorithms allow these three conditions.
Iteratively Linearized Reweighted Alternating Direction Method of Multipliers for a Class of Nonconvex Problems
The traditional alternating direction method of multipliers encounters troubles in both mathematics and computations in solving the nonconvex and nonsmooth subproblem.
Exploiting Color Name Space for Salient Object Detection
In this paper, we will investigate the contribution of color names for the task of salient object detection.
Scalable low dimensional manifold model in the reconstruction of noisy and incomplete hyperspectral images
We present a scalable low dimensional manifold model for the reconstruction of noisy and incomplete hyperspectral images.
Unsupervised Classification in Hyperspectral Imagery with Nonlocal Total Variation and Primal-Dual Hybrid Gradient Algorithm
In this paper, a graph-based nonlocal total variation method (NLTV) is proposed for unsupervised classification of hyperspectral images (HSI).
RCR: Robust Compound Regression for Robust Estimation of Errors-in-Variables Model
The errors-in-variables (EIV) regression model, being more realistic by accounting for measurement errors in both the dependent and the independent variables, is widely adopted in applied sciences.
