Search Results for author: Pengtao Xie

Found 92 papers, 29 papers with code

Integrating Document Clustering and Topic Modeling

no code implementations26 Sep 2013 Pengtao Xie, Eric P. Xing

Document clustering and topic modeling are two closely related tasks which can mutually benefit each other.

Clustering Topic Models +1

Petuum: A New Platform for Distributed Machine Learning on Big Data

no code implementations30 Dec 2013 Eric P. Xing, Qirong Ho, Wei Dai, Jin Kyu Kim, Jinliang Wei, Seunghak Lee, Xun Zheng, Pengtao Xie, Abhimanu Kumar, Yao-Liang Yu

What is a systematic way to efficiently apply a wide spectrum of advanced ML programs to industrial scale problems, using Big Models (up to 100s of billions of parameters) on Big Data (up to terabytes or petabytes)?

BIG-bench Machine Learning Scheduling

Distributed Machine Learning via Sufficient Factor Broadcasting

no code implementations19 Sep 2014 Pengtao Xie, Jin Kyu Kim, Yi Zhou, Qirong Ho, Abhimanu Kumar, Yao-Liang Yu, Eric Xing

Matrix-parametrized models, including multiclass logistic regression and sparse coding, are used in machine learning (ML) applications ranging from computer vision to computational biology.

BIG-bench Machine Learning

Crypto-Nets: Neural Networks over Encrypted Data

1 code implementation18 Dec 2014 Pengtao Xie, Misha Bilenko, Tom Finley, Ran Gilad-Bachrach, Kristin Lauter, Michael Naehrig

To achieve the privacy requirements, we use homomorphic encryption in the following protocol: the data owner encrypts the data and sends the ciphertexts to the third party to obtain a prediction from a trained model.

Large Scale Distributed Distance Metric Learning

no code implementations18 Dec 2014 Pengtao Xie, Eric Xing

In large scale machine learning and data mining problems with high feature dimensionality, the Euclidean distance between data points can be uninformative, and Distance Metric Learning (DML) is often desired to learn a proper similarity measure (using side information such as example data pairs being similar or dissimilar).

Metric Learning

Cauchy Principal Component Analysis

no code implementations19 Dec 2014 Pengtao Xie, Eric Xing

In this paper, we propose Cauchy Principal Component Analysis (Cauchy PCA), a very simple yet effective PCA method which is robust to various types of noise.

On the Generalization Error Bounds of Neural Networks under Diversity-Inducing Mutual Angular Regularization

no code implementations23 Nov 2015 Pengtao Xie, Yuntian Deng, Eric Xing

Recently diversity-inducing regularization methods for latent variable models (LVMs), which encourage the components in LVMs to be diverse, have been studied to address several issues involved in latent variable modeling: (1) how to capture long-tail patterns underlying data; (2) how to reduce model complexity without sacrificing expressivity; (3) how to improve the interpretability of learned patterns.

Distributed Machine Learning via Sufficient Factor Broadcasting

no code implementations26 Nov 2015 Pengtao Xie, Jin Kyu Kim, Yi Zhou, Qirong Ho, Abhimanu Kumar, Yao-Liang Yu, Eric Xing

Matrix-parametrized models, including multiclass logistic regression and sparse coding, are used in machine learning (ML) applications ranging from computer vision to computational biology.

BIG-bench Machine Learning

Distributed Training of Deep Neural Networks with Theoretical Analysis: Under SSP Setting

no code implementations9 Dec 2015 Abhimanu Kumar, Pengtao Xie, Junming Yin, Eric P. Xing

We propose a distributed approach to train deep neural networks (DNNs), which has guaranteed convergence theoretically and great scalability empirically: close to 6 times faster on instance of ImageNet data set when run with 6 machines.

General Classification Image Classification

Poseidon: A System Architecture for Efficient GPU-based Deep Learning on Multiple Machines

no code implementations19 Dec 2015 Hao Zhang, Zhiting Hu, Jinliang Wei, Pengtao Xie, Gunhee Kim, Qirong Ho, Eric Xing

To investigate how to adapt existing frameworks to efficiently support distributed GPUs, we propose Poseidon, a scalable system architecture for distributed inter-machine communication in existing DL frameworks.

Object Recognition

Latent Variable Modeling with Diversity-Inducing Mutual Angular Regularization

no code implementations23 Dec 2015 Pengtao Xie, Yuntian Deng, Eric Xing

On two popular latent variable models --- restricted Boltzmann machine and distance metric learning, we demonstrate that MAR can effectively capture long-tail patterns, reduce model complexity without sacrificing expressivity and improve interpretability.

Metric Learning

Strategies and Principles of Distributed Machine Learning on Big Data

no code implementations31 Dec 2015 Eric P. Xing, Qirong Ho, Pengtao Xie, Wei Dai

Taking the view that Big ML systems can benefit greatly from ML-rooted statistical and algorithmic insights --- and that ML researchers should therefore not shy away from such systems design --- we discuss a series of principles and strategies distilled from our recent efforts on industrial-scale ML solutions.

BIG-bench Machine Learning

Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters

no code implementations11 Jun 2017 Hao Zhang, Zeyu Zheng, Shizhen Xu, Wei Dai, Qirong Ho, Xiaodan Liang, Zhiting Hu, Jinliang Wei, Pengtao Xie, Eric P. Xing

We show that Poseidon enables Caffe and TensorFlow to achieve 15. 5x speed-up on 16 single-GPU machines, even with limited bandwidth (10GbE) and the challenging VGG19-22K network for image classification.

Image Classification

Learning Latent Space Models with Angular Constraints

no code implementations ICML 2017 Pengtao Xie, Yuntian Deng, Yi Zhou, Abhimanu Kumar, Yao-Liang Yu, James Zou, Eric P. Xing

The large model capacity of latent space models (LSMs) enables them to achieve great performance on various applications, but meanwhile renders LSMs to be prone to overfitting.

Uncorrelation and Evenness: a New Diversity-Promoting Regularizer

no code implementations ICML 2017 Pengtao Xie, Aarti Singh, Eric P. Xing

Latent space models (LSMs) provide a principled and effective way to extract hidden patterns from observed data.

Predicting Discharge Medications at Admission Time Based on Deep Learning

no code implementations4 Nov 2017 Yuan Yang, Pengtao Xie, Xin Gao, Carol Cheng, Christy Li, Hongbao Zhang, Eric Xing

Predicting discharge medications right after a patient being admitted is an important clinical decision, which provides physicians with guidance on what type of medication regimen to plan for and what possible changes on initial medication may occur during an inpatient stay.

Towards Automated ICD Coding Using Deep Learning

no code implementations11 Nov 2017 Haoran Shi, Pengtao Xie, Zhiting Hu, Ming Zhang, Eric P. Xing

Considering the complicated and dedicated process to assign correct codes to each patient admission based on overall diagnosis, we propose a hierarchical deep learning model with attention mechanism which can automatically assign ICD diagnostic codes given written diagnosis.

General Classification Management

Medical Diagnosis From Laboratory Tests by Combining Generative and Discriminative Learning

no code implementations12 Nov 2017 Shiyue Zhang, Pengtao Xie, Dong Wang, Eric P. Xing

In hospital, physicians rely on massive clinical data to make diagnosis decisions, among which laboratory tests are one of the most important resources.

Decision Making Imputation +1

Effective Use of Bidirectional Language Modeling for Transfer Learning in Biomedical Named Entity Recognition

2 code implementations21 Nov 2017 Devendra Singh Sachan, Pengtao Xie, Mrinmaya Sachan, Eric P. Xing

We also show that BiLM weight transfer leads to a faster model training and the pretrained model requires fewer training examples to achieve a particular F1 score.

Language Modelling named-entity-recognition +3

On the Automatic Generation of Medical Imaging Reports

4 code implementations ACL 2018 Baoyu Jing, Pengtao Xie, Eric Xing

To cope with these challenges, we (1) build a multi-task learning framework which jointly performs the pre- diction of tags and the generation of para- graphs, (2) propose a co-attention mechanism to localize regions containing abnormalities and generate narrations for them, (3) develop a hierarchical LSTM model to generate long paragraphs.

Medical Report Generation Multi-Task Learning

Diversity-Promoting Bayesian Learning of Latent Variable Models

no code implementations23 Nov 2017 Pengtao Xie, Jun Zhu, Eric P. Xing

We also extend our approach to "diversify" Bayesian nonparametric models where the number of components is infinite.

Variational Inference

Learning Less-Overlapping Representations

no code implementations ICLR 2018 Pengtao Xie, Hongbao Zhang, Eric P. Xing

In representation learning (RL), how to make the learned representations easy to interpret and less overfitted to training data are two important but challenging issues.

Representation Learning

Stacked Kernel Network

no code implementations25 Nov 2017 Shuai Zhang, Jian-Xin Li, Pengtao Xie, Yingchun Zhang, Minglai Shao, Haoyi Zhou, Mengyi Yan

Similar to DNNs, a SKN is composed of multiple layers of hidden units, but each parameterized by a RKHS function rather than a finite-dimensional vector.

Orthogonality-Promoting Distance Metric Learning: Convex Relaxation and Theoretical Analysis

no code implementations ICML 2018 Pengtao Xie, Wei Wu, Yichen Zhu, Eric P. Xing

In this paper, we address these three issues by (1) seeking convex relaxations of the original nonconvex problems so that the global optimal is guaranteed to be achievable; (2) providing a formal analysis on OPR's capability of promoting balancedness; (3) providing a theoretical analysis that directly reveals the relationship between OPR and generalization performance.

Metric Learning

A Neural Architecture for Automated ICD Coding

no code implementations ACL 2018 Pengtao Xie, Eric Xing

The International Classification of Diseases (ICD) provides a hierarchy of diagnostic codes for classifying diseases.

Nonoverlap-Promoting Variable Selection

no code implementations ICML 2018 Pengtao Xie, Hongbao Zhang, Yichen Zhu, Eric Xing

Variable selection is a classic problem in machine learning (ML), widely used to find important explanatory factors, and improve generalization performance and interpretability of ML models.

Variable Selection

Missing Value Imputation Based on Deep Generative Models

no code implementations5 Aug 2018 Hongbao Zhang, Pengtao Xie, Eric Xing

In this paper, we propose a probabilistic framework based on deep generative models for MVI.

Imputation

Unsupervised Pseudo-Labeling for Extractive Summarization on Electronic Health Records

no code implementations20 Nov 2018 Xiangan Liu, Keyang Xu, Pengtao Xie, Eric Xing

Extractive summarization is very useful for physicians to better manage and digest Electronic Health Records (EHRs).

Extractive Summarization

Explaining a black-box using Deep Variational Information Bottleneck Approach

3 code implementations19 Feb 2019 Seojin Bang, Pengtao Xie, Heewook Lee, Wei Wu, Eric Xing

Briefness and comprehensiveness are necessary in order to provide a large amount of information concisely when explaining a black-box decision system.

BIG-bench Machine Learning Interpretable Machine Learning

Adversarial Domain Adaptation Being Aware of Class Relationships

no code implementations28 May 2019 Zeya Wang, Baoyu Jing, Yang Ni, Nanqing Dong, Pengtao Xie, Eric P. Xing

In this paper, we propose a novel relationship-aware adversarial domain adaptation (RADA) algorithm, which first utilizes a single multi-class domain discriminator to enforce the learning of inter-class dependency structure during domain-adversarial training and then aligns this structure with the inter-class dependencies that are characterized from training the label predictor on source domain.

Domain Adaptation Transfer Learning

Explaining A Black-box By Using A Deep Variational Information Bottleneck Approach

no code implementations25 Sep 2019 Seojin Bang, Pengtao Xie, Heewook Lee, Wei Wu, Eric Xing

Briefness and comprehensiveness are necessary in order to provide a large amount of information concisely when explaining a black-box decision system.

BIG-bench Machine Learning Interpretable Machine Learning

Generalized Zero-shot ICD Coding

no code implementations28 Sep 2019 Congzheng Song, Shanghang Zhang, Najmeh Sadoughi, Pengtao Xie, Eric Xing

The International Classification of Diseases (ICD) is a list of classification codes for the diagnoses.

General Classification Generalized Zero-Shot Learning +3

PathVQA: 30000+ Questions for Medical Visual Question Answering

6 code implementations7 Mar 2020 Xuehai He, Yichen Zhang, Luntian Mou, Eric Xing, Pengtao Xie

To achieve this goal, the first step is to create a visual question answering (VQA) dataset where the AI agent is presented with a pathology image together with a question and is asked to give the correct answer.

Medical Visual Question Answering Question Answering +1

COVID-CT-Dataset: A CT Scan Dataset about COVID-19

19 code implementations30 Mar 2020 Xingyi Yang, Xuehai He, Jinyu Zhao, Yichen Zhang, Shanghang Zhang, Pengtao Xie

Using this dataset, we develop diagnosis methods based on multi-task learning and self-supervised learning, that achieve an F1 of 0. 90, an AUC of 0. 98, and an accuracy of 0. 89.

Computed Tomography (CT) COVID-19 Diagnosis +2

Identifying Radiological Findings Related to COVID-19 from Medical Literature

no code implementations4 Apr 2020 Yuxiao Liang, Pengtao Xie

Coronavirus disease 2019 (COVID-19) has infected more than one million individuals all over the world and caused more than 55, 000 deaths, as of April 3 in 2020.

MedDialog: Two Large-scale Medical Dialogue Datasets

1 code implementation arXiv 2020 Xuehai He, Shu Chen, Zeqian Ju, Xiangyu Dong, Hongchao Fang, Sicheng Wang, Yue Yang, Jiaqi Zeng, Ruisi Zhang, Ruoyu Zhang, Meng Zhou, Penghui Zhu, Pengtao Xie

Medical dialogue systems are promising in assisting in telemedicine to increase access to healthcare services, improve the quality of patient care, and reduce medical costs.

Vocal Bursts Valence Prediction

CERT: Contrastive Self-supervised Learning for Language Understanding

1 code implementation16 May 2020 Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, Pengtao Xie

We evaluate CERT on 11 natural language understanding tasks in the GLUE benchmark where CERT outperforms BERT on 7 tasks, achieves the same performance as BERT on 2 tasks, and performs worse than BERT on 2 tasks.

Natural Language Understanding Self-Supervised Learning +2

Differentially-private Federated Neural Architecture Search

1 code implementation16 Jun 2020 Ishika Singh, Haoyi Zhou, Kunlin Yang, Meng Ding, Bill Lin, Pengtao Xie

To address this problem, we propose federated neural architecture search (FNAS), where different parties collectively search for a differentiable architecture by exchanging gradients of architecture variables without exposing their data to other parties.

Neural Architecture Search

XRayGAN: Consistency-preserving Generation of X-ray Images from Radiology Reports

1 code implementation17 Jun 2020 Xingyi Yang, Nandiraju Gireesh, Eric Xing, Pengtao Xie

To address this problem, we develop methods to generate view-consistent, high-fidelity, and high-resolution X-ray images from radiology reports to facilitate radiology training of medical students.

Transfer Learning or Self-supervised Learning? A Tale of Two Pretraining Paradigms

1 code implementation19 Jun 2020 Xingyi Yang, Xuehai He, Yuxiao Liang, Yue Yang, Shanghang Zhang, Pengtao Xie

There has not been a clear understanding on what properties of data and tasks render one approach outperforms the other.

Self-Supervised Learning Transfer Learning

Contrastive Self-supervised Learning for Graph Classification

no code implementations13 Sep 2020 Jiaqi Zeng, Pengtao Xie

A contrastive loss is defined to learn graph encoders by judging whether two augmented graphs are from the same original graph.

Data Augmentation General Classification +2

TreeGAN: Incorporating Class Hierarchy into Image Generation

no code implementations16 Sep 2020 Ruisi Zhang, Luntian Mou, Pengtao Xie

Based on these two ideas, we propose a TreeGAN model which consists of three modules: (1) a class hierarchy encoder (CHE) which takes the hierarchical structure of classes and their textual names as inputs and learns an embedding for each class; the embedding captures the hierarchical relationship among classes; (2) a conditional image generator (CIG) which takes the CHE-generated embedding of a class as input and generates a set of images belonging to this class; (3) a consistency checker which performs hierarchical classification on the generated images and checks whether the generated images are compatible with the class hierarchy; the consistency score is used to guide the CIG to generate hierarchy-compatible images.

Conditional Image Generation

Pathological Visual Question Answering

no code implementations6 Oct 2020 Xuehai He, Zhuo Cai, Wenlan Wei, Yichen Zhang, Luntian Mou, Eric Xing, Pengtao Xie

To deal with the issue that a publicly available pathology VQA dataset is lacking, we create PathVQA dataset.

Question Answering Self-Supervised Learning +1

Discriminative Cross-Modal Data Augmentation for Medical Imaging Applications

no code implementations7 Oct 2020 Yue Yang, Pengtao Xie

While deep learning methods have shown great success in medical image analysis, they require a number of medical images to train.

Data Augmentation Image-to-Image Translation +1

Validate and Enable Machine Learning in Industrial AI

no code implementations30 Oct 2020 Hongbo Zou, Guangjing Chen, Pengtao Xie, Sean Chen, Yongtian He, Hochih Huang, Zheng Nie, Hongbao Zhang, Tristan Bala, Kazi Tulip, Yuqi Wang, Shenlin Qin, Eric P. Xing

However, manufacturers and solution partners need to understand how to implement and integrate an AI model into the existing industrial control system.

BIG-bench Machine Learning

Learning by Passing Tests, with Application to Neural Architecture Search

no code implementations30 Nov 2020 Xuefeng Du, Haochen Zhang, Pengtao Xie

We propose a multi-level optimization framework to formulate LPT, where the tester learns to create difficult and meaningful tests and the learner learns to pass these tests.

Neural Architecture Search

Skillearn: Machine Learning Inspired by Humans' Learning Skills

no code implementations9 Dec 2020 Pengtao Xie, Xuefeng Du, Hao Ban

To achieve this goal, we develop a general framework -- Skillearn, which provides a principled way to represent humans' learning skills mathematically and use the formally-represented skills to improve the training of ML models.

BIG-bench Machine Learning Neural Architecture Search

DSRNA: Differentiable Search of Robust Neural Architectures

no code implementations CVPR 2021 Ramtin Hosseini, Xingyi Yang, Pengtao Xie

To address this problem, we propose methods to perform differentiable search of robust neural architectures.

Small-Group Learning, with Application to Neural Architecture Search

1 code implementation23 Dec 2020 Xuefeng Du, Pengtao Xie

SGL is formulated as a multi-level optimization framework consisting of three learning stages: each learner trains a model independently and uses this model to perform pseudo-labeling; each learner trains another model using datasets pseudo-labeled by other learners; learners improve their architectures by minimizing validation losses.

Neural Architecture Search

Learning by Self-Explanation, with Application to Neural Architecture Search

no code implementations23 Dec 2020 Ramtin Hosseini, Pengtao Xie

In our approach, an explainer model improves its learning ability by trying to clearly explain to an audience model regarding how a prediction outcome is made.

BIG-bench Machine Learning Neural Architecture Search

Self-supervised Regularization for Text Classification

1 code implementation9 Mar 2021 Meng Zhou, Zechen Li, Pengtao Xie

The SSL task is unsupervised, which is defined purely on input texts without using any human-provided labels.

General Classification Self-Supervised Learning +2

Learning by Teaching, with Application to Neural Architecture Search

no code implementations11 Mar 2021 Parth Sheth, Yueyu Jiang, Pengtao Xie

In the LBT framework, a teacher model improves itself by teaching a student model to learn well.

Neural Architecture Search

Interleaving Learning, with Application to Neural Architecture Search

no code implementations12 Mar 2021 Hao Ban, Pengtao Xie

Inspired by the interleaving learning technique of humans, in this paper we explore whether this learning methodology is beneficial for improving the performance of machine learning models as well.

BIG-bench Machine Learning Image Classification +1

Towards Visual Question Answering on Pathology Images

1 code implementation ACL 2021 Xuehai He, Zhuo Cai, Wenlan Wei, Yichen Zhang, Luntian Mou, Eric Xing, Pengtao Xie

In this paper, we aim to develop a pathological visual question answering framework to analyze pathology images and answer medical questions related to these images.

Decision Making Question Answering +1

Learning by Examples Based on Multi-level Optimization

no code implementations22 Sep 2021 Shentong Mo, Pengtao Xie

Learning by examples, which learns to solve a new problem by looking into how similar problems are solved, is an effective learning method in human learning.

Few-Shot Learning

EViT: Expediting Vision Transformers via Token Reorganizations

1 code implementation ICLR 2022 Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, Pengtao Xie

Second, by maintaining the same computational cost, our method empowers ViTs to take more image tokens as input for recognition accuracy improvement, where the image tokens are from higher resolution images.

Learning from Mistakes -- A Framework for Neural Architecture Search

1 code implementation11 Nov 2021 Bhanu Garg, Li Zhang, Pradyumna Sridhara, Ramtin Hosseini, Eric Xing, Pengtao Xie

We propose a novel machine learning method called Learning From Mistakes (LFM), wherein the learner improves its ability to learn by focusing more on the mistakes during revision.

BIG-bench Machine Learning Neural Architecture Search

Learning from Mistakes based on Class Weighting with Application to Neural Architecture Search

no code implementations1 Dec 2021 Jay Gala, Pengtao Xie

In this work, we aim to investigate how effectively we can leverage this exceptional learning ability to improve machine learning models.

BIG-bench Machine Learning Image Classification +1

Performance-Aware Mutual Knowledge Distillation for Improving Neural Architecture Search

no code implementations CVPR 2022 Pengtao Xie, Xuefeng Du

In existing MKD methods, mutual knowledge distillation is performed between models without scrutiny: a worse-performing model is allowed to generate knowledge to train a better-performing model, which may lead to collective failures.

Knowledge Distillation Neural Architecture Search

Self-directed Machine Learning

no code implementations4 Jan 2022 Wenwu Zhu, Xin Wang, Pengtao Xie

Inspired by the concept of self-directed human learning, we introduce the principal concept of Self-directed Machine Learning (SDML) and propose a framework for SDML.

BIG-bench Machine Learning Model Selection

Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations

1 code implementation16 Feb 2022 Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, Pengtao Xie

Second, by maintaining the same computational cost, our method empowers ViTs to take more image tokens as input for recognition accuracy improvement, where the image tokens are from higher resolution images.

Betty: An Automatic Differentiation Library for Multilevel Optimization

1 code implementation5 Jul 2022 Sang Keun Choe, Willie Neiswanger, Pengtao Xie, Eric Xing

Gradient-based multilevel optimization (MLO) has gained attention as a framework for studying numerous problems, ranging from hyperparameter optimization and meta-learning to neural architecture search and reinforcement learning.

Hyperparameter Optimization Meta-Learning +1

Type Information Utilized Event Detection via Multi-Channel GNNs in Electrical Power Systems

no code implementations15 Nov 2022 Qian Li, JianXin Li, Lihong Wang, Cheng Ji, Yiming Hei, Jiawei Sheng, Qingyun Sun, Shan Xue, Pengtao Xie

To address the above issues, we propose a Multi-Channel graph neural network utilizing Type information for Event Detection in power systems, named MC-TED, leveraging a semantic channel and a topological channel to enrich information interaction from short texts.

Event Detection Semantic Similarity +2

Joint Self-Supervised Image-Volume Representation Learning with Intra-Inter Contrastive Clustering

no code implementations4 Dec 2022 Duy M. H. Nguyen, Hoang Nguyen, Mai T. N. Truong, Tri Cao, Binh T. Nguyen, Nhat Ho, Paul Swoboda, Shadi Albarqouni, Pengtao Xie, Daniel Sonntag

Recent breakthroughs in self-supervised learning (SSL) offer the ability to overcome the lack of labeled training samples by learning feature representations from unlabeled data.

Brain Segmentation Clustering +3

A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT

no code implementations18 Feb 2023 Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, Hao Peng, JianXin Li, Jia Wu, Ziwei Liu, Pengtao Xie, Caiming Xiong, Jian Pei, Philip S. Yu, Lichao Sun

This study provides a comprehensive review of recent research advancements, challenges, and opportunities for PFMs in text, image, graph, as well as other data modalities.

Graph Learning Language Modelling +1

Learning by Grouping: A Multilevel Optimization Framework for Improving Fairness in Classification without Losing Accuracy

no code implementations2 Apr 2023 Ramtin Hosseini, Li Zhang, Bhanu Garg, Pengtao Xie

Our proposed framework involves three stages of learning, which are formulated as a three-level optimization problem: (i) learning to group problems into different subgroups; (ii) learning group-specific sub-models for problem-solving; and (iii) updating group assignments of training examples by minimizing the validation loss.

Decision Making Domain Adaptation +2

DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphs

1 code implementation18 May 2023 Youwei Liang, Ruiyi Zhang, Li Zhang, Pengtao Xie

The DrugChat system consists of a graph neural network (GNN), a large language model (LLM), and an adaptor.

Drug Discovery Language Modelling +1

LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching

1 code implementation NeurIPS 2023 Duy M. H. Nguyen, Hoang Nguyen, Nghiem T. Diep, Tan N. Pham, Tri Cao, Binh T. Nguyen, Paul Swoboda, Nhat Ho, Shadi Albarqouni, Pengtao Xie, Daniel Sonntag, Mathias Niepert

While pre-trained deep networks on ImageNet and vision-language foundation models trained on web-scale data are prevailing approaches, their effectiveness on medical tasks is limited due to the significant domain shift between natural and medical images.

Contrastive Learning Diabetic Retinopathy Grading +3

BLO-SAM: Bi-level Optimization Based Overfitting-Preventing Finetuning of SAM

no code implementations26 Feb 2024 Li Zhang, Youwei Liang, Ruiyi Zhang, Amirhosein Javadi, Pengtao Xie

Secondly, SAM faces challenges in excelling at specific downstream tasks, like medical imaging, due to a disparity between the distribution of its pretraining data, which predominantly consists of general-domain images, and the data used in downstream tasks.

Image Segmentation Segmentation +1

Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models

1 code implementation28 Feb 2024 Mingjia Huo, Sai Ashish Somayajula, Youwei Liang, Ruisi Zhang, Farinaz Koushanfar, Pengtao Xie

Large language models generate high-quality responses with potential misinformation, underscoring the need for regulation by distinguishing AI-generated and human-written texts.

Misinformation

BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models

no code implementations19 Mar 2024 Rushi Qiang, Ruiyi Zhang, Pengtao Xie

Low-rank adaptation (LoRA) is a popular method for fine-tuning large-scale pre-trained models in downstream tasks by learning low-rank incremental matrices.

Natural Language Understanding

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts

1 code implementation19 Mar 2024 Sai Ashish Somayajula, Youwei Liang, Abhishek Singh, Li Zhang, Pengtao Xie

Pretrained Language Models (PLMs) have advanced Natural Language Processing (NLP) tasks significantly, but finetuning PLMs on low-resource datasets poses significant challenges such as instability and overfitting.

MetaWeighting: Learning to Weight Tasks in Multi-Task Learning

no code implementations Findings (ACL) 2022 YUREN MAO, Zekai Wang, Weiwei Liu, Xuemin Lin, Pengtao Xie

Task weighting, which assigns weights on the including tasks during training, significantly matters the performance of Multi-task Learning (MTL); thus, recently, there has been an explosive interest in it.

Multi-Task Learning text-classification +1

Cannot find the paper you are looking for? You can Submit a new open access paper.