Search Results for author: Fei Tan

Found 20 papers, 5 papers with code

TNT: Text Normalization based Pre-training of Transformers for Content Moderation

no code implementations EMNLP 2020 Fei Tan, Yifan Hu, Changwei Hu, Keqian Li, Kevin Yen

In this work, we present a new language pre-training model TNT (Text Normalization based pre-training of Transformers) for content moderation.

SynthDoc: Bilingual Documents Synthesis for Visual Document Understanding

no code implementations27 Aug 2024 Chuanghao Ding, Xuejing Liu, Wei Tang, Juan Li, Xiaoliang Wang, Rui Zhao, Cam-Tu Nguyen, Fei Tan

This paper introduces SynthDoc, a novel synthetic document generation pipeline designed to enhance Visual Document Understanding (VDU) by generating high-quality, diverse datasets that include text, images, tables, and charts.

document understanding

Reward Difference Optimization For Sample Reweighting In Offline RLHF

no code implementations18 Aug 2024 Shiqi Wang, Zhengze Zhang, Rui Zhao, Fei Tan, Cam Tu Nguyen

Experiments with 7B LLMs on the HH and TL;DR datasets substantiate the effectiveness of our method in both automatic metrics and human evaluation, thereby highlighting its potential for aligning LLMs with human intent and values

SimCT: A Simple Consistency Test Protocol in LLMs Development Lifecycle

no code implementations24 Jul 2024 Fufangchen Zhao, Guoqiang Jin, Rui Zhao, Jiangheng Huang, Fei Tan

SimCT is mainly to proactively check the consistency across different development stages of "bare metal" LLMs or associated services without accessing the model artifacts, in an attempt to expedite the delivery by reducing the back-and-forth alignment communications among multiple teams involved in different development stages.

Language Modelling Large Language Model

CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models

no code implementations24 Jul 2024 Jiawei Gu, Zacc Yang, Chuanghao Ding, Rui Zhao, Fei Tan

We formalize the trade-off between general and domain-specific capabilities, leading to a well-defined Critical Mixture Ratio (CMR) of general and domain data.

What Makes Good Few-shot Examples for Vision-Language Models?

no code implementations22 May 2024 Zhaojun Guo, Jinghui Lu, Xuejing Liu, Rui Zhao, Zhenxing Qian, Fei Tan

Despite the notable advancements achieved by leveraging pre-trained vision-language (VL) models through few-shot tuning for downstream tasks, our detailed empirical study highlights a significant dependence of few-shot learning outcomes on the careful selection of training examples - a facet that has been previously overlooked in research.

Active Learning Few-Shot Learning

Balancing Speciality and Versatility: a Coarse to Fine Framework for Supervised Fine-tuning Large Language Model

1 code implementation16 Apr 2024 Hengyuan Zhang, Yanru Wu, Dawei Li, Sak Yang, Rui Zhao, Yong Jiang, Fei Tan

In an overall evaluation of both speciality and versatility, CoFiTune consistently outperforms baseline methods across diverse tasks and model scales.

Language Modelling Large Language Model

Consistency Matters: Explore LLMs Consistency From a Black-Box Perspective

no code implementations27 Feb 2024 Fufangchen Zhao, Guoqiang Jin, Jiaheng Huang, Rui Zhao, Fei Tan

The solution to this problem is often time-consuming and labor-intensive, and there is also an additional cost of secondary deployment, resulting in economic and time losses.

What Large Language Models Bring to Text-rich VQA?

no code implementations13 Nov 2023 Xuejing Liu, Wei Tang, Xinzhe Ni, Jinghui Lu, Rui Zhao, Zechao Li, Fei Tan

This pipeline achieved superior performance compared to the majority of existing Multimodal Large Language Models (MLLM) on four text-rich VQA datasets.

Image Comprehension Optical Character Recognition (OCR) +2

Deeply Coupled Cross-Modal Prompt Learning

1 code implementation29 May 2023 Xuejing Liu, Wei Tang, Jinghui Lu, Rui Zhao, Zhaojun Guo, Fei Tan

Recent advancements in multimodal foundation models (e. g., CLIP) have excelled in zero-shot generalization.

Domain Adaptation Few-Shot Learning +3

High-fidelity Direct Contrast Synthesis from Magnetic Resonance Fingerprinting

no code implementations21 Dec 2022 Ke Wang, Mariya Doneva, Jakob Meineke, Thomas Amthor, Ekin Karasan, Fei Tan, Jonathan I. Tamir, Stella X. Yu, Michael Lustig

Here we propose a supervised learning-based method that directly synthesizes contrast-weighted images from the MRF data without going through the quantitative mapping and spin-dynamics simulation.

Generative Adversarial Network Magnetic Resonance Fingerprinting +1

PUnifiedNER: A Prompting-based Unified NER System for Diverse Datasets

1 code implementation27 Nov 2022 Jinghui Lu, Rui Zhao, Brian Mac Namee, Fei Tan

In this work, we present a ``versatile'' model -- the Prompting-based Unified NER system (PUnifiedNER) -- that works with data from different domains and can recognise up to 37 entity types simultaneously, and theoretically it could be as many as possible.

named-entity-recognition Named Entity Recognition +1

What Makes Pre-trained Language Models Better Zero-shot Learners?

1 code implementation30 Sep 2022 Jinghui Lu, Dongsheng Zhu, Weidong Han, Rui Zhao, Brian Mac Namee, Fei Tan

Current methods for prompt learning in zeroshot scenarios widely rely on a development set with sufficient human-annotated data to select the best-performing prompt template a posteriori.

Language Modelling text-classification +2

MetaCon: Unified Predictive Segments System with Trillion Concept Meta-Learning

no code implementations9 Mar 2022 Keqian Li, Yifan Hu, Logan Palanisamy, Lisa Jones, Akshay Gupta, Jason Grigsby, Ili Selinger, Matt Gillingham, Fei Tan

Accurate understanding of users in terms of predicative segments play an essential role in the day to day operation of modern internet enterprises.

Meta-Learning

BERT-Beta: A Proactive Probabilistic Approach to Text Moderation

no code implementations EMNLP 2021 Fei Tan, Yifan Hu, Kevin Yen, Changwei Hu

Text moderation for user generated content, which helps to promote healthy interaction among users, has been widely studied and many machine learning models have been proposed.

regression

HABERTOR: An Efficient and Effective Deep Hatespeech Detector

no code implementations EMNLP 2020 Thanh Tran, Yifan Hu, Changwei Hu, Kevin Yen, Fei Tan, Kyumin Lee, Serim Park

HABERTOR inherits BERT's architecture, but is different in four aspects: (i) it generates its own vocabularies and is pre-trained from the scratch using the largest scale hatespeech dataset; (ii) it consists of Quaternion-based factorized components, resulting in a much smaller number of parameters, faster training and inferencing, as well as less memory usage; (iii) it uses our proposed multi-source ensemble heads with a pooling layer for separate input sources, to further enhance its effectiveness; and (iv) it uses a regularized adversarial training with our proposed fine-grained and adaptive noise magnitude to enhance its robustness.

DeepVar: An End-to-End Deep Learning Approach for Genomic Variant Recognition in Biomedical Literature

no code implementations5 Jun 2020 Chaoran Cheng, Fei Tan, Zhi Wei

We consider the problem of Named Entity Recognition (NER) on biomedical scientific literature, and more specifically the genomic variants recognition in this work.

Feature Engineering named-entity-recognition +2

A Blended Deep Learning Approach for Predicting User Intended Actions

no code implementations11 Oct 2018 Fei Tan, Zhi Wei, Jun He, Xiang Wu, Bo Peng, Haoran Liu, Zhenyu Yan

In this work, we focus on pre- dicting attrition, which is one of typical user intended actions.

Deep Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.