Search Results for author:
Found 20 papers, 5 papers with code
TNT: Text Normalization based Pre-training of Transformers for Content Moderation
In this work, we present a new language pre-training model TNT (Text Normalization based pre-training of Transformers) for content moderation.
SynthDoc: Bilingual Documents Synthesis for Visual Document Understanding
This paper introduces SynthDoc, a novel synthetic document generation pipeline designed to enhance Visual Document Understanding (VDU) by generating high-quality, diverse datasets that include text, images, tables, and charts.
Reward Difference Optimization For Sample Reweighting In Offline RLHF
Experiments with 7B LLMs on the HH and TL;DR datasets substantiate the effectiveness of our method in both automatic metrics and human evaluation, thereby highlighting its potential for aligning LLMs with human intent and values
SimCT: A Simple Consistency Test Protocol in LLMs Development Lifecycle
SimCT is mainly to proactively check the consistency across different development stages of "bare metal" LLMs or associated services without accessing the model artifacts, in an attempt to expedite the delivery by reducing the back-and-forth alignment communications among multiple teams involved in different development stages.
CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models
We formalize the trade-off between general and domain-specific capabilities, leading to a well-defined Critical Mixture Ratio (CMR) of general and domain data.
What Makes Good Few-shot Examples for Vision-Language Models?
Despite the notable advancements achieved by leveraging pre-trained vision-language (VL) models through few-shot tuning for downstream tasks, our detailed empirical study highlights a significant dependence of few-shot learning outcomes on the careful selection of training examples - a facet that has been previously overlooked in research.
Balancing Speciality and Versatility: a Coarse to Fine Framework for Supervised Fine-tuning Large Language Model
In an overall evaluation of both speciality and versatility, CoFiTune consistently outperforms baseline methods across diverse tasks and model scales.
Consistency Matters: Explore LLMs Consistency From a Black-Box Perspective
The solution to this problem is often time-consuming and labor-intensive, and there is also an additional cost of secondary deployment, resulting in economic and time losses.
What Large Language Models Bring to Text-rich VQA?
This pipeline achieved superior performance compared to the majority of existing Multimodal Large Language Models (MLLM) on four text-rich VQA datasets.
Deeply Coupled Cross-Modal Prompt Learning
Recent advancements in multimodal foundation models (e. g., CLIP) have excelled in zero-shot generalization.
High-fidelity Direct Contrast Synthesis from Magnetic Resonance Fingerprinting
Here we propose a supervised learning-based method that directly synthesizes contrast-weighted images from the MRF data without going through the quantitative mapping and spin-dynamics simulation.
Generative Adversarial Network
Magnetic Resonance Fingerprinting
+1
PUnifiedNER: A Prompting-based Unified NER System for Diverse Datasets
In this work, we present a ``versatile'' model -- the Prompting-based Unified NER system (PUnifiedNER) -- that works with data from different domains and can recognise up to 37 entity types simultaneously, and theoretically it could be as many as possible.
SDA: Simple Discrete Augmentation for Contrastive Sentence Representation Learning
Contrastive learning has recently achieved compelling performance in unsupervised sentence representation.
What Makes Pre-trained Language Models Better Zero-shot Learners?
Current methods for prompt learning in zeroshot scenarios widely rely on a development set with sufficient human-annotated data to select the best-performing prompt template a posteriori.
MetaCon: Unified Predictive Segments System with Trillion Concept Meta-Learning
Accurate understanding of users in terms of predicative segments play an essential role in the day to day operation of modern internet enterprises.
BERT-Beta: A Proactive Probabilistic Approach to Text Moderation
Text moderation for user generated content, which helps to promote healthy interaction among users, has been widely studied and many machine learning models have been proposed.
HABERTOR: An Efficient and Effective Deep Hatespeech Detector
HABERTOR inherits BERT's architecture, but is different in four aspects: (i) it generates its own vocabularies and is pre-trained from the scratch using the largest scale hatespeech dataset; (ii) it consists of Quaternion-based factorized components, resulting in a much smaller number of parameters, faster training and inferencing, as well as less memory usage; (iii) it uses our proposed multi-source ensemble heads with a pooling layer for separate input sources, to further enhance its effectiveness; and (iv) it uses a regularized adversarial training with our proposed fine-grained and adaptive noise magnitude to enhance its robustness.
Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference
The neural attention mechanism plays an important role in many natural language processing applications.
DeepVar: An End-to-End Deep Learning Approach for Genomic Variant Recognition in Biomedical Literature
We consider the problem of Named Entity Recognition (NER) on biomedical scientific literature, and more specifically the genomic variants recognition in this work.
A Blended Deep Learning Approach for Predicting User Intended Actions
In this work, we focus on pre- dicting attrition, which is one of typical user intended actions.
