Search Results for author:
Found 18 papers, 7 papers with code
Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
We present an approach for estimating the fraction of text in a large corpus which is likely to be substantially modified or produced by a large language model (LLM).
DOF: Accelerating High-order Differential Operators with Forward Propagation
Solving partial differential equations (PDEs) efficiently is essential for analyzing complex physical systems.
Selecting Large Language Model to Fine-tune via Rectified Scaling Law
The ever-growing ecosystem of LLMs has posed a challenge in selecting the most appropriate pre-trained model to fine-tune amidst a sea of options.
TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models
As a result, mPLMs present a script barrier: representations from different scripts are located in different subspaces, which is a strong indicator of why crosslingual transfer involving languages of different scripts shows sub-optimal performance.
MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer
In this paper, we introduce MoSECroT Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer), a novel and challenging task that is especially relevant to low-resource languages for which static word embeddings are available.
In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering
On a new query, instead of adding demonstrations to the prompt, we shift the latent states of the LLM using the ICV.
Forward Laplacian: A New Computational Framework for Neural Network-based Variational Monte Carlo
Neural network-based variational Monte Carlo (NN-VMC) has emerged as a promising cutting-edge technique of ab initio quantum chemistry.
Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective
By using circuit complexity theory, we first give impossibility results showing that bounded-depth Transformers are unable to directly produce correct answers for basic arithmetic/equation tasks unless the model size grows super-polynomially with respect to the input length.
A study of conceptual language similarity: comparison and evaluation
An interesting line of research in natural language processing (NLP) aims to incorporate linguistic typology to bridge linguistic diversity and assist the research of low-resource languages.
Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs
ColexNet's nodes are concepts and its edges are colexifications.
A Crosslingual Investigation of Conceptualization in 1335 Languages
The resulting measure for the conceptual similarity of two languages is complementary to standard genealogical, typological, and surface similarity measures.
Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages
While natural language processing tools have been developed extensively for some of the world's languages, a significant portion of the world's over 7000 languages are still neglected.
Discovering Latent Knowledge in Language Models Without Supervision
Existing techniques for training language models can be misaligned with the truth: if we train models with imitation learning, they may reproduce errors that humans make; if we train them to generate text that humans rate highly, they may output errors that human evaluators can't detect.
Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise
This opens a promising strategy to first train a feature learner rather than a classifier, and then perform linear probing (last layer retraining) in the test environment.
On the Power of Pre-training for Generalization in RL: Provable Benefits and Hardness
Generalization in Reinforcement Learning (RL) aims to learn an agent during training that generalizes to the target environment.
Towards a Theoretical Framework of Out-of-Distribution Generalization
We also introduce a new concept of expansion function, which characterizes to what extent the variance is amplified in the test domains over the training domains, and therefore give a quantitative meaning of invariant features.
Out-of-Distribution Generalization Analysis via Influence Function
One of the definitions of OOD accuracy is worst-domain accuracy.
Risk Variance Penalization
The key of the out-of-distribution (OOD) generalization is to generalize invariance from training domains to target domains.
