Search Results for author:
Found 27 papers, 12 papers with code
Heterogeneous Federated Learning with Splited Language Model
Federated Split Learning (FSL) is a promising distributed learning paradigm in practice, which gathers the strengths of both Federated Learning (FL) and Split Learning (SL) paradigms, to ensure model privacy while diminishing the resource overhead of each client, especially on large transformer models in a resource-constrained environment, e. g., Internet of Things (IoT).
VideoAgent: Long-form Video Understanding with Large Language Model as Agent
Long-form video understanding represents a significant challenge within computer vision, demanding a model capable of reasoning over long multi-modal sequences.
Ranked #1 on
Zero-Shot Video Question Answer
on NExT-QA
Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data
However, this assumption is under-explored due to the poorly understood geometry of the multi-modal contrastive space, where a modality gap exists.
Describing Differences in Image Sets with Natural Language
To aid in this discovery process, we explore the task of automatically describing the differences between two $\textbf{sets}$ of images, which we term Set Difference Captioning.
Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation
Recent advances in image tokenizers, such as VQ-VAE, have enabled text-to-image generation using auto-regressive methods, similar to language modeling.
Can large language models provide useful feedback on research papers? A large-scale empirical analysis
We first quantitatively compared GPT-4's generated feedback with human peer reviewer feedback in 15 Nature family journals (3, 096 papers in total) and the ICLR machine learning conference (1, 709 papers).
Machine Learning-guided Lipid Nanoparticle Design for mRNA Delivery
While RNA technologies hold immense therapeutic potential in a range of applications from vaccination to gene editing, the broad implementation of these technologies is hindered by the challenge of delivering these agents effectively.
Inverse Scaling: When Bigger Isn't Better
Here, we present evidence for the claim that LMs may show inverse scaling, or worse task performance with increased scale, e. g., due to flaws in the training objective and data.
FedMLSecurity: A Benchmark for Attacks and Defenses in Federated Learning and Federated LLMs
This paper introduces FedSecurity, an end-to-end benchmark designed to simulate adversarial attacks and corresponding defense mechanisms in Federated Learning (FL).
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models
Language models have been shown to exhibit positive scaling, where performance improves as models are scaled up in terms of size, compute, or data.
Denoising Cosine Similarity: A Theory-Driven Approach for Efficient Representation Learning
To this end, inspired by recent works on denoising and the success of the cosine-similarity-based objective functions in representation learning, we propose the denoising Cosine-Similarity (dCS) loss.
Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysis
We first prove that the formulation characterizes the structure of representations learned with the kernel-based contrastive learning framework.
Deep Clustering with a Constraint for Topological Invariance based on Symmetric InfoNCE
To address the problem, we propose a constraint utilizing symmetric InfoNCE, which helps an objective of deep clustering method in the scenario train the model so as to be efficient for not only non-complex topology but also complex topology datasets.
DGP-Net: Dense Graph Prototype Network for Few-Shot SAR Target Recognition
The inevitable feature deviation of synthetic aperture radar (SAR) image due to the special imaging principle (depression angle variation) leads to poor recognition accuracy, especially in few-shot learning (FSL).
Adapting Pre-trained Vision Transformers from 2D to 3D through Weight Inflation Improves Medical Image Segmentation
Given the prevalence of 3D medical imaging technologies such as MRI and CT that are widely used in diagnosing and treating diverse diseases, 3D segmentation is one of the fundamental tasks of medical image analysis.
Diagnosing and Rectifying Vision Models using Language
Our proposed method can discover high-error data slices, identify influential attributes and further rectify undesirable model behaviors, without requiring any visual data.
Holistic Evaluation of Language Models
We present Holistic Evaluation of Language Models (HELM) to improve the transparency of language models.
Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning
Our systematic analysis demonstrates that this gap is caused by a combination of model initialization and contrastive learning optimization.
Language Models as Recommender Systems: Evaluations and Limitations
Pre-trained language models (PLMs) such as BERT and GPT learn general text representations and encode extensive world knowledge; thus, they can be efficiently and accurately adapted to various downstream tasks.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Collision dominated, ballistic, and viscous regimes of terahertz plasmonic detection by graphene
When the kinematic viscosity ({\nu}) is above a certain critical viscosity value, {\nu}NR, the plasmonic FETs always operates in the viscous non-resonant regime regardless of channel length (L).
Applied Physics Plasma Physics
Biomedical and Clinical English Model Packages in the Stanza Python NLP Library
We introduce biomedical and clinical English model packages for the Stanza Python NLP library.
Inducing Grammar from Long Short-Term Memory Networks by Shapley Decomposition
The principle of compositionality has deep roots in linguistics: the meaning of an expression is determined by its structure and the meanings of its constituents.
Enhancing Transformer with Sememe Knowledge
While large-scale pretraining has achieved great success in many NLP tasks, it has not been fully studied whether external linguistic knowledge can improve data-driven models.
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages
We introduce Stanza, an open-source Python natural language processing toolkit supporting 66 human languages.
Jiuge: A Human-Machine Collaborative Chinese Classical Poetry Generation System
By exposing the options of poetry genres, styles and revision modes, Jiuge, acting as a professional assistant, allows constant and active participation of users in poetic creation.
Large-scale Generative Modeling to Improve Automated Veterinary Disease Coding
We compare the performance of our model with several baselines in a challenging cross-hospital setting with substantial domain shift.
