Search Results for author:
Found 9 papers, 5 papers with code
Empirical Study on Updating Key-Value Memories in Transformer Feed-forward Layers
The feed-forward networks (FFNs) in transformers are recognized as a group of key-value neural memories to restore abstract high-level knowledge.
Unlocking Emergent Modularity in Large Language Models
Despite the benefits of modularity, most Language Models (LMs) are still treated as monolithic models in the pre-train and fine-tune paradigm, with their emergent modularity locked and underutilized.
Transformer-Patcher: One Mistake worth One Neuron
Our method outperforms previous fine-tuning and HyperNetwork-based methods and achieves state-of-the-art performance for Sequential Model Editing (SME).
ARO-Net: Learning Implicit Fields from Anchored Radial Observations
We introduce anchored radial observations (ARO), a novel shape encoding for learning implicit field representation of 3D shapes that is category-agnostic and generalizable amid significant shape variations.
NIFT: Neural Interaction Field and Template for Object Manipulation
Given a few object manipulation demos, NIFT guides the generation of the interaction imitation for a new object instance by matching the Neural Interaction Template (NIT) extracted from the demos in the target Neural Interaction Field (NIF) defined for the new object.
Mixture of Attention Heads: Selecting Attention Heads Per Token
This paper proposes the Mixture of Attention Heads (MoA), a new architecture that combines multi-head attention with the MoE mechanism.
Air-to-Ground Channel Characterization for Low-Height UAVs in Realistic Network Deployments
Due to the decrease in cost, size and weight, \acp{UAV} are becoming more and more popular for general-purpose civil and commercial applications.
Graph2Plan: Learning Floorplan Generation from Layout Graphs
The core component of our learning framework is a deep neural network, Graph2Plan, which converts a layout graph, along with a building boundary, into a floorplan that fulfills both the layout and boundary constraints.
Entity Extraction with Knowledge from Web Scale Corpora
Entity extraction is an important task in text mining and natural language processing.
