Search Results for author:
Found 27 papers, 12 papers with code
SHARP: Search-Based Adversarial Attack for Structured Prediction
Adversarial attack of structured prediction models faces various challenges such as the difficulty of perturbing discrete words, the sentence quality issue, and the sensitivity of outputs to small perturbations.
MindAgent: Emergent Gaming Interaction
Large Language Models (LLMs) have the capacity of performing complex scheduling in a multi-agent system and can coordinate these agents into completing sophisticated tasks that require extensive collaboration.
MindDial: Belief Dynamics Tracking with Theory-of-Mind Modeling for Situated Neural Dialogue Generation
We design an explicit mind module that can track three-level beliefs -- the speaker's belief, the speaker's prediction of the listener's belief, and the common belief based on the gap between the first two.
DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning
Pragmatic reasoning plays a pivotal role in deciphering implicit meanings that frequently arise in real-life conversations and is essential for the development of communicative social agents.
MoviePuzzle: Visual Narrative Reasoning through Multimodal Order Learning
We introduce MoviePuzzle, a novel challenge that targets visual narrative reasoning and holistic movie understanding.
Shuo Wen Jie Zi: Rethinking Dictionaries and Glyphs for Chinese Language Pre-training
We introduce CDBERT, a new learning paradigm that enhances the semantics understanding ability of the Chinese PLMs with dictionary knowledge and structure of Chinese characters.
VSTAR: A Video-grounded Dialogue Dataset for Situated Semantic Understanding with Scene and Topic Transitions
Video-grounded dialogue understanding is a challenging problem that requires machine to perceive, parse and reason over situated semantics extracted from weakly aligned video and dialogues.
Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
On the whole, our analysis provides a novel perspective on the role of semantics in developing and evaluating language models' reasoning abilities.
Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Field
Prior works on joint Information Extraction (IE) typically model instance (e. g., event triggers, entities, roles, relations) interactions by representation enhancement, type dependencies scoring, or global decoding.
SQA3D: Situated Question Answering in 3D Scenes
We propose a new task to benchmark scene understanding of embodied agents: Situated Question Answering in 3D Scenes (SQA3D).
Ranked #1 on
Referring Expression
on SQA3D
VGStore: A Multimodal Extension to SPARQL for Querying RDF Scene Graph
Semantic Web technology has successfully facilitated many RDF models with rich data representation methods.
Unsupervised Vision-Language Parsing: Seamlessly Bridging Visual Scene Graphs with Language Structures via Dependency Relationships
Our goal is to bridge the visual scene graphs and linguistic dependency trees seamlessly.
Unsupervised Vision-Language Grammar Induction with Shared Structure Modeling
We introduce a new task, unsupervised vision-language (VL) grammar induction.
Energy-Based Generative Cooperative Saliency Prediction
In this paper, to model the uncertainty of visual saliency, we study the saliency prediction problem from the perspective of generative models by learning a conditional probability distribution over the saliency map given an input image, and treating the saliency prediction as a sampling process from the learned distribution.
Patchwise Generative ConvNet: Training Energy-Based Models From a Single Natural Image for Internal Learning
Exploiting internal statistics of a single natural image has long been recognized as a significant research paradigm where the goal is to learn the distribution of patches within the image without relying on external training data.
Learning Triadic Belief Dynamics in Nonverbal Communication from Videos
By aggregating different beliefs and true world states, our model essentially forms "five minds" during the interactions between two agents.
Learning Cycle-Consistent Cooperative Networks via Alternating MCMC Teaching for Unsupervised Cross-Domain Translation
This paper studies the unsupervised cross-domain translation problem by proposing a generative framework, in which the probability distribution of each domain is represented by a generative cooperative network that consists of an energy-based model and a latent variable model.
Learning Energy-Based Model with Variational Auto-Encoder as Amortized Sampler
In this paper, we propose to learn a variational auto-encoder (VAE) to initialize the finite-step MCMC, such as Langevin dynamics that is derived from the energy function, for efficient amortized sampling of the EBM.
Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis
3D data that contains rich geometry information of objects and scenes is valuable for understanding 3D physical world.
Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs
Aiming to understand how human (false-)belief--a core socio-cognitive ability--would affect human interactions with robots, this paper proposes to adopt a graphical model to unify the representation of object states, robot knowledge, and human (false-)beliefs.
Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification
We propose a generative model of unordered point sets, such as point clouds, in the form of an energy-based model, where the energy function is parameterized by an input-permutation-invariant bottom-up neural network.
Motion-Based Generator Model: Unsupervised Disentanglement of Appearance, Trackable and Intrackable Motions in Dynamic Patterns
To model the motions explicitly, it is natural for the model to be based on the motions or the displacement fields of the pixels.
Reasoning Visual Dialogs with Structural and Partial Observations
The answer to a given question is represented by a node with missing value.
Ranked #14 on
Visual Dialog
on VisDial v0.9 val
Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning
This paper studies the problem of learning the conditional distribution of a high-dimensional output given an input, where the output and input may belong to two different domains, e. g., the output is a photo image and the input is a sketch image.
Learning Dynamic Generator Model by Alternating Back-Propagation Through Time
The non-linear transformation of this transition model can be parametrized by a feedforward neural network.
Learning Descriptor Networks for 3D Shape Synthesis and Analysis
This paper proposes a 3D shape descriptor network, which is a deep convolutional energy-based model, for modeling volumetric shape patterns.
