Search Results for author:
Found 34 papers, 19 papers with code
Stereo-Talker: Audio-driven 3D Human Synthesis with Prior-Guided Mixture-of-Experts
This paper introduces Stereo-Talker, a novel one-shot audio-driven human video synthesis system that generates 3D talking videos with precise lip synchronization, expressive body gestures, temporally consistent photo-realistic quality, and continuous viewpoint control.
EPD: Long-term Memory Extraction, Context-awared Planning and Multi-iteration Decision @ EgoPlan Challenge ICML 2024
To address the real-world egocentric task planning problem, we introduce a novel planning framework which comprises three stages: long-term memory Extraction, context-awared Planning, and multi-iteration Decision, named EPD.
Hardware Neural Control of CartPole and F1TENTH Race Car
We demonstrate kHz control rates for a physical cartpole and offloading control to the FPGA hardware on the F1TENTH car.
Decision Mamba: A Multi-Grained State Space Model with Self-Evolution Regularization for Offline RL
To capture the relationship among state-action-RTG triplets, a fine-grained SSM module is designed and integrated into the original coarse-grained SSM in mamba, resulting in a novel mamba architecture tailored for offline RL.
Dual-View Visual Contextualization for Web Navigation
Automatic web navigation aims to build a web agent that can follow language instructions to execute complex and diverse tasks on real-world websites.
LION: Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge
1) Progressive incorporation of fine-grained spatial-aware visual knowledge.
RAM-Avatar: Real-time Photo-Realistic Avatar from Monocular Videos with Full-body Control
This paper focuses on advancing the applicability of human avatar learning methods by proposing RAM-Avatar which learns a Real-time photo-realistic Avatar that supports full-body control from Monocular videos.
GMTalker: Gaussian Mixture-based Audio-Driven Emotional talking video Portraits
Finally, we propose a personalized emotion-guided head generator with an emotion mapping network that can synthesize high-fidelity and faithful emotional video portraits.
LION : Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge
1) Progressive incorporation of fine-grained spatial-aware visual knowledge.
AgentBench: Evaluating LLMs as Agents
We present AgentBench, a multi-dimensional evolving benchmark that currently consists of 8 distinct environments to assess LLM-as-Agent's reasoning and decision-making abilities in a multi-turn open-ended generation setting.
Roll Up Your Sleeves: Working with a Collaborative and Engaging Task-Oriented Dialogue System
We introduce TacoBot, a user-centered task-oriented digital assistant designed to guide users through complex real-world tasks with multiple steps.
Mind2Web: Towards a Generalist Agent for the Web
We introduce Mind2Web, the first dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website.
Exploring Chain-of-Thought Style Prompting for Text-to-SQL
Thus, we systematically study how to enhance LLMs' reasoning ability through chain of thought (CoT) style prompting, including the original chain-of-thought prompting (Wei et al., 2022b) and least-to-most prompting (Zhou et al., 2023).
What do LLMs Know about Financial Markets? A Case Study on Reddit Market Sentiment Analysis
Market sentiment analysis on social media content requires knowledge of both financial markets and social media jargon, which makes it a challenging task for human raters.
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs).
Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments
Most existing work for grounded language understanding uses LMs to directly generate plans that can be executed in the environment to achieve the desired effects.
Bootstrapping a User-Centered Task-Oriented Dialogue System
We present TacoBot, a task-oriented dialogue system built for the inaugural Alexa Prize TaskBot Challenge, which assists users in completing multi-step cooking and home improvement tasks.
Reducing Flipping Errors in Deep Neural Networks
Deep neural networks (DNNs) have been widely applied in various domains in artificial intelligence including computer vision and natural language processing.
Iteratively Prompt Pre-trained Language Models for Chain of Thought
While Pre-trained Language Models (PLMs) internalize a great amount of world knowledge, they have been shown incapable of recalling these knowledge to solve tasks requiring complex & multi-step reasoning.
DOM-LM: Learning Generalizable Representations for HTML Documents
We argue that the text and HTML structure together convey important semantics of the content and therefore warrant a special treatment for their representation learning.
Ranked #2 on
Attribute Extraction
on SWDE
Comprehensive Knowledge Distillation with Causal Intervention
Knowledge distillation (KD) addresses model compression by distilling knowledge from a large model (teacher) to a smaller one (student).
ReasonBERT: Pre-trained to Reason with Distant Supervision
We present ReasonBert, a pre-training method that augments language models with the ability to reason over long-range relations and multiple, possibly hybrid contexts.
Ranked #1 on
Semantic Parsing
on GraphQuestions
Graph-Free Knowledge Distillation for Graph Neural Networks
In this paper, we propose to our best knowledge the first dedicated approach to distilling knowledge from a GNN without graph data.
Can Students Outperform Teachers in Knowledge Distillation based Model Compression?
By designing exploratory experiments, we find that model capacity differences are not necessarily the root reason, and the distillation data matters when the student capacity is greater than a threshold.
Learning with Retrospection
Deep neural networks have been successfully deployed in various domains of artificial intelligence, including computer vision and natural language processing.
Sparsity-Control Ternary Weight Networks
However, the existing approaches to training ternary weight networks cannot control the sparsity (i. e., percentage of 0s) of the ternary weights, which undermines the advantage of ternary weights.
Structure-Grounded Pretraining for Text-to-SQL
Additionally, to evaluate different methods under more realistic text-table alignment settings, we create a new evaluation set Spider-Realistic based on Spider dev set with explicit mentions of column names removed, and adopt eight existing text-to-SQL datasets for cross-database evaluation.
Locally Linear Region Knowledge Distillation
To the end, the student is able to better capture the local shape of the teacher function and thus achieves a better performance.
Deep Collective Learning: Learning Optimal Inputs and Weights Jointly in Deep Neural Networks
It is well observed that in deep learning and computer vision literature, visual data are always represented in a manually designed coding scheme (eg., RGB images are represented as integers ranging from 0 to 255 for each channel) when they are input to an end-to-end deep neural network (DNN) for any learning task.
TURL: Table Understanding through Representation Learning
In this paper, we present TURL, a novel framework that introduces the pre-training/fine-tuning paradigm to relational Web tables.
Ranked #1 on
Column Type Annotation
on WikipediaGS-CTA
Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning?
In this paper, we propose a novel meta-learning based training procedure (MLTP) for DNNs and demonstrate that the meta-learning idea can indeed improve the generalization abilities of DNNs.
Easy-to-Hard: Leveraging Simple Questions for Complex Question Generation
This paper makes one of the first efforts toward automatically generating complex questions from knowledge graphs.
Automatic Table completion using Knowledge Base
In this work, we propose to resolve a new type of heterogeneous query viz: tabular query, which contains a natural language query description, column names of the desired table, and an example row.
Leveraging 2-hop Distant Supervision from Table Entity Pairs for Relation Extraction
Given two entities, distant supervision exploits sentences that directly mention them for predicting their semantic relation.
