Search Results for author:
Found 21 papers, 6 papers with code
TLCR: Token-Level Continuous Reward for Fine-grained Reinforcement Learning from Human Feedback
These human preference data, however, are labeled at the sequence level, creating a mismatch between sequence-level preference labels and tokens, which are autoregressively generated from the language model.
General Item Representation Learning for Cold-start Content Recommendations
Cold-start item recommendation is a long-standing challenge in recommendation systems.
Binary Classifier Optimization for Large Language Model Alignment
In the process of this discovery, we identified two techniques for effective alignment: reward shift and underlying distribution matching.
Semiparametric Token-Sequence Co-Supervision
Especially, the robustness of parametric token space which is established during the pretraining step tends to effectively enhance the stability of nonparametric sequence embedding space, a new space established by another language model.
How Well Do Large Language Models Truly Ground?
To reduce issues like hallucinations and lack of control in Large Language Models (LLMs), a common method is to generate responses by grounding on external contexts given as input, known as knowledge-augmented models.
Hexa: Self-Improving for Knowledge-Grounded Dialogue System
A common practice in knowledge-grounded dialogue generation is to explicitly utilize intermediate steps (e. g., web-search, memory retrieval) with modular approaches.
Exploiting the Potential of Seq2Seq Models as Robust Few-Shot Learners
In-context learning, which offers substantial advantages over fine-tuning, is predominantly observed in decoder-only models, while encoder-decoder (i. e., seq2seq) models excel in methods that rely on weight updates.
Effortless Integration of Memory Management into Open-Domain Conversation Systems
Open-domain conversation systems integrate multiple conversation skills into a single system through a modular approach.
MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models
Therefore, we propose MEta Loss TRansformer (MELTR), a plug-in module that automatically and non-linearly combines various loss functions to aid learning the target task via auxiliary learning.
Ranked #2 on
Video Captioning
on YouCook2
Video-Text Representation Learning via Differentiable Weak Temporal Alignment
In this paper, we propose a novel multi-modal self-supervised framework Video-Text Temporally Weak Alignment-based Contrastive Learning (VT-TWINS) to capture significant information from noisy and weakly correlated data using a variant of Dynamic Time Warping (DTW).
MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detection
Human-Object Interaction (HOI) detection is the task of identifying a set of <human, object, interaction> triplets from an image.
Winning the ICCV'2021 VALUE Challenge: Task-aware Ensemble and Transfer Learning with Visual Concepts
The VALUE (Video-And-Language Understanding Evaluation) benchmark is newly introduced to evaluate and analyze multi-modal representation learning algorithms on three video-and-language tasks: Retrieval, QA, and Captioning.
Spectrally Similar Graph Pooling
To further investigate the effectiveness of our proposed method, we evaluate our approach on a real-world problem, image retrieval with visual scene graphs.
ColdExpand: Semi-Supervised Graph Learning in Cold Start
Most real-world graphs are dynamic and eventually face the cold start problem.
Toward General Scene Graph: Integration of Visual Semantic Knowledge with Entity Synset Alignment
In experiment, the integrated scene graph is applied to the image-caption retrieval task as a down-stream task.
DramaQA: Character-Centered Video Story Understanding with Hierarchical QA
Despite recent progress on computer vision and natural language processing, developing a machine that can understand video story is still hard to achieve due to the intrinsic difficulty of video story.
Cut-Based Graph Learning Networks to Discover Compositional Structure of Sequential Video Data
Here, we propose Cut-Based Graph Learning Networks (CB-GLNs) for learning video data by discovering these complex structures of the video.
Compositional Structure Learning for Sequential Video Data
However, most of sequential data, as seen with videos, have complex temporal dependencies that imply variable-length semantic flows and their compositions, and those are hard to be captured by conventional methods.
Constructing Hierarchical Q&A Datasets for Video Story Understanding
Video understanding is emerging as a new paradigm for studying human-like AI.
Visualizing Semantic Structures of Sequential Data by Learning Temporal Dependencies
While conventional methods for sequential learning focus on interaction between consecutive inputs, we suggest a new method which captures composite semantic flows with variable-length dependencies.
Hadamard Product for Low-rank Bilinear Pooling
Bilinear models provide rich representations compared with linear models.
