Search Results for author:
Found 73 papers, 33 papers with code
Swarm Intelligence in Geo-Localization: A Multi-Agent Large Vision-Language Model Collaborative Framework
Along this line, in this paper, we introduce a novel visual geo-localization framework called \name\ that integrates the inherent knowledge of multiple LVLM agents via inter-agent communication to achieve effective geo-localization of images.
Generative Organizational Behavior Simulation using Large Language Model based Autonomous Agents: A Holacracy Perspective
In this paper, we present the technical details and periodic findings of our project, CareerAgent, which aims to build a generative simulation framework for a Holacracy organization using Large Language Model-based Autonomous Agents.
Mixture of In-Context Experts Enhance LLMs' Long Context Awareness
Many studies have revealed that large language models (LLMs) exhibit uneven awareness of different contextual positions. Their limited context awareness can lead to overlooking critical information and subsequent task failures.
Adapting Job Recommendations to User Preference Drift with Behavioral-Semantic Fusion Learning
To address this issue, we propose a novel session-based framework, BISTRO, to timely model user preference through fusion learning of semantic and behavioral information.
Job-SDF: A Multi-Granularity Dataset for Job Skill Demand Forecasting and Benchmarking
We benchmark a range of models on this dataset, evaluating their performance in standard scenarios, in predictions focused on lower value ranges, and in the presence of structural breaks, providing new insights for further research.
Nemotron-4 340B Technical Report
We release the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward.
OPTune: Efficient Online Preference Tuning
In this paper, we propose a more efficient data exploration strategy for online preference tuning (OPTune), which does not rely on human-curated or pre-collected teacher responses but dynamically samples informative responses for on-policy preference alignment.
Facilitating Multi-Role and Multi-Behavior Collaboration of Large Language Models for Online Job Seeking and Recruiting
To this end, we propose MockLLM, a novel applicable framework that divides the person-job matching process into two modules: mock interview generation and two-sided evaluation in handshake protocol, jointly enhancing their performance through collaborative behaviors between interviewers and candidates.
Airport Delay Prediction with Temporal Fusion Transformers
Since flight delay hurts passengers, airlines, and airports, its prediction becomes crucial for the decision-making of all stakeholders in the aviation industry and thus has been attempted by various previous research.
Deep Reinforcement Learning for Real-Time Ground Delay Program Revision and Corresponding Flight Delay Assignments
This paper explores the optimization of Ground Delay Programs (GDP), a prevalent Traffic Management Initiative used in Air Traffic Management (ATM) to reconcile capacity and demand discrepancies at airports.
Towards Efficient Resume Understanding: A Multi-Granularity Multi-Modal Pre-Training Approach
In the contemporary era of widespread online recruitment, resume understanding has been widely acknowledged as a fundamental and crucial task, which aims to extract structured information from resume documents automatically.
The Solution for the ICCV 2023 1st Scientific Figure Captioning Challenge
In this paper, we propose a solution for improving the quality of captions generated for figures in papers.
Nemotron-4 15B Technical Report
We introduce Nemotron-4 15B, a 15-billion-parameter large multilingual language model trained on 8 trillion text tokens.
KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graph
To guarantee the effectiveness, we leverage program language to formulate the multi-hop reasoning process over the KG, and synthesize a code-based instruction dataset to fine-tune the base LLM.
ODIN: Disentangled Reward Mitigates Hacking in RLHF
In this work, we study the issue of reward hacking on the response length, a challenge emerging in Reinforcement Learning from Human Feedback (RLHF) on LLMs.
LiRank: Industrial Large Scale Ranking Models at LinkedIn
We present LiRank, a large-scale ranking framework at LinkedIn that brings to production state-of-the-art modeling architectures and optimization methods.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
Robust Target Detection of Intelligent Integrated Optical Camera and mmWave Radar System
To improve the target detection performance under complex real-world scenarios, this paper proposes an intelligent integrated optical camera and millimeter-wave (mmWave) radar system.
Implicit-explicit Integrated Representations for Multi-view Video Compression
To further enhance the reconstruction quality from the INR codec, we leverage the high-quality reconstructed frames from the explicit codec to achieve inter-view compensation.
Vertiport Navigation Requirements and Multisensor Architecture Considerations for Urban Air Mobility
Communication, Navigation and Surveillance (CNS) technologies are key enablers for future safe operation of drones in urban environments.
Bridging the Information Gap Between Domain-Specific Model and General LLM for Personalized Recommendation
Nevertheless, these methods encounter the certain issue that information such as community behavior pattern in RS domain is challenging to express in natural language, which limits the capability of LLMs to surpass state-of-the-art domain-specific models.
Retrieval meets Long Context Large Language Models
Perhaps surprisingly, we find that LLM with 4K context window using simple retrieval-augmentation at generation can achieve comparable performance to finetuned LLM with 16K context window via positional interpolation on long context tasks, while taking much less computation.
Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks
By classifying infinite-width neural networks and identifying the *optimal* limit, Tensor Programs IV and V demonstrated a universal way, called $\mu$P, for *widthwise hyperparameter transfer*, i. e., predicting optimal hyperparameters of wide neural networks from narrow ones.
Can Urban Air Mobility become reality? Opportunities, challenges and selected research results
Urban Air Mobility (UAM) is a new air transportation system for passengers and cargo in urban environments, enabled by new technologies and integrated into multimodal transportation systems.
Physics-Guided Human Motion Capture with Pose Probability Modeling
To address the obstacles, our key-idea is to employ physics as denoising guidance in the reverse diffusion process to reconstruct physically plausible human motion from a modeled pose probability distribution.
Multi-Grained Multimodal Interaction Network for Entity Linking
Then, to derive the similarity matching score for each mention-entity pair, we device three interaction units to comprehensively explore the intra-modal interaction and inter-modal fusion among features of entities and mentions.
A Comprehensive Survey of Artificial Intelligence Techniques for Talent Analytics
To this end, we present an up-to-date and comprehensive survey on AI technologies used for talent analytics in the field of human resource management.
On the Exploitability of Instruction Tuning
In this work, we investigate how an adversary can exploit instruction tuning by injecting specific instruction-following examples into the training data that intentionally changes the model's behavior.
A Survey on Large Language Models for Recommendation
Large Language Models (LLMs) have emerged as powerful tools in the field of Natural Language Processing (NLP) and have recently gained significant attention in the domain of Recommendation Systems (RS).
Ranked #1 on
on Amazon Review 2023
DELTA: Dynamic Embedding Learning with Truncated Conscious Attention for CTR Prediction
To tackle this problem, inspired by the Global Workspace Theory in conscious processing, which posits that only a specific subset of the product features are pertinent while the rest can be noisy and even detrimental to human-click behaviors, we propose a CTR model that enables Dynamic Embedding Learning with Truncated Conscious Attention for CTR prediction, termed DELTA.
Optimization of binding affinities in chemical space with generative pretrained transformer and deep reinforcement learning
The SGPT-RL method achieved better results than Reinvent on the ACE2 task, where molecular docking was used as the optimization goal.
How Many Demonstrations Do You Need for In-context Learning?
Moreover, ICL (with and w/o CoT) using only one correct demo significantly outperforms all-demo ICL adopted by most previous works, indicating the weakness of LLMs in finding correct demo(s) for input queries, which is difficult to evaluate on the biased datasets.
Multi-Level Association Rule Mining for Wireless Network Time Series Data
Key performance indicators(KPIs) are of great significance in the monitoring of wireless network service quality.
CPMLHO:Hyperparameter Tuning via Cutting Plane and Mixed-Level Optimization
The bilevel optimization is used to automatically update the hyperparameter, and the gradient of the hyperparameter is the approximate gradient based on the best response function.
Heterogeneous Trajectory Forecasting via Risk and Scene Graph Learning
Heterogeneous trajectory forecasting is critical for intelligent transportation systems, but it is challenging because of the difficulty of modeling the complex interaction relations among the heterogeneous road agents as well as their agent-environment constraints.
Faithful Abstractive Summarization via Fact-aware Consistency-constrained Transformer
To deal with this challenge, in this paper, we propose a novel fact-aware abstractive summarization model, named Entity-Relation Pointer Generator Network (ERPGN).
Pre-Train Your Loss: Easy Bayesian Transfer Learning with Informative Priors
Deep learning is increasingly moving towards a transfer learning paradigm whereby large foundation models are fine-tuned on downstream tasks, starting from an initialization learned on the source task.
Plug-In Inversion: Model-Agnostic Inversion for Vision with Data Augmentations
Existing techniques for model inversion typically rely on hard-to-tune regularizers, such as total variation or feature regularization, which must be individually calibrated for each network in order to produce adequate images.
Complexity-Oriented Per-shot Video Coding Optimization
Current per-shot encoding schemes aim to improve the compression efficiency by shot-level optimization.
VQ-GNN: A Universal Framework to Scale up Graph Neural Networks using Vector Quantization
Our framework avoids the "neighbor explosion" problem of GNNs using quantized representations combined with a low-rank version of the graph convolution matrix.
Ranked #12 on
Node Classification
on Reddit
Understanding the Role of Self-Supervised Learning in Out-of-Distribution Detection Task
In this paper, we study how SSL can enhance the performance of the out-of-distribution (OOD) detection task.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
+1
Diurnal or Nocturnal? Federated Learning of Multi-branch Networks from Periodically Shifting Distributions
Federated learning has been deployed to train machine learning models from decentralized client data on mobile devices in practice.
A Field Guide to Federated Optimization
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection.
Long-Short Transformer: Efficient Transformers for Language and Vision
For instance, Transformer-LS achieves 0. 97 test BPC on enwik8 using half the number of parameters than previous method, while being faster and is able to handle 3x as long sequences compared to its full-attention version on the same hardware.
Ranked #1 on
Language Modelling
on enwik8 dev
Inductive Predictions of Extreme Hydrologic Events in The Wabash River Watershed
We present a machine learning method to predict extreme hydrologic events from spatially and temporally varying hydrological and meteorological data.
The Intrinsic Dimension of Images and Its Impact on Learning
We find that common natural image datasets indeed have very low intrinsic dimension relative to the high number of pixels in the images.
GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training
Innovations in neural architectures have fostered significant breakthroughs in language modeling and computer vision.
Ranked #145 on
Image Classification
on CIFAR-10
Modifying Memories in Transformer Models
In this paper, we propose a new task of \emph{explicitly modifying specific factual knowledge in Transformer models while ensuring the model performance does not degrade on the unmodified facts}.
Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attacks
Adversarial Training is proved to be an efficient method to defend against adversarial examples, being one of the few defenses that withstand strong attacks.
Robust Optimization as Data Augmentation for Large-scale Graphs
Data augmentation helps neural networks generalize better by enlarging the training set, but it remains an open question how to effectively augment graph data to enhance the performance of GNNs (Graph Neural Networks).
Ranked #1 on
Graph Property Prediction
on ogbg-ppa
Reducing the Teacher-Student Gap via Spherical Knowledge Disitllation
We investigate this problem by study the gap of confidence between teacher and student.
Towards Accurate Quantization and Pruning via Data-free Knowledge Transfer
Further, we demonstrate that the compact structure and corresponding initialization from the Lottery Ticket Hypothesis can also help in data-free training.
MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients
Adaptive gradient methods such as RMSProp and Adam use exponential moving estimate of the squared gradient to compute adaptive step sizes, achieving better convergence than SGD in face of noisy objectives.
Large-Scale Adversarial Training for Vision-and-Language Representation Learning
We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning.
Ranked #7 on
Visual Entailment
on SNLI-VE val
(using extra training data)
Headless Horseman: Adversarial Attacks on Transfer Learning Models
We first demonstrate successful transfer attacks against a victim network using \textit{only} its feature extractor.
Certified Defenses for Adversarial Patches
Adversarial patch attacks are among one of the most practical threat models against real-world computer vision systems.
SetRank: A Setwise Bayesian Approach for Collaborative Ranking from Implicit Feedback
The recent development of online recommender systems has a focus on collaborative ranking from implicit feedback, such as user clicks and purchases.
Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers
Convex relaxations are effective for training and certifying neural networks against norm-bounded adversarial attacks, but they leave a large gap between certifiable and empirical robustness.
Adaptive Densely Connected Super-Resolution Reconstruction
For a better performance in single image super-resolution(SISR), we present an image super-resolution algorithm based on adaptive dense connection (ADCSR).
Learning from Noisy Anchors for One-stage Object Detection
State-of-the-art object detectors rely on regressing and classifying an extensive list of possible anchors, which are divided into positive and negative samples based on their intersection-over-union (IoU) with corresponding groundtruth objects.
Deep k-NN Defense against Clean-label Data Poisoning Attacks
Targeted clean-label data poisoning is a type of adversarial attack on machine learning systems in which an adversary injects a few correctly-labeled, minimally-perturbed samples into the training data, causing a model to misclassify a particular test sample during inference.
Improved Training of Certifiably Robust Models
Convex relaxations are effective for training and certifying neural networks against norm-bounded adversarial attacks, but they leave a large gap between certifiable and empirical (PGD) robustness.
FreeLB: Enhanced Adversarial Training for Natural Language Understanding
Adversarial training, which minimizes the maximal risk for label-preserving input perturbations, has proved to be effective for improving the generalization of language models.
Adversarially robust transfer learning
By training classifiers on top of these feature extractors, we produce new models that inherit the robustness of their parent networks.
Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
Clean-label poisoning attacks inject innocuous looking (and "correctly" labeled) poison images into training data, causing a model to misclassify a targeted image after being trained on this data.
Enhancing Person-Job Fit for Talent Recruitment: An Ability-aware Neural Network Approach
The wide spread use of online recruitment services has led to information explosion in the job market.
Fine-grained Video Categorization with Redundancy Reduction Attention
Specifically, it firstly summarizes the video by weight-summing all feature vectors in the feature maps of selected frames with a spatio-temporal soft attention, and then predicts which channels to suppress or to enhance according to this summary with a learned non-linear transform.
Ranked #12 on
Action Recognition
on ActivityNet
Person-Job Fit: Adapting the Right Talent for the Right Job with Joint Representation Learning
To this end, in this paper, we propose a novel end-to-end data-driven model based on Convolutional Neural Network (CNN), namely Person-Job Fit Neural Network (PJFNN), for matching a talent qualification to the requirements of a job.
Compressing Neural Networks using the Variational Information Bottelneck
Neural networks can be compressed to reduce memory and computational requirements, or to increase accuracy by facilitating the use of a larger base architecture.
Learning Visual Knowledge Memory Networks for Visual Question Answering
Second is the mechanism for handling multiple knowledge facts expanding from question and answer pairs.
Compressing Neural Networks using the Variational Information Bottleneck
Neural networks can be compressed to reduce memory and computational requirements, or to increase accuracy by facilitating the use of a larger base architecture.
Recruitment Market Trend Analysis with Sequential Latent Variable Models
To this end, in this paper, we propose a new research paradigm for recruitment market analysis by leveraging unsupervised learning techniques for automatically discovering recruitment market trends based on large-scale recruitment data.
Structured Attentions for Visual Question Answering
In this paper, we demonstrate the importance of encoding such relations by showing the limited effective receptive field of ResNet on two datasets, and propose to model the visual attention as a multivariate distribution over a grid-structured Conditional Random Field on image regions.
