Search Results for author: Yiming Zhang

Found 58 papers, 21 papers with code

FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Sounds

1 code implementation1 Jul 2024 Yiming Zhang, Yicheng Gu, Yanhong Zeng, Zhening Xing, Yuancheng Wang, Zhizheng Wu, Kai Chen

Meanwhile, the temporal controller incorporates an onset detector and a timestampbased adapter to achieve precise audio-video alignment.

Audio Generation Video Alignment +1

DORY: Deliberative Prompt Recovery for LLM

no code implementations31 May 2024 Lirong Gao, Ru Peng, Yiming Zhang, Junbo Zhao

Prompt recovery in large language models (LLMs) is crucial for understanding how LLMs work and addressing concerns regarding privacy, copyright, etc.

Forcing Diffuse Distributions out of Language Models

1 code implementation16 Apr 2024 Yiming Zhang, Avi Schwarzschild, Nicholas Carlini, Zico Kolter, Daphne Ippolito

Despite being trained specifically to follow user instructions, today's language models perform poorly when instructed to produce random outputs.

Diversity Language Modelling +1

RankCLIP: Ranking-Consistent Language-Image Pretraining

no code implementations15 Apr 2024 Yiming Zhang, Zhuokai Zhao, Zhaorun Chen, Zhili Feng, Zenghui Ding, Yining Sun

Self-supervised contrastive learning models, such as CLIP, have set new benchmarks for vision-language models in many downstream tasks.

Contrastive Learning

EFSA: Towards Event-Level Financial Sentiment Analysis

no code implementations8 Apr 2024 Tianyu Chen, Yiming Zhang, Guoxin Yu, Dapeng Zhang, Li Zeng, Qing He, Xiang Ao

In this paper, we extend financial sentiment analysis~(FSA) to event-level since events usually serve as the subject of the sentiment in financial text.

Benchmarking Event Extraction +1

RECOST: External Knowledge Guided Data-efficient Instruction Tuning

no code implementations27 Feb 2024 Qi Zhang, Yiming Zhang, Haobo Wang, Junbo Zhao

When it comes to datasets synthesized by LLMs, a common scenario in this field, dirty samples will even be selected with a higher probability than other samples.

Diversity Re-Ranking

OpenHEXAI: An Open-Source Framework for Human-Centered Evaluation of Explainable Machine Learning

no code implementations20 Feb 2024 Jiaqi Ma, Vivian Lai, Yiming Zhang, Chacha Chen, Paul Hamilton, Davor Ljubenkov, Himabindu Lakkaraju, Chenhao Tan

However, properly evaluating the effectiveness of the XAI methods inevitably requires the involvement of human subjects, and conducting human-centered benchmarks is challenging in a number of ways: designing and implementing user studies is complex; numerous design choices in the design space of user study lead to problems of reproducibility; and running user studies can be challenging and even daunting for machine learning researchers.

Decision Making Fairness

PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models

1 code implementation CVPR 2024 Yiming Zhang, Zhening Xing, Yanhong Zeng, Youqing Fang, Kai Chen

Recent advancements in personalized text-to-image (T2I) models have revolutionized content creation, empowering non-experts to generate stunning images with unique styles.

Image Animation

Revisiting the Knowledge Injection Frameworks

no code implementations2 Nov 2023 Peng Fu, Yiming Zhang, Haobo Wang, Weikang Qiu, Junbo Zhao

Briefly, the core of this technique is rooted in an ideological emphasis on the pruning and purification of the external knowledge base to be injected into LLMs.

Towards Multi-Subsession Conversational Recommendation

no code implementations20 Oct 2023 Yu Ji, Qi Shen, Shixuan Zhu, Hang Yu, Yiming Zhang, Chuan Cui, Zhihua Wei

Therefore, we propose a novel conversational recommendation scenario named Multi-Subsession Multi-round Conversational Recommendation (MSMCR), where user would still resort to CRS after several subsessions and might preserve vague interests, and system would proactively ask attributes to activate user interests in the current subsession.

Attribute Recommendation Systems

Multi3DRefer: Grounding Text Description to Multiple 3D Objects

no code implementations ICCV 2023 Yiming Zhang, ZeMing Gong, Angel X. Chang

We introduce the task of localizing a flexible number of objects in real-world 3D scenes using natural language descriptions.

3D visual grounding Contrastive Learning +3

Self-Sampling Meta SAM: Enhancing Few-shot Medical Image Segmentation with Meta-Learning

1 code implementation31 Aug 2023 Yiming Zhang, Tianang Leng, Kun Han, Xiaohui Xie

In conclusion, we present a novel approach for rapid online adaptation in interactive image segmentation, adapting to a new organ in just 0. 83 minutes.

Decoder Few-Shot Learning +4

Effective Prompt Extraction from Language Models

no code implementations13 Jul 2023 Yiming Zhang, Nicholas Carlini, Daphne Ippolito

In experiments with 3 different sources of prompts and 11 underlying large language models, we find that simple text-based attacks can in fact reveal prompts with high probability.


FLamE: Few-shot Learning from Natural Language Explanations

no code implementations13 Jun 2023 Yangqiaoyu Zhou, Yiming Zhang, Chenhao Tan

Natural language explanations have the potential to provide rich information that in principle guides model reasoning.

Classification Few-Shot Learning +1

BiasX: "Thinking Slow" in Toxic Content Moderation with Explanations of Implied Social Biases

no code implementations23 May 2023 Yiming Zhang, Sravani Nanduri, Liwei Jiang, Tongshuang Wu, Maarten Sap

Toxicity annotators and content moderators often default to mental shortcuts when making decisions.

Maybe Only 0.5% Data is Needed: A Preliminary Exploration of Low Training Data Instruction Tuning

no code implementations16 May 2023 Hao Chen, Yiming Zhang, Qi Zhang, Hantao Yang, Xiaomeng Hu, Xuetao Ma, Yifan Yanggong, Junbo Zhao

Instruction tuning for large language models (LLMs) has gained attention from researchers due to its ability to unlock the potential of LLMs in following instructions.

Active Example Selection for In-Context Learning

1 code implementation8 Nov 2022 Yiming Zhang, Shi Feng, Chenhao Tan

For GPT-2, our learned policies demonstrate strong abilities of generalizing to unseen tasks in training, with a $5. 8\%$ improvement on average.

In-Context Learning

Aircraft Ground Taxiing Deduction and Conflict Early Warning Method Based on Control Command Information

no code implementations4 Nov 2022 Jingchang Zhuge, Huiyuan Liang, Yiming Zhang, Shichao Li, Xinyu Yang, Jun Wu

Aircraft taxiing conflict is a threat to the safety of airport operations, mainly due to the human error in control command infor-mation.

Data-Augmented Counterfactual Learning for Bundle Recommendation

no code implementations19 Oct 2022 Shixuan Zhu, Qi Shen, Yiming Zhang, Zhenwei Dong, Zhihua Wei

In this paper, we propose a novel graph learning paradigm called Counterfactual Learning for Bundle Recommendation (CLBR) to mitigate the impact of data sparsity problem and improve bundle recommendation.

counterfactual Data Augmentation +2

CW-ERM: Improving Autonomous Driving Planning with Closed-loop Weighted Empirical Risk Minimization

1 code implementation5 Oct 2022 Eesha Kumar, Yiming Zhang, Stefano Pini, Simon Stent, Ana Ferreira, Sergey Zagoruyko, Christian S. Perone

The imitation learning of self-driving vehicle policies through behavioral cloning is often carried out in an open-loop fashion, ignoring the effect of actions to future states.

Autonomous Driving Imitation Learning

Learning to Ignore Adversarial Attacks

no code implementations23 May 2022 Yiming Zhang, Yangqiaoyu Zhou, Samuel Carton, Chenhao Tan

Despite the strong performance of current NLP models, they can be brittle against adversarial attacks.

Data Augmentation

Caption Feature Space Regularization for Audio Captioning

1 code implementation18 Apr 2022 Yiming Zhang, Hong Yu, Ruoyi Du, Zhanyu Ma, Yuan Dong

To eliminate this negative effect, in this paper, we propose a two-stage framework for audio captioning: (i) in the first stage, via the contrastive learning, we construct a proxy feature space to reduce the distances between captions correlated to the same audio, and (ii) in the second stage, the proxy feature space is utilized as additional supervision to encourage the model to be optimized in the direction that benefits all the correlated captions.

Audio captioning Contrastive Learning +1

Towards Unifying the Label Space for Aspect- and Sentence-based Sentiment Analysis

1 code implementation Findings (ACL) 2022 Yiming Zhang, Min Zhang, Sai Wu, Junbo Zhao

The aspect-based sentiment analysis (ABSA) is a fine-grained task that aims to determine the sentiment polarity towards targeted aspect terms occurring in the sentence.

Aspect-Based Sentiment Analysis Aspect-Based Sentiment Analysis (ABSA) +3

Temporal aware Multi-Interest Graph Neural Network For Session-based Recommendation

no code implementations31 Dec 2021 Qi Shen, Shixuan Zhu, Yitong Pang, Yiming Zhang, Zhihua Wei

Session-based recommendation (SBR) is a challenging task, which aims at recommending next items based on anonymous interaction sequences.

Graph Neural Network Relation +1

Intention Adaptive Graph Neural Network for Category-aware Session-based Recommendation

1 code implementation31 Dec 2021 Chuan Cui, Qi Shen, Shixuan Zhu, Yitong Pang, Yiming Zhang, Hanning Gao, Zhihua Wei

Session-based recommendation (SBR) is proposed to recommend items within short sessions given that user profiles are invisible in various scenarios nowadays, such as e-commerce and short video recommendation.

Graph Neural Network Session-Based Recommendations

Automatic Configuration for Optimal Communication Scheduling in DNN Training

no code implementations27 Dec 2021 Yiqing Ma, Hao Wang, Yiming Zhang, Kai Chen

ByteScheduler partitions and rearranges tensor transmissions to improve the communication efficiency of distributed Deep Neural Network (DNN) training.

Bayesian Optimization Scheduling

Multiple Choice Questions based Multi-Interest Policy Learning for Conversational Recommendation

1 code implementation22 Dec 2021 Yiming Zhang, Lingfei Wu, Qi Shen, Yitong Pang, Zhihua Wei, Fangli Xu, Bo Long, Jian Pei

As a result, we first propose a more realistic CRS learning setting, namely Multi-Interest Multi-round Conversational Recommendation, where users may have multiple interests in attribute instance combinations and accept multiple items with partially overlapped combinations of attribute instances.

Attribute Multiple-choice

Distilling Meta Knowledge on Heterogeneous Graph for Illicit Drug Trafficker Detection on Social Media

1 code implementation NeurIPS 2021 Yiyue Qian, Yiming Zhang, Yanfang Ye, Chuxu Zhang

In this paper, we propose a holistic framework named MetaHG to automatically detect illicit drug traffickers on social media (i. e., Instagram), by tackling the following two new challenges: (1) different from existing works which merely focus on analyzing post content, MetaHG is capable of jointly modeling multi-modal content and relational structured information on social media for illicit drug trafficker detection; (2) in addition, through the proposed meta-learning technique, MetaHG addresses the issue of requiring sufficient data for model training.

Knowledge Distillation Marketing +3

Aggressive Q-Learning with Ensembles: Achieving Both High Sample Efficiency and High Asymptotic Performance

no code implementations17 Nov 2021 Yanqiu Wu, Xinyue Chen, Che Wang, Yiming Zhang, Keith W. Ross

In particular, Truncated Quantile Critics (TQC) achieves state-of-the-art asymptotic training performance on the MuJoCo benchmark with a distributional representation of critics; and Randomized Ensemble Double Q-Learning (REDQ) achieves high sample efficiency that is competitive with state-of-the-art model-based methods using a high update-to-data ratio and target randomization.

Continuous Control Q-Learning +1

EmbRace: Accelerating Sparse Communication for Distributed Training of NLP Neural Networks

no code implementations18 Oct 2021 Shengwei Li, Zhiquan Lai, Dongsheng Li, Yiming Zhang, Xiangyu Ye, Yabo Duan

EmbRace introduces Sparsity-aware Hybrid Communication, which integrates AlltoAll and model parallelism into data-parallel training, so as to reduce the communication overhead of highly sparse parameters.

Image Classification Scheduling

Graph Learning Augmented Heterogeneous Graph Neural Network for Social Recommendation

no code implementations24 Sep 2021 Yiming Zhang, Lingfei Wu, Qi Shen, Yitong Pang, Zhihua Wei, Fangli Xu, Ethan Chang, Bo Long

In this work, we propose an end-to-end heterogeneous global graph learning framework, namely Graph Learning Augmented Heterogeneous Graph Neural Network (GL-HGNN) for social recommendation.

Graph Learning Graph Neural Network

Multi-behavior Graph Contextual Aware Network for Session-based Recommendation

no code implementations24 Sep 2021 Qi Shen, Lingfei Wu, Yitong Pang, Yiming Zhang, Zhihua Wei, Fangli Xu, Bo Long

Based on the global graph, MGCNet attaches the global interest representation to final item representation based on local contextual intention to address the limitation (iii).

Session-Based Recommendations

Nonholonomic dynamics and control of road vehicles: moving toward automation

no code implementations4 Aug 2021 Wubing B. Qin, Yiming Zhang, Dénes Takács, Gábor Stépán, Gábor Orosz

The models are categorized based on how they represent the wheel-ground contact, whether they incorporate the longitudinal dynamics, and whether they consider the steering dynamics.

Heterogeneous Global Graph Neural Networks for Personalized Session-based Recommendation

1 code implementation8 Jul 2021 Yitong Pang, Lingfei Wu, Qi Shen, Yiming Zhang, Zhihua Wei, Fangli Xu, Ethan Chang, Bo Long, Jian Pei

Additionally, existing personalized session-based recommenders capture user preference only based on the sessions of the current user, but ignore the useful item-transition patterns from other user's historical sessions.

Graph Neural Network Session-Based Recommendations

On-Policy Deep Reinforcement Learning for the Average-Reward Criterion

no code implementations14 Jun 2021 Yiming Zhang, Keith W. Ross

Based on this bound, we develop an iterative procedure which produces a sequence of monotonically improved policies for the average reward criterion.

reinforcement-learning Reinforcement Learning (RL)

Hierarchical Adaptive Pooling by Capturing High-order Dependency for Graph Representation Learning

no code implementations13 Apr 2021 Ning Liu, Songlei Jian, Dongsheng Li, Yiming Zhang, Zhiquan Lai, Hongzuo Xu

Graph neural networks (GNN) have been proven to be mature enough for handling graph-structured data on node-level graph representation learning tasks.

Graph Classification Graph Matching +2

Discovery of Physics and Characterization of Microstructure from Data with Bayesian Hidden Physics Models

1 code implementation12 Mar 2021 Steven Atkinson, Yiming Zhang, Liping Wang

Remarkably, we find that the physics learned from the first specimen allows us to understand the backscattering observed in the latter sample, a qualitative feature that is wholly absent from the specimen from which the physics were inferred.

Conversations Gone Alright: Quantifying and Predicting Prosocial Outcomes in Online Conversations

1 code implementation16 Feb 2021 Jiajun Bao, Junjie Wu, Yiming Zhang, Eshwar Chandrasekharan, David Jurgens

Online conversations can go in many directions: some turn out poorly due to antisocial behavior, while others turn out positively to the benefit of all.

MAX Phase Zr2SeC and Its Thermal Conduction Behavior

no code implementations4 Feb 2021 Ke Chen, Xiaojing Bai, Xulin Mu, Pengfei Yan, Nianxiang Qiu, Youbing Li, Jie zhou, Yujie Song, Yiming Zhang, Shiyu Du, Zhifang Chai, Qing Huang

The elemental diversity is crucial to screen out ternary MAX phases with outstanding properties via tuning of bonding types and strength between constitutive atoms.

Materials Science

Average Reward Reinforcement Learning with Monotonic Policy Improvement

no code implementations1 Jan 2021 Yiming Zhang, Keith W. Ross

In continuing control tasks, an agent’s average reward per time step is a more natural performance measure compared to the commonly used discounting framework as it can better capture an agent’s long-term behavior.

reinforcement-learning Reinforcement Learning (RL)

Data-Informed Decomposition for Localized Uncertainty Quantification of Dynamical Systems

no code implementations14 Aug 2020 Waad Subber, Sayan Ghosh, Piyush Pandita, Yiming Zhang, Liping Wang

The region of interest can be specified based on the localization features of the solution, user interest, and correlation length of the random material properties.

Bayesian Inference Uncertainty Quantification

Advances in Bayesian Probabilistic Modeling for Industrial Applications

no code implementations26 Mar 2020 Sayan Ghosh, Piyush Pandita, Steven Atkinson, Waad Subber, Yiming Zhang, Natarajan Chennimalai Kumar, Suryarghya Chakrabarti, Liping Wang

The methodology, called GE's Bayesian Hybrid Modeling (GEBHM), is a probabilistic modeling method, based on the Kennedy and O'Hagan framework, that has been continuously scaled-up and industrialized over several years.

Physical Intuition

First Order Constrained Optimization in Policy Space

2 code implementations NeurIPS 2020 Yiming Zhang, Quan Vuong, Keith W. Ross

We propose a novel approach called First Order Constrained Optimization in Policy Space (FOCOPS) which maximizes an agent's overall reward while ensuring the agent satisfies a set of cost constraints.

D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension

1 code implementation WS 2019 Hongyu Li, Xiyuan Zhang, Yibing Liu, Yiming Zhang, Quan Wang, Xiangyang Zhou, Jing Liu, Hua Wu, Haifeng Wang

In this paper, we introduce a simple system Baidu submitted for MRQA (Machine Reading for Question Answering) 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models.

Machine Reading Comprehension Multi-Task Learning +1

A Strategy for Adaptive Sampling of Multi-fidelity Gaussian Process to Reduce Predictive Uncertainty

no code implementations26 Jul 2019 Sayan Ghosh, Jesper Kristensen, Yiming Zhang, Waad Subber, Liping Wang

Multi-fidelity Gaussian process is a common approach to address the extensive computationally demanding algorithms such as optimization, calibration and uncertainty quantification.

Uncertainty Quantification


1 code implementation ICLR 2019 Quan Vuong, Yiming Zhang, Keith W. Ross

We show how the Natural Policy Gradient and Trust Region Policy Optimization (NPG/TRPO) problems, and the Proximal Policy Optimization (PPO) problem can be addressed by this methodology.

Reinforcement Learning (RL)

Supervised Policy Update for Deep Reinforcement Learning

1 code implementation ICLR 2019 Quan Vuong, Yiming Zhang, Keith W. Ross

We show how the Natural Policy Gradient and Trust Region Policy Optimization (NPG/TRPO) problems, and the Proximal Policy Optimization (PPO) problem can be addressed by this methodology.

reinforcement-learning Reinforcement Learning (RL)

Diagonalwise Refactorization: An Efficient Training Method for Depthwise Convolutions

3 code implementations27 Mar 2018 Zheng Qin, Zhaoning Zhang, Dongsheng Li, Yiming Zhang, Yuxing Peng

Depthwise convolutions provide significant performance benefits owing to the reduction in both parameters and mult-adds.

Policy Gradient For Multidimensional Action Spaces: Action Sampling and Entropy Bonus

no code implementations ICLR 2018 Vuong Ho Quan, Yiming Zhang, Kenny Song, Xiao-Yue Gong, Keith W. Ross

In the case of high-dimensional action spaces, calculating the entropy and the gradient of the entropy requires enumerating all the actions in the action space and running forward and backpropagation for each action, which may be computationally infeasible.

Atari Games reinforcement-learning +1

Cannot find the paper you are looking for? You can Submit a new open access paper.