Search Results for author: Haotian Zhang

Found 54 papers, 20 papers with code

Scheduling Drone and Mobile Charger via Hybrid-Action Deep Reinforcement Learning

no code implementations16 Mar 2024 Jizhe Dou, Haotian Zhang, Guodong Sun

To address this issue, we present a hybrid-action deep reinforcement learning framework, called HaDMC, which uses a standard policy learning algorithm to generate latent continuous actions.

How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts

no code implementations20 Feb 2024 Yusu Qian, Haotian Zhang, Yinfei Yang, Zhe Gan

The remarkable advancements in Multimodal Large Language Models (MLLMs) have not rendered them immune to challenges, particularly in the context of handling deceptive information in prompts, thus producing hallucinated responses under such conditions.

Pixel-Level Change Detection Pseudo-Label Learning for Remote Sensing Change Captioning

no code implementations23 Dec 2023 Chenyang Liu, Keyan Chen, Zipeng Qi, Haotian Zhang, Zhengxia Zou, Zhenwei Shi

The existing methods for Remote Sensing Image Change Captioning (RSICC) perform well in simple scenes but exhibit poorer performance in complex scenes.

Change Detection Pseudo Label

Large Scale Foundation Models for Intelligent Manufacturing Applications: A Survey

no code implementations11 Dec 2023 Haotian Zhang, Semujju Stuart Dereck, Zhicheng Wang, Xianwei Lv, Kang Xu, Liang Wu, Ye Jia, Jing Wu, Zhuo Long, Wensheng Liang, X. G. Ma, Ruiyan Zhuang

Although the applications of artificial intelligence especially deep learning had greatly improved various aspects of intelligent manufacturing, they still face challenges for wide employment due to the poor generalization ability, difficulties to establish high-quality training datasets, and unsatisfactory performance of deep learning methods.

Ferret: Refer and Ground Anything Anywhere at Any Granularity

1 code implementation11 Oct 2023 Haoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du, BoWen Zhang, ZiRui Wang, Liangliang Cao, Shih-Fu Chang, Yinfei Yang

We introduce Ferret, a new Multimodal Large Language Model (MLLM) capable of understanding spatial referring of any shape or granularity within an image and accurately grounding open-vocabulary descriptions.

Hallucination Language Modelling +1

Integrated Sensing and Communications Towards Proactive Beamforming in mmWave V2I via Multi-Modal Feature Fusion (MMFF)

no code implementations4 Oct 2023 Haotian Zhang, Shijian Gao, Xiang Cheng, Liuqing Yang

The future of vehicular communication networks relies on mmWave massive multi-input-multi-output antenna arrays for intensive data transfer and massive vehicle access.

On Uniform Scalar Quantization for Learned Image Compression

no code implementations29 Sep 2023 Haotian Zhang, Li Li, Dong Liu

In principle, we find two factors crucial: one is the discrepancy between the surrogate and rounding, leading to train-test mismatch; the other is gradient estimation risk due to the surrogate, which consists of bias and variance of the gradient estimation.

Image Compression Quantization

Offline and Online Optical Flow Enhancement for Deep Video Compression

no code implementations11 Jul 2023 Chuanbo Tang, Xihua Sheng, Zhuoyuan Li, Haotian Zhang, Li Li, Dong Liu

In the offline stage, we fine-tune a trained optical flow estimation network with the motion information provided by a traditional (non-deep) video compression scheme, e. g. H. 266/VVC, as we believe the motion information of H. 266/VVC achieves a better rate-distortion trade-off.

Motion Estimation Optical Flow Estimation +1

RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model

1 code implementation28 Jun 2023 Keyan Chen, Chenyang Liu, Hao Chen, Haotian Zhang, Wenyuan Li, Zhengxia Zou, Zhenwei Shi

We also propose several ongoing derivatives for instance segmentation tasks, drawing on recent advancements within the SAM community, and compare their performance with RSPrompter.

Image Segmentation Instance Segmentation +2

Intelligent Multi-Modal Sensing-Communication Integration: Synesthesia of Machines

no code implementations25 Jun 2023 Xiang Cheng, Haotian Zhang, Jianan Zhang, Shijian Gao, Sijiang Li, Ziwei Huang, Lu Bai, Zonghui Yang, Xinhu Zheng, Liuqing Yang

Currently, some research efforts have been devoted to exploring multi-modal sensing-communication integration but still lack a comprehensive review.

Continuous Cross-resolution Remote Sensing Image Change Detection

1 code implementation24 May 2023 Hao Chen, Haotian Zhang, Keyan Chen, Chenyao Zhou, Song Chen, Zhengxia Zou, Zhenwei Shi

Toward continuous cross-resolution CD, we propose scale-invariant learning to enforce the model consistently predicting HR results given synthesized samples of varying resolution differences.

Change Detection

Breaking the Curse of Quality Saturation with User-Centric Ranking

no code implementations24 May 2023 Zhuokai Zhao, Yang Yang, Wenyu Wang, Chihuang Liu, Yu Shi, Wenjie Hu, Haotian Zhang, Shuang Yang

A key puzzle in search, ads, and recommendation is that the ranking model can only utilize a small portion of the vastly available user interaction data.

Infinite Physical Monkey: Do Deep Learning Methods Really Perform Better in Conformation Generation?

no code implementations8 Mar 2023 Haotian Zhang, Jintu Zhang, Huifeng Zhao, Dejun Jiang, Yafeng Deng

Recently, with the development of geometric neural networks, the data-driven schemes have been successfully applied in this field, both for molecular conformation generation (in vacuum) and binding pose generation (in protein pocket).

Drug Discovery Molecular Docking +1

Innovative Drug-like Molecule Generation from Flow-based Generative Model

no code implementations12 Nov 2022 Haotian Zhang, Linxiaoyi Wan

To design a drug given a biological molecule by using deep learning methods, there are many successful models published recently.

Molecular Docking

APGKT: Exploiting Associative Path on Skills Graph for Knowledge Tracing

no code implementations5 Oct 2022 Haotian Zhang, Chenyang Bu, Fei Liu, Shuochen Liu, Yuhong Zhang, Xuegang Hu

If a student wants to answer a question correctly, the student should not only master the set of skills involved in the question but also think and obtain the associative path on the skills graph.

Knowledge Tracing Question Answering

Sobolev Training for Implicit Neural Representations with Approximated Image Derivatives

1 code implementation21 Jul 2022 Wentao Yuan, Qingtian Zhu, Xiangyue Liu, Yikang Ding, Haotian Zhang, Chi Zhang

Recently, Implicit Neural Representations (INRs) parameterized by neural networks have emerged as a powerful and promising tool to represent different kinds of signals due to its continuous, differentiable properties, showing superiorities to classical discretized representations.

Inverse Rendering

KD-MVS: Knowledge Distillation Based Self-supervised Learning for Multi-view Stereo

1 code implementation21 Jul 2022 Yikang Ding, Qingtian Zhu, Xiangyue Liu, Wentao Yuan, Haotian Zhang, Chi Zhang

Supervised multi-view stereo (MVS) methods have achieved remarkable progress in terms of reconstruction quality, but suffer from the challenge of collecting large-scale ground-truth depth.

Knowledge Distillation Self-Supervised Learning

Spotting Temporally Precise, Fine-Grained Events in Video

2 code implementations20 Jul 2022 James Hong, Haotian Zhang, Michaël Gharbi, Matthew Fisher, Kayvon Fatahalian

We introduce the task of spotting temporally precise, fine-grained events in video (detecting the precise moment in time events occur).

Action Detection Action Spotting +2

COEM: Cross-Modal Embedding for MetaCell Identification

no code implementations15 Jul 2022 Haiyi Mao, Minxue Jia, Jason Xiaotian Dou, Haotian Zhang, Panayiotis V. Benos

Metacells are disjoint and homogeneous groups of single-cell profiles, representing discrete and highly granular cell states.

GLIPv2: Unifying Localization and Vision-Language Understanding

1 code implementation12 Jun 2022 Haotian Zhang, Pengchuan Zhang, Xiaowei Hu, Yen-Chun Chen, Liunian Harold Li, Xiyang Dai, Lijuan Wang, Lu Yuan, Jenq-Neng Hwang, Jianfeng Gao

We present GLIPv2, a grounded VL understanding model, that serves both localization tasks (e. g., object detection, instance segmentation) and Vision-Language (VL) understanding tasks (e. g., VQA, image captioning).

 Ranked #1 on Phrase Grounding on Flickr30k Entities Test (using extra training data)

Contrastive Learning Image Captioning +7

Grounded Language-Image Pre-training

2 code implementations CVPR 2022 Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, Jianfeng Gao

The unification brings two benefits: 1) it allows GLIP to learn from both detection and grounding data to improve both tasks and bootstrap a good grounding model; 2) GLIP can leverage massive image-text pairs by generating grounding boxes in a self-training fashion, making the learned representation semantic-rich.

Few-Shot Object Detection Zero-Shot Object Detection

Forecasting battery capacity and power degradation with multi-task learning

no code implementations29 Nov 2021 Weihan Li, Haotian Zhang, Bruis van Vlijmen, Philipp Dechent, Dirk Uwe Sauer

In this paper, we propose a data-driven prognostics framework to predict both capacity and power fade simultaneously with multi-task learning.

Multi-Task Learning

TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers

1 code implementation CVPR 2022 Yikang Ding, Wentao Yuan, Qingtian Zhu, Haotian Zhang, Xiangyue Liu, Yuanjiang Wang, Xiao Liu

We analogize MVS back to its nature of a feature matching task and therefore propose a powerful Feature Matching Transformer (FMT) to leverage intra- (self-) and inter- (cross-) attention to aggregate long-range context information within and across images.

3D Reconstruction Feature Correlation

MegLoc: A Robust and Accurate Visual Localization Pipeline

no code implementations25 Nov 2021 Shuxue Peng, Zihang He, Haotian Zhang, Ran Yan, Chuting Wang, Qingtian Zhu, Xiao Liu

In this paper, we present a visual localization pipeline, namely MegLoc, for robust and accurate 6-DoF pose estimation under varying scenarios, including indoor and outdoor scenes, different time across a day, different seasons across a year, and even across years.

Autonomous Driving Pose Estimation +1

Method Towards CVPR 2021 Image Matching Challenge

no code implementations10 Aug 2021 Xiaopeng Bi, Yu Chen, Xinyang Liu, Dehao Zhang, Ran Yan, Zheng Chai, Haotian Zhang, Xiao Liu

This report describes Megvii-3D team's approach towards CVPR 2021 Image Matching Workshop.

Method Towards CVPR 2021 SimLocMatch Challenge

no code implementations10 Aug 2021 Xiaopeng Bi, Ran Yan, Zheng Chai, Haotian Zhang, Xiao Liu

This report describes Megvii-3D team's approach towards SimLocMatch Challenge @ CVPR 2021 Image Matching Workshop.

Amortized Variational Deep Q Network

1 code implementation3 Nov 2020 Haotian Zhang, Yuhao Wang, Jianyong Sun, Zongben Xu

Efficient exploration is one of the most important issues in deep reinforcement learning.

Efficient Exploration OpenAI Gym +1

Recurrent Inference in Text Editing

1 code implementation Findings of the Association for Computational Linguistics 2020 Ning Shi, Ziheng Zeng, Haotian Zhang, Yichen Gong

In neural text editing, prevalent sequence-to-sequence based approaches directly map the unedited text either to the edited text or the editing operations, in which the performance is degraded by the limited source text encoding and long, varying decoding steps.

IA-MOT: Instance-Aware Multi-Object Tracking with Motion Consistency

no code implementations24 Jun 2020 Jiarui Cai, Yizhou Wang, Haotian Zhang, Hung-Min Hsu, Chengqian Ma, Jenq-Neng Hwang

Meanwhile, the spatial attention, which focuses on the foreground within the bounding boxes, is generated from the given instance masks and applied to the extracted embedding features.

Multi-Object Tracking Multiple Object Tracking +1

Learning to be Global Optimizer

no code implementations10 Mar 2020 Haotian Zhang, Jianyong Sun, Zongben Xu

This paper proposes to learn a two-phase (including a minimization phase and an escaping phase) global optimization algorithm for smooth non-convex functions.

Image Classification

On Hyper-parameter Tuning for Stochastic Optimization Algorithms

no code implementations4 Mar 2020 Haotian Zhang, Jianyong Sun, Zongben Xu

This paper proposes the first-ever algorithmic framework for tuning hyper-parameters of stochastic optimization algorithm based on reinforcement learning.

Bayesian Optimization Evolutionary Algorithms

Adaptive Structural Hyper-Parameter Configuration by Q-Learning

no code implementations2 Mar 2020 Haotian Zhang, Jianyong Sun, Zongben Xu

Tuning hyper-parameters for evolutionary algorithms is an important issue in computational intelligence.

Evolutionary Algorithms Q-Learning +3

Learning Neural Surrogate Model for Warm-Starting Bayesian Optimization

no code implementations ICLR 2020 Haotian Zhang, Jian Sun, Zongben Xu

Bayesian optimization is an effective tool to optimize black-box functions and popular for hyper-parameter tuning in machine learning.

Bayesian Optimization

Applying BERT to Document Retrieval with Birch

no code implementations IJCNLP 2019 Zeynep Akkalyoncu Yilmaz, Shengjin Wang, Wei Yang, Haotian Zhang, Jimmy Lin

We present Birch, a system that applies BERT to document retrieval via integration with the open-source Anserini information retrieval toolkit to demonstrate end-to-end search over large document collections.

Information Retrieval Retrieval

Cross-Domain Modeling of Sentence-Level Evidence for Document Retrieval

no code implementations IJCNLP 2019 Zeynep Akkalyoncu Yilmaz, Wei Yang, Haotian Zhang, Jimmy Lin

This paper applies BERT to ad hoc document retrieval on news articles, which requires addressing two challenges: relevance judgments in existing test collections are typically provided only at the document level, and documents often exceed the length that BERT was designed to handle.

Retrieval Sentence

Eye in the Sky: Drone-Based Object Tracking and 3D Localization

no code implementations18 Oct 2019 Haotian Zhang, Gaoang Wang, Zhichao Lei, Jenq-Neng Hwang

Drones, or general UAVs, equipped with a single camera have been widely deployed to a broad range of applications, such as aerial photography, fast goods delivery and most importantly, surveillance.

drone-based object tracking Multi-Object Tracking +3

GetNet: Get Target Area for Image Pairing

no code implementations8 Oct 2019 Henry H. Yu, Jiang Liu, Hao Sun, Ziwen Wang, Haotian Zhang

Image pairing is an important research task in the field of computer vision.

Person Re-Identification

Rekall: Specifying Video Events using Compositions of Spatiotemporal Labels

1 code implementation7 Oct 2019 Daniel Y. Fu, Will Crichton, James Hong, Xinwei Yao, Haotian Zhang, Anh Truong, Avanika Narayan, Maneesh Agrawala, Christopher Ré, Kayvon Fatahalian

Many real-world video analysis applications require the ability to identify domain-specific events in video, such as interviews and commercials in TV news broadcasts, or action sequences in film.

An Internal Learning Approach to Video Inpainting

1 code implementation ICCV 2019 Haotian Zhang, Long Mai, Ning Xu, Zhaowen Wang, John Collomosse, Hailin Jin

We propose a novel video inpainting algorithm that simultaneously hallucinates missing appearance and motion (optical flow) information, building upon the recent 'Deep Image Prior' (DIP) that exploits convolutional network architectures to enforce plausible texture in static images.

Optical Flow Estimation Video Inpainting

Simple Applications of BERT for Ad Hoc Document Retrieval

2 code implementations26 Mar 2019 Wei Yang, Haotian Zhang, Jimmy Lin

Following recent successes in applying BERT to question answering, we explore simple applications to ad hoc document retrieval.

Ad-Hoc Information Retrieval Question Answering +2

TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes

1 code implementation CVPR 2019 Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Nießner, Leonidas Guibas

We introduce, TextureNet, a neural network architecture designed to extract features from high-resolution signals associated with 3D surface meshes (e. g., color texture maps).

3D Semantic Segmentation

Exploit the Connectivity: Multi-Object Tracking with TrackletNet

1 code implementation18 Nov 2018 Gaoang Wang, Yizhou Wang, Haotian Zhang, Renshu Gu, Jenq-Neng Hwang

Multi-object tracking (MOT) is an important and practical task related to both surveillance systems and moving camera applications, such as autonomous driving and robotic vision.

Autonomous Driving Multi-Object Tracking +1

Evaluating Sentence-Level Relevance Feedback for High-Recall Information Retrieval

no code implementations23 Mar 2018 Haotian Zhang, Gordon V. Cormack, Maura R. Grossman, Mark D. Smucker

This study uses a novel simulation framework to evaluate whether the time and effort necessary to achieve high recall using active learning is reduced by presenting the reviewer with isolated sentences, as opposed to full documents, for relevance feedback.

Active Learning Information Retrieval +3

Exploring the Effectiveness of Convolutional Neural Networks for Answer Selection in End-to-End Question Answering

no code implementations25 Jul 2017 Royal Sequiera, Gaurav Baruah, Zhucheng Tu, Salman Mohammed, Jinfeng Rao, Haotian Zhang, Jimmy Lin

Most work on natural language question answering today focuses on answer selection: given a candidate list of sentences, determine which contains the answer.

Answer Selection Retrieval

Integrating Lexical and Temporal Signals in Neural Ranking Models for Searching Social Media Streams

no code implementations25 Jul 2017 Jinfeng Rao, Hua He, Haotian Zhang, Ferhan Ture, Royal Sequiera, Salman Mohammed, Jimmy Lin

To our knowledge, we are the first to integrate lexical and temporal signals in an end-to-end neural network architecture, in which existing neural ranking models are used to generate query-document similarity vectors that feed into a bidirectional LSTM layer for temporal modeling.

Density Estimation Document Ranking

Cannot find the paper you are looking for? You can Submit a new open access paper.