Search Results for author: Haoran Zhao

Found 11 papers, 4 papers with code

Agent as Cerebrum, Controller as Cerebellum: Implementing an Embodied LMM-based Agent on Drones

no code implementations25 Nov 2023 Haoran Zhao, Fengxing Pan, Huqiuyue Ping, Yaoming Zhou

In this study, we present a novel paradigm for industrial robotic embodied agents, encapsulating an 'agent as cerebrum, controller as cerebellum' architecture.

Bit Cipher -- A Simple yet Powerful Word Representation System that Integrates Efficiently with Language Models

no code implementations18 Nov 2023 Haoran Zhao, Jake Ryland Williams

While Large Language Models (LLMs) become ever more dominant, classic pre-trained word embeddings sustain their relevance through computational efficiency and nuanced linguistic interpretation.

Dimensionality Reduction named-entity-recognition +6

Explicit Foundation Model Optimization with Self-Attentive Feed-Forward Neural Units

no code implementations13 Nov 2023 Jake Ryland Williams, Haoran Zhao

We will discuss a general result about feed-forward neural networks and then extend this solution to compositional (mult-layer) networks, which are applied to a simplified transformer block containing feed-forward and self-attention layers.

Model Optimization

Reducing the Need for Backpropagation and Discovering Better Optima With Explicit Optimizations of Neural Networks

no code implementations13 Nov 2023 Jake Ryland Williams, Haoran Zhao

Iterative differential approximation methods that rely upon backpropagation have enabled the optimization of neural networks; however, at present, they remain computationally expensive, especially when training models at scale.

Language Modelling

ROMO: Retrieval-enhanced Offline Model-based Optimization

1 code implementation11 Oct 2023 Mingcheng Chen, Haoran Zhao, Yuxiang Zhao, Hulei Fan, Hongqiao Gao, Yong Yu, Zheng Tian

Data-driven black-box model-based optimization (MBO) problems arise in a great number of practical application scenarios, where the goal is to find a design over the whole space maximizing a black-box target function based on a static offline dataset.


AgentSims: An Open-Source Sandbox for Large Language Model Evaluation

1 code implementation8 Aug 2023 Jiaju Lin, Haoran Zhao, Aochi Zhang, Yiting Wu, Huqiuyue Ping, Qin Chen

With ChatGPT-like large language models (LLM) prevailing in the community, how to evaluate the ability of LLMs is an open question.

Language Modelling Large Language Model +1

SurroundNet: Towards Effective Low-Light Image Enhancement

1 code implementation11 Oct 2021 Fei Zhou, Xin Sun, Junyu Dong, Haoran Zhao, Xiao Xiang Zhu

Although Convolution Neural Networks (CNNs) has made substantial progress in the low-light image enhancement task, one critical problem of CNNs is the paradox of model complexity and performance.

Low-Light Image Enhancement

Knowledge Distillation via Instance-level Sequence Learning

no code implementations21 Jun 2021 Haoran Zhao, Xin Sun, Junyu Dong, Zihe Dong, Qiong Li

Recently, distillation approaches are suggested to extract general knowledge from a teacher network to guide a student network.

General Knowledge Knowledge Distillation

Dual Discriminator Adversarial Distillation for Data-free Model Compression

no code implementations12 Apr 2021 Haoran Zhao, Xin Sun, Junyu Dong, Hui Yu, Huiyu Zhou

Then the generated samples are used to train the compact student network under the supervision of the teacher.

Knowledge Distillation Model Compression +1

Similarity Transfer for Knowledge Distillation

no code implementations18 Mar 2021 Haoran Zhao, Kun Gong, Xin Sun, Junyu Dong, Hui Yu

The proposed approach promotes the performance of student model as the virtual sample created by multiple images produces a similar probability distribution in the teacher and student networks.

Knowledge Distillation

Highlight Every Step: Knowledge Distillation via Collaborative Teaching

1 code implementation23 Jul 2019 Haoran Zhao, Xin Sun, Junyu Dong, Changrui Chen, Zihe Dong

Knowledge distillation aims to train a compact student network by transferring knowledge from a larger pre-trained teacher model.

Knowledge Distillation

Cannot find the paper you are looking for? You can Submit a new open access paper.