Search Results for author: Ce Zhang

Found 174 papers, 79 papers with code

Don't Waste Your Bits! Squeeze Activations and Gradients for Deep Neural Networks via TinyScript

no code implementations ICML 2020 Fangcheng Fu, Yuzheng Hu, Yihan He, Jiawei Jiang, Yingxia Shao, Ce Zhang, Bin Cui

Recent years have witnessed intensive research interests on training deep neural networks (DNNs) more efficiently by quantization-based compression methods, which facilitate DNNs training in two ways: (1) activations are quantized to shrink the memory consumption, and (2) gradients are quantized to decrease the communication cost.


Learning to Adapt CLIP for Few-Shot Monocular Depth Estimation

no code implementations2 Nov 2023 Xueting Hu, Ce Zhang, Yi Zhang, Bowen Hai, Ke Yu, Zhihai He

When CLIP is used for depth estimation tasks, the patches, divided from the input images, can be combined with a series of semantic descriptions of the depth information to obtain similarity results.

Monocular Depth Estimation

Object-centric Video Representation for Long-term Action Anticipation

1 code implementation31 Oct 2023 Ce Zhang, Changcheng Fu, Shijie Wang, Nakul Agarwal, Kwonjoon Lee, Chiho Choi, Chen Sun

To recognize and predict human-object interactions, we use a Transformer-based neural architecture which allows the "retrieval" of relevant objects for action anticipation at various time scales.

Action Anticipation Human-Object Interaction Detection +3

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time

1 code implementation26 Oct 2023 Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Re, Beidi Chen

We show that contextual sparsity exists, that it can be accurately predicted, and that we can exploit it to speed up LLM inference in wall-clock time without compromising LLM's quality or in-context learning ability.

Effective and Efficient Federated Tree Learning on Hybrid Data

no code implementations18 Oct 2023 Qinbin Li, Chulin Xie, Xiaojun Xu, Xiaoyuan Liu, Ce Zhang, Bo Li, Bingsheng He, Dawn Song

To address this, we propose HybridTree, a novel federated learning approach that enables federated tree learning on hybrid data.

Federated Learning

In-Context Few-Shot Relation Extraction via Pre-Trained Language Models

1 code implementation17 Oct 2023 Yilmazcan Ozyurt, Stefan Feuerriegel, Ce Zhang

To the best of our knowledge, we are the first to reformulate the relation extraction task as a tailored in-context few-shot learning paradigm.

Document-level Relation Extraction Few-Shot Learning +2

DSG: An End-to-End Document Structure Generator

1 code implementation13 Oct 2023 Johannes Rausch, Gentiana Rashiti, Maxim Gusev, Ce Zhang, Stefan Feuerriegel

To the best of our knowledge, our DSG system is the first end-to-end trainable system for hierarchical document parsing.

Optical Character Recognition (OCR)

Towards General and Efficient Online Tuning for Spark

no code implementations5 Sep 2023 Yang Li, Huaijun Jiang, Yu Shen, Yide Fang, Xiaofeng Yang, Danqing Huang, Xinyi Zhang, Wentao Zhang, Ce Zhang, Peng Chen, Bin Cui

The distributed data analytic system -- Spark is a common choice for processing massive volumes of heterogeneous data, while it is challenging to tune its parameters to achieve high performance.

Bayesian Optimization Meta-Learning

BDC-Adapter: Brownian Distance Covariance for Better Vision-Language Reasoning

no code implementations3 Sep 2023 Yi Zhang, Ce Zhang, Zihan Liao, Yushun Tang, Zhihai He

Large-scale pre-trained Vision-Language Models (VLMs), such as CLIP and ALIGN, have introduced a new paradigm for learning transferable visual representations.

BenchTemp: A General Benchmark for Evaluating Temporal Graph Neural Networks

1 code implementation31 Aug 2023 Qiang Huang, Jiawei Jiang, Xi Susie Rao, Ce Zhang, Zhichao Han, Zitao Zhang, Xin Wang, Yongjun He, Quanqing Xu, Yang Zhao, Chuang Hu, Shuo Shang, Bo Du

To handle graphs in which features or connectivities are evolving over time, a series of temporal graph neural networks (TGNNs) have been proposed.

Link Prediction Node Classification

Unsupervised Prototype Adapter for Vision-Language Models

no code implementations22 Aug 2023 Yi Zhang, Ce Zhang, Xueting Hu, Zhihai He

To leverage the valuable knowledge encoded within these models for downstream tasks, several fine-tuning approaches, including prompt tuning methods and adapter-based methods, have been developed to adapt vision-language models effectively with supervision.

Domain Generalization

AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?

no code implementations31 Jul 2023 Qi Zhao, Shijie Wang, Ce Zhang, Changcheng Fu, Minh Quan Do, Nakul Agarwal, Kwonjoon Lee, Chen Sun

We propose to formulate the LTA task from two perspectives: a bottom-up approach that predicts the next actions autoregressively by modeling temporal dynamics; and a top-down approach that infers the goal of the actor and plans the needed procedure to accomplish the goal.

Action Anticipation counterfactual +1

Cross-Modal Concept Learning and Inference for Vision-Language Models

no code implementations28 Jul 2023 Yi Zhang, Ce Zhang, Yushun Tang, Zhihai He

Based on these visual concepts, we construct a discriminative representation of images and learn a concept inference network to perform downstream image classification tasks, such as few-shot learning and domain generalization.

Domain Generalization Few-Shot Learning +1

Improving Retrieval-Augmented Large Language Models via Data Importance Learning

1 code implementation6 Jul 2023 Xiaozhong Lyu, Stefan Grafberger, Samantha Biegel, Shaopeng Wei, Meng Cao, Sebastian Schelter, Ce Zhang

There are exponentially many terms in the multilinear extension, and one key contribution of this paper is a polynomial time algorithm that computes exactly, given a retrieval-augmented model with an additive utility function and a validation set, the data importance of data points in the retrieval corpus using the multilinear extension of the model's utility function.

Imputation Question Answering +1

SalienDet: A Saliency-based Feature Enhancement Algorithm for Object Detection for Autonomous Driving

1 code implementation11 May 2023 Ning Ding, Ce Zhang, Azim Eskandarian

On the other hand, unknown objects, which have not been seen in training sample set, are one of the reasons that hinder autonomous vehicles from driving beyond the operational domain.

Autonomous Driving Incremental Learning +2

OpenBox: A Python Toolkit for Generalized Black-box Optimization

1 code implementation26 Apr 2023 Huaijun Jiang, Yu Shen, Yang Li, Wentao Zhang, Ce Zhang, Bin Cui

Black-box optimization (BBO) has a broad range of applications, including automatic machine learning, experimental design, and database knob tuning.

Experimental Design

Critical Sampling for Robust Evolution Operator Learning of Unknown Dynamical Systems

no code implementations15 Apr 2023 Ce Zhang, Kailiang Wu, Zhihai He

Given an unknown dynamical system, what is the minimum number of samples needed for effective learning of its governing laws and accurate prediction of its future evolution behavior, and how to select these critical samples?

Operator learning

FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU

1 code implementation13 Mar 2023 Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Daniel Y. Fu, Zhiqiang Xie, Beidi Chen, Clark Barrett, Joseph E. Gonzalez, Percy Liang, Christopher Ré, Ion Stoica, Ce Zhang

As a result, when running OPT-175B on a single 16GB GPU, FlexGen achieves significantly higher throughput compared to state-of-the-art offloading systems, reaching a generation throughput of 1 token/s for the first time with an effective batch size of 144.

Language Modelling Large Language Model

Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptation

no code implementations CVPR 2023 Yushun Tang, Ce Zhang, Heng Xu, Shuoshuo Chen, Jie Cheng, Luziwei Leng, Qinghai Guo, Zhihai He

We observe that the performance of this feed-forward Hebbian learning for fully test-time adaptation can be significantly improved by incorporating a feedback neuro-modulation layer.


Hierarchical Classification of Research Fields in the "Web of Science" Using Deep Learning

no code implementations1 Feb 2023 Susie Xi Rao, Peter H. Egger, Ce Zhang

This paper presents a hierarchical classification system that automatically categorizes a scholarly publication using its abstract into a three-tier hierarchical label set (discipline, field, subfield) in a multi-class setting.


Convolution-enhanced Evolving Attention Networks

1 code implementation16 Dec 2022 Yujing Wang, Yaming Yang, Zhuo Li, Jiangang Bai, Mingliang Zhang, Xiangtai Li, Jing Yu, Ce Zhang, Gao Huang, Yunhai Tong

To the best of our knowledge, this is the first work that explicitly models the layer-wise evolution of attention maps.

Image Classification Machine Translation +3

Number-Adaptive Prototype Learning for 3D Point Cloud Semantic Segmentation

no code implementations18 Oct 2022 Yangheng Zhao, Jun Wang, Xiaolong Li, Yue Hu, Ce Zhang, Yanfeng Wang, Siheng Chen

Instead of learning a single prototype for each class, in this paper, we propose to use an adaptive number of prototypes to dynamically describe the different point patterns within a semantic class.

3D Semantic Segmentation Scene Understanding +1

Neural Methods for Logical Reasoning Over Knowledge Graphs

1 code implementation ICLR 2022 Alfonso Amayuelas, Shuai Zhang, Susie Xi Rao, Ce Zhang

We introduce a set of models that use Neural Networks to create one-point vector embeddings to answer the queries.

Benchmarking Knowledge Graphs +1

CARE: Certifiably Robust Learning with Reasoning via Variational Inference

1 code implementation12 Sep 2022 Jiawei Zhang, Linyi Li, Ce Zhang, Bo Li

In particular, we propose a certifiably robust learning with reasoning pipeline (CARE), which consists of a learning component and a reasoning component.

Variational Inference

Improving Privacy-Preserving Vertical Federated Learning by Efficient Communication with ADMM

no code implementations20 Jul 2022 Chulin Xie, Pin-Yu Chen, Ce Zhang, Bo Li

Moreover, we show that a byproduct of our framework is that the weights of learned linear heads reflect the importance of local clients.

Denoising Federated Learning +1

GraphFramEx: Towards Systematic Evaluation of Explainability Methods for Graph Neural Networks

1 code implementation20 Jun 2022 Kenza Amara, Rex Ying, Zitao Zhang, Zhihao Han, Yinan Shan, Ulrik Brandes, Sebastian Schemm, Ce Zhang

As GNN models are deployed to more mission-critical applications, we are in dire need for a common evaluation protocol of explainability methods of GNNs.

Node Classification

Efficient End-to-End AutoML via Scalable Search Space Decomposition

1 code implementation19 Jun 2022 Yang Li, Yu Shen, Wentao Zhang, Ce Zhang, Bin Cui

End-to-end AutoML has attracted intensive interests from both academia and industry which automatically searches for ML pipelines in a space induced by feature engineering, algorithm/model selection, and hyper-parameter tuning.

AutoML Feature Engineering +1

Contrastive Learning for Unsupervised Domain Adaptation of Time Series

1 code implementation13 Jun 2022 Yilmazcan Ozyurt, Stefan Feuerriegel, Ce Zhang

To the best of our knowledge, ours is the first framework to learn domain-invariant, contextual representation for UDA of time series data.

Contrastive Learning Time Series +2

Stochastic Gradient Descent without Full Data Shuffle

1 code implementation12 Jun 2022 Lijie Xu, Shuang Qiu, Binhang Yuan, Jiawei Jiang, Cedric Renggli, Shaoduo Gan, Kaan Kara, Guoliang Li, Ji Liu, Wentao Wu, Jieping Ye, Ce Zhang

In this paper, we first conduct a systematic empirical study on existing data shuffling strategies, which reveals that all existing strategies have room for improvement -- they all suffer in terms of I/O performance or convergence rate.

FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimization

1 code implementation8 Jun 2022 Zhen Wang, Weirui Kuang, Ce Zhang, Bolin Ding, Yaliang Li

Due to this uniqueness, existing HPO benchmarks no longer satisfy the need to compare HPO methods in the FL setting.

Benchmarking Federated Learning +1

Transfer Learning based Search Space Design for Hyperparameter Tuning

no code implementations6 Jun 2022 Yang Li, Yu Shen, Huaijun Jiang, Tianyi Bai, Wentao Zhang, Ce Zhang, Bin Cui

The extensive experiments show that our approach considerably boosts BO by designing a promising and compact search space instead of using the entire space, and outperforms the state-of-the-arts on a wide range of benchmarks, including machine learning and deep learning tuning tasks, and neural architecture search.

Bayesian Optimization BIG-bench Machine Learning +2

TransBO: Hyperparameter Optimization via Two-Phase Transfer Learning

no code implementations6 Jun 2022 Yang Li, Yu Shen, Huaijun Jiang, Wentao Zhang, Zhi Yang, Ce Zhang, Bin Cui

With the extensive applications of machine learning models, automatic hyperparameter optimization (HPO) has become increasingly important.

Hyperparameter Optimization Neural Architecture Search +2

Decentralized Training of Foundation Models in Heterogeneous Environments

1 code implementation2 Jun 2022 Binhang Yuan, Yongjun He, Jared Quincy Davis, Tianyi Zhang, Tri Dao, Beidi Chen, Percy Liang, Christopher Re, Ce Zhang

Our key technical contribution is a scheduling algorithm that allocates different computational "tasklets" in the training of foundation models to a group of decentralized GPU devices connected by a slow heterogeneous network.


Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees

1 code implementation2 Jun 2022 Jue Wang, Binhang Yuan, Luka Rimanic, Yongjun He, Tri Dao, Beidi Chen, Christopher Re, Ce Zhang

Communication compression is a crucial technique for modern distributed learning systems to alleviate their communication bottlenecks over slower networks.

Certifying Some Distributional Fairness with Subpopulation Decomposition

1 code implementation31 May 2022 Mintong Kang, Linyi Li, Maurice Weber, Yang Liu, Ce Zhang, Bo Li

In this paper, we first formulate the certified fairness of an ML model trained on a given data distribution as an optimization problem based on the model performance loss bound on a fairness constrained distribution, which is within bounded distributional distance with the training distribution.


BRIGHT -- Graph Neural Networks in Real-Time Fraud Detection

no code implementations25 May 2022 Mingxuan Lu, Zhichao Han, Susie Xi Rao, Zitao Zhang, Yang Zhao, Yinan Shan, Ramesh Raghunathan, Ce Zhang, Jiawei Jiang

Apart from rule-based and machine learning filters that are already deployed in production, we want to enable efficient real-time inference with graph neural networks (GNNs), which is useful to catch multihop risk propagation in a transaction graph.

Entity Embeddings Fraud Detection

Data Debugging with Shapley Importance over End-to-End Machine Learning Pipelines

1 code implementation23 Apr 2022 Bojan Karlaš, David Dao, Matteo Interlandi, Bo Li, Sebastian Schelter, Wentao Wu, Ce Zhang

We present DataScope (ease. ml/datascope), the first system that efficiently computes Shapley values of training examples over an end-to-end ML pipeline, and illustrate its applications in data debugging for ML training.

BIG-bench Machine Learning Fairness

Modelling graph dynamics in fraud detection with "Attention"

1 code implementation22 Apr 2022 Susie Xi Rao, Clémence Lanfranchi, Shuai Zhang, Zhichao Han, Zitao Zhang, Wei Min, Mo Cheng, Yinan Shan, Yang Zhao, Ce Zhang

At online retail platforms, detecting fraudulent accounts and transactions is crucial to improve customer experience, minimize loss, and avoid unauthorized transactions.

Fraud Detection

SHiFT: An Efficient, Flexible Search Engine for Transfer Learning

1 code implementation4 Apr 2022 Cedric Renggli, Xiaozhe Yao, Luka Kolar, Luka Rimanic, Ana Klimovic, Ce Zhang

Transfer learning can be seen as a data- and compute-efficient alternative to training models from scratch.

Transfer Learning

A Quality Index Metric and Method for Online Self-Assessment of Autonomous Vehicles Sensory Perception

no code implementations4 Mar 2022 Ce Zhang, Azim Eskandarian

The results demonstrate that the proposed evaluation metric accurately assesses the detection quality of camera-based systems in autonomous driving environments.

Autonomous Driving object-detection +2

An attention-based U-Net for detecting deforestation within satellite sensor imagery

1 code implementation International Journal of Applied Earth Observation and Geoinformation 2022 David John, Ce Zhang

In this paper, we implement and analyse an Attention U-Net deep network for semantic segmentation using Sentinel-2 satellite sensor imagery, for the purpose of detecting deforestation within two forest biomes in South America, the Amazon Rainforest and the Atlantic Forest.

Segmentation Semantic Segmentation

Certifying Out-of-Domain Generalization for Blackbox Functions

1 code implementation3 Feb 2022 Maurice Weber, Linyi Li, Boxin Wang, Zhikuan Zhao, Bo Li, Ce Zhang

As a result, the wider application of these techniques is currently limited by its scalability and flexibility -- these techniques often do not scale to large-scale datasets with modern deep neural networks or cannot handle loss functions which may be non-smooth such as the 0-1 loss.

Domain Generalization

ReforesTree: A Dataset for Estimating Tropical Forest Carbon Stock with Deep Learning and Aerial Imagery

no code implementations26 Jan 2022 Gyri Reiersen, David Dao, Björn Lütjens, Konstantin Klemmer, Kenza Amara, Attila Steinegger, Ce Zhang, Xiaoxiang Zhu

The potential for impact and scale of leveraging advancements in machine learning and remote sensing technologies is promising but needs to be of high quality in order to replace the current forest stock protocols for certifications.

Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale

no code implementations18 Jan 2022 Yang Li, Yu Shen, Huaijun Jiang, Wentao Zhang, Jixiang Li, Ji Liu, Ce Zhang, Bin Cui

The ever-growing demand and complexity of machine learning are putting pressure on hyper-parameter tuning systems: while the evaluation cost of models continues to increase, the scalability of state-of-the-arts starts to become a crucial bottleneck.


Dynamic Human Evaluation for Relative Model Comparisons

1 code implementation LREC 2022 Thórhildur Thorleiksdóttir, Cedric Renggli, Nora Hollenstein, Ce Zhang

Collecting human judgements is currently the most reliable evaluation method for natural language generation systems.

Text Generation

Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters

1 code implementation10 Nov 2021 Xiangru Lian, Binhang Yuan, XueFeng Zhu, Yulong Wang, Yongjun He, Honghuan Wu, Lei Sun, Haodong Lyu, Chengjun Liu, Xing Dong, Yiqiao Liao, Mingnan Luo, Congfei Zhang, Jingru Xie, Haonan Li, Lei Chen, Renjie Huang, Jianying Lin, Chengchun Shu, Xuezhong Qiu, Zhishan Liu, Dongying Kong, Lei Yuan, Hai Yu, Sen yang, Ce Zhang, Ji Liu

Specifically, in order to ensure both the training efficiency and the training accuracy, we design a novel hybrid training algorithm, where the embedding layer and the dense neural network are handled by different synchronization mechanisms; then we build a system called Persia (short for parallel recommendation training system with hybrid acceleration) to support this hybrid training algorithm.

Recommendation Systems

iFlood: A Stable and Effective Regularizer

no code implementations ICLR 2022 Yuexiang Xie, Zhen Wang, Yaliang Li, Ce Zhang, Jingren Zhou, Bolin Ding

However, our further studies uncover that the design of the loss function of Flooding can lead to a discrepancy between its objective and implementation, and cause the instability issue.

Image Classification

Towards Automatic Bias Detection in Knowledge Graphs

1 code implementation Findings (EMNLP) 2021 Daphna Keidar, Mian Zhong, Ce Zhang, Yash Raj Shrestha, Bibek Paudel

With the recent surge in social applications relying on knowledge graphs, the need for techniques to ensure fairness in KG based methods is becoming increasingly evident.

Bias Detection Fairness +2

Evaluating Bayes Error Estimators on Real-World Datasets with FeeBee

1 code implementation30 Aug 2021 Cedric Renggli, Luka Rimanic, Nora Hollenstein, Ce Zhang

The Bayes error rate (BER) is a fundamental concept in machine learning that quantifies the best possible accuracy any classifier can achieve on a fixed probability distribution.

LinkTeller: Recovering Private Edges from Graph Neural Networks via Influence Analysis

no code implementations14 Aug 2021 Fan Wu, Yunhui Long, Ce Zhang, Bo Li

We show that these DP GCN mechanisms are not always resilient against LinkTeller empirically under mild privacy guarantees ($\varepsilon>5$).

Privacy Preserving Recommendation Systems +1

Tackling the Overestimation of Forest Carbon with Deep Learning and Aerial Imagery

no code implementations23 Jul 2021 Gyri Reiersen, David Dao, Björn Lütjens, Konstantin Klemmer, Xiaoxiang Zhu, Ce Zhang

This proposal paper describes the first systematic comparison of forest carbon estimation from aerial imagery, satellite imagery, and ground-truth field measurements via deep learning-based algorithms for a tropical reforestation project.

VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition

3 code implementations19 Jul 2021 Yang Li, Yu Shen, Wentao Zhang, Jiawei Jiang, Bolin Ding, Yaliang Li, Jingren Zhou, Zhi Yang, Wentao Wu, Ce Zhang, Bin Cui

End-to-end AutoML has attracted intensive interests from both academia and industry, which automatically searches for ML pipelines in a space induced by feature engineering, algorithm/model selection, and hyper-parameter tuning.

AutoML Feature Engineering +1

Knowledge Enhanced Machine Learning Pipeline against Diverse Adversarial Attacks

1 code implementation11 Jun 2021 Nezihe Merve Gürel, Xiangyu Qi, Luka Rimanic, Ce Zhang, Bo Li

In particular, we develop KEMLP by integrating a diverse set of weak auxiliary models based on their logical relationships to the main DNN model that performs the target task.

BIG-bench Machine Learning

Knowledge Router: Learning Disentangled Representations for Knowledge Graphs

no code implementations NAACL 2021 Shuai Zhang, Xi Rao, Yi Tay, Ce Zhang

To this end, this paper proposes to learn disentangled representations of KG entities - a new method that disentangles the inner latent properties of KG entities.

Knowledge Graphs Representation Learning

OpenBox: A Generalized Black-box Optimization Service

6 code implementations1 Jun 2021 Yang Li, Yu Shen, Wentao Zhang, Yuanwei Chen, Huaijun Jiang, Mingchao Liu, Jiawei Jiang, Jinyang Gao, Wentao Wu, Zhi Yang, Ce Zhang, Bin Cui

Black-box optimization (BBO) has a broad range of applications, including automatic machine learning, engineering, physics, and experimental design.

Experimental Design Transfer Learning

TRS: Transferability Reduced Ensemble via Promoting Gradient Diversity and Model Smoothness

no code implementations NeurIPS 2021 Zhuolin Yang, Linyi Li, Xiaojun Xu, Shiliang Zuo, Qian Chen, Pan Zhou, Benjamin I. P. Rubinstein, Ce Zhang, Bo Li

To answer these questions, in this work we first theoretically analyze and outline sufficient conditions for adversarial transferability between models; then propose a practical algorithm to reduce the transferability between base models within an ensemble to improve its robustness.

Towards Demystifying Serverless Machine Learning Training

1 code implementation17 May 2021 Jiawei Jiang, Shaoduo Gan, Yue Liu, Fanlin Wang, Gustavo Alonso, Ana Klimovic, Ankit Singla, Wentao Wu, Ce Zhang

The appeal of serverless (FaaS) has triggered a growing interest on how to use it in data-intensive applications such as ETL, query processing, or machine learning (ML).

BIG-bench Machine Learning

A Novel Transformer Based Semantic Segmentation Scheme for Fine-Resolution Remote Sensing Images

1 code implementation25 Apr 2021 Libo Wang, Rui Li, Chenxi Duan, Ce Zhang, Xiaoliang Meng, Shenghui Fang

The fully convolutional network (FCN) with an encoder-decoder architecture has been the standard paradigm for semantic segmentation.

Ranked #3 on Semantic Segmentation on ISPRS Potsdam (using extra training data)

Segmentation Semantic Segmentation

Distributed Learning Systems with First-order Methods

1 code implementation12 Apr 2021 Ji Liu, Ce Zhang

Scalable and efficient distributed learning is one of the main driving forces behind the recent rapid advancement of machine learning and artificial intelligence.

BIG-bench Machine Learning Management +1

TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothness

1 code implementation NeurIPS 2021 Zhuolin Yang, Linyi Li, Xiaojun Xu, Shiliang Zuo, Qian Chen, Benjamin Rubinstein, Pan Zhou, Ce Zhang, Bo Li

To answer these questions, in this work we first theoretically analyze and outline sufficient conditions for adversarial transferability between models; then propose a practical algorithm to reduce the transferability between base models within an ensemble to improve its robustness.

DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation

2 code implementations20 Mar 2021 Boxin Wang, Fan Wu, Yunhui Long, Luka Rimanic, Ce Zhang, Bo Li

In this paper, we aim to explore the power of generative models and gradient sparsity, and propose a scalable privacy-preserving generative model DATALENS.

Dimensionality Reduction Navigate +1

Scale-aware Neural Network for Semantic Segmentation of Multi-resolution Remote Sensing Images

no code implementations14 Mar 2021 Libo Wang, Ce Zhang, Rui Li, Chenxi Duan, Xiaoliang Meng, Peter M. Atkinson

However, MSR images suffer from two critical issues: 1) increased scale variation of geo-objects and 2) loss of detailed information at coarse spatial resolutions.

Scene Understanding Segmentation +1

Evolving Attention with Residual Convolutions

2 code implementations20 Feb 2021 Yujing Wang, Yaming Yang, Jiangang Bai, Mingliang Zhang, Jing Bai, Jing Yu, Ce Zhang, Gao Huang, Yunhai Tong

In this paper, we propose a novel and generic mechanism based on evolving attention to improve the performance of transformers.

Image Classification Machine Translation +2

Decoding EEG Brain Activity for Multi-Modal Natural Language Processing

no code implementations17 Feb 2021 Nora Hollenstein, Cedric Renggli, Benjamin Glaus, Maria Barrett, Marius Troendle, Nicolas Langer, Ce Zhang

In this paper, we present the first large-scale study of systematically analyzing the potential of EEG brain activity data for improving natural language processing tasks, with a special focus on which features of the signal are most beneficial.

BIG-bench Machine Learning EEG +3

Switch Spaces: Learning Product Spaces with Sparse Gating

no code implementations17 Feb 2021 Shuai Zhang, Yi Tay, Wenqi Jiang, Da-Cheng Juan, Ce Zhang

In order for learned representations to be effective and efficient, it is ideal that the geometric inductive bias aligns well with the underlying structure of the data.

Inductive Bias Knowledge Graph Completion +1

A2-FPN for Semantic Segmentation of Fine-Resolution Remotely Sensed Images

2 code implementations16 Feb 2021 Rui Li, Shunyi Zheng, Ce Zhang, Chenxi Duan, Libo Wang

Based on FPN and AAM, a novel framework named Attention Aggregation Feature Pyramid Network (A2-FPN) is developed for semantic segmentation of fine-resolution remotely sensed images.

Decision Making Scene Understanding +2

A Data Quality-Driven View of MLOps

no code implementations15 Feb 2021 Cedric Renggli, Luka Rimanic, Nezihe Merve Gürel, Bojan Karlaš, Wentao Wu, Ce Zhang

Developing machine learning models can be seen as a process similar to the one established for traditional software development.

BIG-bench Machine Learning

1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed

2 code implementations4 Feb 2021 Hanlin Tang, Shaoduo Gan, Ammar Ahmad Awan, Samyam Rajbhandari, Conglong Li, Xiangru Lian, Ji Liu, Ce Zhang, Yuxiong He

One of the most effective methods is error-compensated compression, which offers robust convergence speed even under 1-bit compression.

EEG-Inception: An Accurate and Robust End-to-End Neural Network for EEG-based Motor Imagery Classification

no code implementations24 Jan 2021 Ce Zhang, Young-Keun Kim, Azim Eskandarian

The proposed CNN model, namely EEG-Inception, is built on the backbone of the Inception-Time network, which showed to be highly efficient and accurate for time-series classification.

Classification Data Augmentation +5

Predictive Attention Transformer: Improving Transformer with Attention Map Prediction

no code implementations1 Jan 2021 Yujing Wang, Yaming Yang, Jiangang Bai, Mingliang Zhang, Jing Bai, Jing Yu, Ce Zhang, Yunhai Tong

Instead, we model their dependencies via a chain of prediction models that take previous attention maps as input to predict the attention maps of a new layer through convolutional neural networks.

Machine Translation

Suspicious Massive Registration Detection via Dynamic Heterogeneous Graph Neural Networks

no code implementations20 Dec 2020 Susie Xi Rao, Shuai Zhang, Zhichao Han, Zitao Zhang, Wei Min, Mo Cheng, Yinan Shan, Yang Zhao, Ce Zhang

Massive account registration has raised concerns on risk management in e-commerce companies, especially when registration increases rapidly within a short time frame.


Efficient Automatic CASH via Rising Bandits

no code implementations8 Dec 2020 Yang Li, Jiawei Jiang, Jinyang Gao, Yingxia Shao, Ce Zhang, Bin Cui

In this framework, the BO methods are used to solve the HPO problem for each ML algorithm separately, incorporating a much smaller hyperparameter space for BO methods.

Bayesian Optimization BIG-bench Machine Learning +2

MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurements

5 code implementations5 Dec 2020 Yang Li, Yu Shen, Jiawei Jiang, Jinyang Gao, Ce Zhang, Bin Cui

Instead of sampling configurations randomly in HB, BOHB samples configurations based on a BO surrogate model, which is constructed with the high-fidelity measurements only.

Bayesian Optimization Hyperparameter Optimization

Learning to Mutate with Hypergradient Guided Population

no code implementations NeurIPS 2020 Zhiqiang Tao, Yaliang Li, Bolin Ding, Ce Zhang, Jingren Zhou, Yun Fu

Computing the gradient of model hyperparameters, i. e., hypergradient, enables a promising and natural way to solve the hyperparameter optimization task.

Hyperparameter Optimization

Multi-stage Attention ResU-Net for Semantic Segmentation of Fine-Resolution Remote Sensing Images

1 code implementation29 Nov 2020 Rui Li, Shunyi Zheng, Chenxi Duan, Jianlin Su, Ce Zhang

The attention mechanism can refine the extracted feature maps and boost the classification performance of the deep network, which has become an essential technique in computer vision and natural language processing.

Semantic Segmentation

xFraud: Explainable Fraud Transaction Detection

1 code implementation24 Nov 2020 Susie Xi Rao, Shuai Zhang, Zhichao Han, Zitao Zhang, Wei Min, Zhiyao Chen, Yinan Shan, Yang Zhao, Ce Zhang

At online retail platforms, it is crucial to actively detect the risks of transactions to improve customer experience and minimize financial loss.

Explainable Models Fraud Detection +1

Robust Unsupervised Small Area Change Detection from SAR Imagery Using Deep Learning

1 code implementation22 Nov 2020 Xinzheng Zhang, Hang Su, Ce Zhang, Xiaowei Gu, Xiaoheng Tan, Peter M. Atkinson

In this paper, a robust unsupervised approach is proposed for small area change detection from multi-temporal SAR images using deep learning.

Change Detection Clustering +1

Learning User Representations with Hypercuboids for Recommender Systems

3 code implementations11 Nov 2020 Shuai Zhang, Huoyu Liu, Aston Zhang, Yue Hu, Ce Zhang, Yumeng Li, Tanchao Zhu, Shaojian He, Wenwu Ou

Furthermore, we present two variants of hypercuboids to enhance the capability in capturing the diversities of user interests.

Collaborative Filtering Recommendation Systems

Online Active Model Selection for Pre-trained Classifiers

1 code implementation19 Oct 2020 Mohammad Reza Karimi, Nezihe Merve Gürel, Bojan Karlaš, Johannes Rausch, Ce Zhang, Andreas Krause

Given $k$ pre-trained classifiers and a stream of unlabeled data examples, how can we actively decide when to query a label so that we can distinguish the best model from the rest while making a small number of queries?

Model Selection

Automatic Feasibility Study via Data Quality Analysis for ML: A Case-Study on Label Noise

2 code implementations16 Oct 2020 Cedric Renggli, Luka Rimanic, Luka Kolar, Wentao Wu, Ce Zhang

In our experience of working with domain experts who are using today's AutoML systems, a common problem we encountered is what we call "unrealistic expectations" -- when users are facing a very challenging task with a noisy data acquisition process, while being expected to achieve startlingly high accuracy with machine learning (ML).

AutoML BIG-bench Machine Learning

On Convergence of Nearest Neighbor Classifiers over Feature Transformations

no code implementations NeurIPS 2020 Luka Rimanic, Cedric Renggli, Bo Li, Ce Zhang

This analysis requires in-depth understanding of the properties that connect both the transformed space and the raw feature space.

Which Model to Transfer? Finding the Needle in the Growing Haystack

no code implementations CVPR 2022 Cedric Renggli, André Susano Pinto, Luka Rimanic, Joan Puigcerver, Carlos Riquelme, Ce Zhang, Mario Lucic

Transfer learning has been recently popularized as a data-efficient alternative to training models from scratch, in particular for computer vision tasks where it provides a remarkably solid baseline.

Transfer Learning

MicroRec: Efficient Recommendation Inference by Hardware and Data Structure Solutions

no code implementations12 Oct 2020 Wenqi Jiang, Zhenhao He, Shuai Zhang, Thomas B. Preußer, Kai Zeng, Liang Feng, Jiansong Zhang, Tongxuan Liu, Yong Li, Jingren Zhou, Ce Zhang, Gustavo Alonso

MicroRec accelerates recommendation inference by (1) redesigning the data structures involved in the embeddings to reduce the number of lookups needed and (2) taking advantage of the availability of High-Bandwidth Memory (HBM) in FPGA accelerators to tackle the latency by enabling parallel lookups.

Recommendation Systems

Optimal Provable Robustness of Quantum Classification via Quantum Hypothesis Testing

no code implementations21 Sep 2020 Maurice Weber, Nana Liu, Bo Li, Ce Zhang, Zhikuan Zhao

This link leads to a tight robustness condition which puts constraints on the amount of noise a classifier can tolerate, independent of whether the noise source is natural or adversarial.

Classification General Classification +2

A Principled Approach to Data Valuation for Federated Learning

no code implementations14 Sep 2020 Tianhao Wang, Johannes Rausch, Ce Zhang, Ruoxi Jia, Dawn Song

The federated SV preserves the desirable properties of the canonical SV while it can be calculated without incurring extra communication cost and is also able to capture the effect of participation order on data value.

Data Summarization Data Valuation +1

Multi-Attention-Network for Semantic Segmentation of Fine Resolution Remote Sensing Images

no code implementations3 Sep 2020 Rui Li, Shunyi Zheng, Chenxi Duan, Ce Zhang, Jianlin Su, P. M. Atkinson

A novel attention mechanism of kernel attention with linear complexity is proposed to alleviate the large computational demand in attention.

Management Semantic Segmentation

APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm

no code implementations26 Aug 2020 Hanlin Tang, Shaoduo Gan, Samyam Rajbhandari, Xiangru Lian, Ji Liu, Yuxiong He, Ce Zhang

Adam is the important optimization algorithm to guarantee efficiency and accuracy for training many important tasks such as BERT and ImageNet.

A Survey and Tutorial of EEG-Based Brain Monitoring for Driver State Analysis

no code implementations25 Aug 2020 Ce Zhang, Azim Eskandarian

Then, the EEG signal preprocessing, feature extraction, and classification algorithms for driver state detection are reviewed.

Autonomous Vehicles EEG +1

A Computationally Efficient Multiclass Time-Frequency Common Spatial Pattern Analysis on EEG Motor Imagery

no code implementations25 Aug 2020 Ce Zhang, Azim Eskandarian

The experiment results show the proposed algorithm average computation time is 37. 22% less than the FBCSP (1st winner in the BCI Competition IV) and 4. 98% longer than the conventional CSP method.

Classification EEG +2

Land Cover Classification from Remote Sensing Images Based on Multi-Scale Fully Convolutional Network

1 code implementation1 Aug 2020 Rui Li, Shunyi Zheng, Chenxi Duan, Ce Zhang

In this paper, a Multi-Scale Fully Convolutional Network (MSFCN) with multi-scale convolutional kernel is proposed to exploit discriminative representations from two-dimensional (2D) satellite images.

General Classification Land Cover Classification

FIVES: Feature Interaction Via Edge Search for Large-Scale Tabular Data

no code implementations29 Jul 2020 Yuexiang Xie, Zhen Wang, Yaliang Li, Bolin Ding, Nezihe Merve Gürel, Ce Zhang, Minlie Huang, Wei. Lin, Jingren Zhou

Then we instantiate this search strategy by optimizing both a dedicated graph neural network (GNN) and the adjacency tensor associated with the defined feature graph.

Recommendation Systems

MACU-Net for Semantic Segmentation of Fine-Resolution Remotely Sensed Images

2 code implementations26 Jul 2020 Rui Li, Chenxi Duan, Shunyi Zheng, Ce Zhang, Peter M. Atkinson

In this Letter, we incorporate multi-scale features generated by different layers of U-Net and design a multi-scale skip connected and asymmetric-convolution-based U-Net (MACU-Net), for segmentation using fine-resolution remotely sensed images.

Image Segmentation Management +3

Adversarial Learning for Debiasing Knowledge Graph Embeddings

no code implementations29 Jun 2020 Mario Arduini, Lorenzo Noci, Federico Pirovano, Ce Zhang, Yash Raj Shrestha, Bibek Paudel

As a second step, we explore gender bias in KGE, and a careful examination of popular KGE algorithms suggest that sensitive attribute like the gender of a person can be predicted from the embedding.

Knowledge Graph Embeddings Knowledge Graphs +1

Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions

1 code implementation11 May 2020 Bojan Karlaš, Peng Li, Renzhi Wu, Nezihe Merve Gürel, Xu Chu, Wentao Wu, Ce Zhang

Machine learning (ML) applications have been thriving recently, largely attributed to the increasing availability of data.

BIG-bench Machine Learning

TrueBranch: Metric Learning-based Verification of Forest Conservation Projects

no code implementations21 Apr 2020 Simona Santamaria, David Dao, Björn Lütjens, Ce Zhang

Recent works propose low-cost and accurate MRV via automatically determining forest carbon from drone imagery, collected by the landowners.

Metric Learning

RAB: Provable Robustness Against Backdoor Attacks

1 code implementation19 Mar 2020 Maurice Weber, Xiaojun Xu, Bojan Karlaš, Ce Zhang, Bo Li

In addition, we theoretically show that it is possible to train the robust smoothed models efficiently for simple models such as K-nearest neighbor classifiers, and we propose an exact smooth-training algorithm that eliminates the need to sample from a noise distribution for such models.

BIG-bench Machine Learning

A Robust Imbalanced SAR Image Change Detection Approach Based on Deep Difference Image and PCANet

no code implementations3 Mar 2020 Xinzheng Zhang, Hang Su, Ce Zhang, Peter M. Atkinson, Xiaoheng Tan, Xiaoping Zeng, Xin Jian

Parallel FCM are utilized on these two mapped DDIs to obtain three types of pseudo-label pixels, namely, changed pixels, unchanged pixels, and intermediate pixels.

Change Detection Clustering +1

Improving Certified Robustness via Statistical Learning with Logical Reasoning

1 code implementation28 Feb 2020 Zhuolin Yang, Zhikuan Zhao, Boxin Wang, Jiawei Zhang, Linyi Li, Hengzhi Pei, Bojan Karlas, Ji Liu, Heng Guo, Ce Zhang, Bo Li

Intensive algorithmic efforts have been made to enable the rapid improvements of certificated robustness for complex ML models recently.

BIG-bench Machine Learning Logical Reasoning

TSS: Transformation-Specific Smoothing for Robustness Certification

1 code implementation27 Feb 2020 Linyi Li, Maurice Weber, Xiaojun Xu, Luka Rimanic, Bhavya Kailkhura, Tao Xie, Ce Zhang, Bo Li

Moreover, to the best of our knowledge, TSS is the first approach that achieves nontrivial certified robustness on the large-scale ImageNet dataset.


Data Science through the looking glass and what we found there

no code implementations19 Dec 2019 Fotis Psallidas, Yiwen Zhu, Bojan Karlas, Matteo Interlandi, Avrilia Floratou, Konstantinos Karanasos, Wentao Wu, Ce Zhang, Subru Krishnan, Carlo Curino, Markus Weimer

The recent success of machine learning (ML) has led to an explosive growth both in terms of new systems and algorithms built in industry and academia, and new applications built by an ever-growing community of data science (DS) practitioners.

Support Vector Machine Classifier via $L_{0/1}$ Soft-Margin Loss

no code implementations16 Dec 2019 Huajun Wang, Yuan-Hai Shao, Shenglong Zhou, Ce Zhang, Naihua Xiu

To distinguish all of them, in this paper, we introduce a new model equipped with an $L_{0/1}$ soft-margin loss (dubbed as $L_{0/1}$-SVM) which well captures the nature of the binary classification.

Binary Classification

ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation

no code implementations LREC 2020 Nora Hollenstein, Marius Troendle, Ce Zhang, Nicolas Langer

We recorded and preprocessed ZuCo 2. 0, a new dataset of simultaneous eye-tracking and electroencephalography during natural reading and during annotation.

Scalability vs. Utility: Do We Have to Sacrifice One for the Other in Data Importance Quantification?

1 code implementation CVPR 2021 Ruoxi Jia, Fan Wu, Xuehui Sun, Jiacen Xu, David Dao, Bhavya Kailkhura, Ce Zhang, Bo Li, Dawn Song

Quantifying the importance of each training point to a learning task is a fundamental problem in machine learning and the estimated importance scores have been leveraged to guide a range of data workflows such as data summarization and domain adaption.

Data Summarization Domain Adaptation

DocParser: Hierarchical Structure Parsing of Document Renderings

2 code implementations5 Nov 2019 Johannes Rausch, Octavio Martinez, Fabian Bissig, Ce Zhang, Stefan Feuerriegel

Translating renderings (e. g. PDFs, scans) into hierarchical document structures is extensively demanded in the daily routines of many real-world applications.

DeGNN: Characterizing and Improving Graph Neural Networks with Graph Decomposition

no code implementations10 Oct 2019 Xupeng Miao, Nezihe Merve Gürel, Wentao Zhang, Zhichao Han, Bo Li, Wei Min, Xi Rao, Hansheng Ren, Yinan Shan, Yingxia Shao, Yujie Wang, Fan Wu, Hui Xue, Yaming Yang, Zitao Zhang, Yang Zhao, Shuai Zhang, Yujing Wang, Bin Cui, Ce Zhang

Despite the wide application of Graph Convolutional Network (GCN), one major limitation is that it does not benefit from the increasing depth and suffers from the oversmoothing problem.

Observer Dependent Lossy Image Compression

1 code implementation8 Oct 2019 Maurice Weber, Cedric Renggli, Helmut Grabner, Ce Zhang

To that end, we use a family of loss functions that allows to optimize deep image compression depending on the observer and to interpolate between human perceived visual quality and classification accuracy, enabling a more unified view on image compression.

Classification General Classification +4

An Empirical and Comparative Analysis of Data Valuation with Scalable Algorithms

no code implementations25 Sep 2019 Ruoxi Jia, Xuehui Sun, Jiacen Xu, Ce Zhang, Bo Li, Dawn Song

Existing approximation algorithms, although achieving great improvement over the exact algorithm, relies on retraining models for multiple times, thus remaining limited when applied to larger-scale learning tasks and real-world datasets.

Data Summarization Data Valuation +1

CogniVal: A Framework for Cognitive Word Embedding Evaluation

1 code implementation CONLL 2019 Nora Hollenstein, Antonio de la Torre, Nicolas Langer, Ce Zhang

An interesting method of evaluating word representations is by how much they reflect the semantic representations in the human brain.

EEG Electroencephalogram (EEG) +1

Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithms

3 code implementations22 Aug 2019 Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nezihe Merve Gurel, Bo Li, Ce Zhang, Costas J. Spanos, Dawn Song

The most surprising result is that for unweighted $K$NN classifiers and regressors, the Shapley value of all $N$ data points can be computed, exactly, in $O(N\log N)$ time -- an exponential improvement on computational complexity!

Data Valuation Fairness

$\texttt{DeepSqueeze}$: Decentralization Meets Error-Compensated Compression

no code implementations17 Jul 2019 Hanlin Tang, Xiangru Lian, Shuang Qiu, Lei Yuan, Ce Zhang, Tong Zhang, Ji Liu

Since the \emph{decentralized} training has been witnessed to be superior to the traditional \emph{centralized} training in the communication restricted scenario, therefore a natural question to ask is "how to apply the error-compensated technology to the decentralized learning to further reduce the communication cost."

CleanML: A Study for Evaluating the Impact of Data Cleaning on ML Classification Tasks

no code implementations20 Apr 2019 Peng Li, Xi Rao, Jennifer Blase, Yue Zhang, Xu Chu, Ce Zhang

Data quality affects machine learning (ML) model performances, and data scientists spend considerable amount of time on data cleaning before model training.

General Classification Two-sample testing

Sensing Social Media Signals for Cryptocurrency News

no code implementations27 Mar 2019 Johannes Beck, Roberta Huang, David Lindner, Tian Guo, Ce Zhang, Dirk Helbing, Nino Antulov-Fantulin

The ability to track and monitor relevant and important news in real-time is of crucial interest in multiple industrial sectors.

BIG-bench Machine Learning

Continuous Integration of Machine Learning Models with Towards a Rigorous Yet Practical Treatment

no code implementations1 Mar 2019 Cedric Renggli, Bojan Karlaš, Bolin Ding, Feng Liu, Kevin Schawinski, Wentao Wu, Ce Zhang

Continuous integration is an indispensable step of modern software engineering practices to systematically manage the life cycles of system development.

BIG-bench Machine Learning Test

Towards Efficient Data Valuation Based on the Shapley Value

1 code implementation27 Feb 2019 Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gurel, Bo Li, Ce Zhang, Dawn Song, Costas Spanos

In this paper, we study the problem of data valuation by utilizing the Shapley value, a popular notion of value which originated in cooperative game theory.

Data Valuation

Entity Recognition at First Sight: Improving NER with Eye Movement Information

1 code implementation NAACL 2019 Nora Hollenstein, Ce Zhang

Previous research shows that eye-tracking data contains information about the lexical and syntactic properties of text, which can be used to improve natural language processing models.

named-entity-recognition Named Entity Recognition +2

Exploring galaxy evolution with generative models

no code implementations3 Dec 2018 Kevin Schawinski, M. Dennis Turp, Ce Zhang

Methods: By learning a latent space representation of the data, we can use this network to forward model and explore hypotheses in a data-driven way.


Distributed Learning over Unreliable Networks

no code implementations17 Oct 2018 Chen Yu, Hanlin Tang, Cedric Renggli, Simon Kassing, Ankit Singla, Dan Alistarh, Ce Zhang, Ji Liu

Most of today's distributed machine learning systems assume {\em reliable networks}: whenever two machines exchange information (e. g., gradients or models), the network should guarantee the delivery of the message.

BIG-bench Machine Learning

$D^2$: Decentralized Training over Decentralized Data

no code implementations ICML 2018 Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, Ji Liu

While training a machine learning model using multiple workers, each of which collects data from its own data source, it would be useful when the data collected from different workers are unique and different.

Image Classification Multi-view Subspace Clustering

Using transfer learning to detect galaxy mergers

no code implementations25 May 2018 Sandro Ackermann, Kevin Schawinski, Ce Zhang, Anna K. Weigel, M. Dennis Turp

We investigate the use of deep convolutional neural networks (deep CNNs) for automatic visual detection of galaxy mergers.

General Classification Test +1

ETH-DS3Lab at SemEval-2018 Task 7: Effectively Combining Recurrent and Convolutional Neural Networks for Relation Classification and Extraction

no code implementations SEMEVAL 2018 Jonathan Rotsztejn, Nora Hollenstein, Ce Zhang

Reliably detecting relevant relations between entities in unstructured text is a valuable resource for knowledge extraction, which is why it has awaken significant interest in the field of Natural Language Processing.

General Classification Relation Classification

PSFGAN: a generative adversarial network system for separating quasar point sources and host galaxy light

no code implementations23 Mar 2018 Dominic Stark, Barthelemy Launet, Kevin Schawinski, Ce Zhang, Michael Koss, M. Dennis Turp, Lia F. Sartori, Hantian Zhang, Yiru Chen, Anna K. Weigel

We test the method using Sloan Digital Sky Survey (SDSS) r-band images with artificial AGN point sources added which are then removed using the GAN and with parametric methods using GALFIT.

Astrophysics of Galaxies Data Analysis, Statistics and Probability

D$^2$: Decentralized Training over Decentralized Data

no code implementations19 Mar 2018 Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, Ji Liu

While training a machine learning model using multiple workers, each of which collects data from their own data sources, it would be most useful when the data collected from different workers can be {\em unique} and {\em different}.

Image Classification

Communication Compression for Decentralized Training

no code implementations NeurIPS 2018 Hanlin Tang, Shaoduo Gan, Ce Zhang, Tong Zhang, Ji Liu

In this paper, We explore a natural question: {\em can the combination of both techniques lead to a system that is robust to both bandwidth and latency?}

AutoML from Service Provider's Perspective: Multi-device, Multi-tenant Model Selection with GP-EI

no code implementations17 Mar 2018 Chen Yu, Bojan Karlas, Jie Zhong, Ce Zhang, Ji Liu

In this paper, we focus on the AutoML problem from the \emph{service provider's perspective}, motivated by the following practical consideration: When an AutoML service needs to serve {\em multiple users} with {\em multiple devices} at the same time, how can we allocate these devices to users in an efficient way?

AutoML Model Selection

DataBright: Towards a Global Exchange for Decentralized Data Ownership and Trusted Computation

1 code implementation13 Feb 2018 David Dao, Dan Alistarh, Claudiu Musat, Ce Zhang

We illustrate that trusted computation can enable the creation of an AI market, where each data point has an exact value that should be paid to its creator.

BIG-bench Machine Learning

Asynchronous Decentralized Parallel Stochastic Gradient Descent

2 code implementations ICML 2018 Xiangru Lian, Wei zhang, Ce Zhang, Ji Liu

Can we design an algorithm that is robust in a heterogeneous environment, while being communication efficient and maintaining the best-possible convergence rate? Towards Multi-tenant Resource Sharing for Machine Learning Workloads

no code implementations24 Aug 2017 Tian Li, Jie Zhong, Ji Liu, Wentao Wu, Ce Zhang

We ask, as a "service provider" that manages a shared cluster of machines among all our users running machine learning workloads, what is the resource allocation strategy that maximizes the global satisfaction of all our users?

Bayesian Optimization BIG-bench Machine Learning +4

ZipML: Training Linear Models with End-to-End Low Precision, and a Little Bit of Deep Learning

no code implementations ICML 2017 Hantian Zhang, Jerry Li, Kaan Kara, Dan Alistarh, Ji Liu, Ce Zhang

We examine training at reduced precision, both from a theoretical and practical perspective, and ask: is it possible to train models at end-to-end low precision with provable guarantees?


MLBench: How Good Are Machine Learning Clouds for Binary Classification Tasks on Structured Data?

no code implementations29 Jul 2017 Yu Liu, Hantian Zhang, Luyuan Zeng, Wentao Wu, Ce Zhang

We then compare the performance of the top winning code available from Kaggle with that of running machine learning clouds from both Azure and Amazon on mlbench.

BIG-bench Machine Learning Binary Classification +1

Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent

2 code implementations NeurIPS 2017 Xiangru Lian, Ce Zhang, huan zhang, Cho-Jui Hsieh, Wei zhang, Ji Liu

On network configurations with low bandwidth or high latency, D-PSGD can be up to one order of magnitude faster than its well-optimized centralized counterparts.