no code implementations • 18 May 2018 • Quanshi Zhang, Yu Yang, Yuchen Liu, Ying Nian Wu, Song-Chun Zhu
Given feature maps of a certain conv-layer of the CNN, the explainer performs like an auto-encoder, which first disentangles the feature maps into object-part features and then inverts object-part features back to features of higher conv-layers of the CNN.
no code implementations • 14 May 2018 • Tian Han, Jiawen Wu, Ying Nian Wu
A recent Cell paper [Chang and Tsao, 2017] reports an interesting discovery.
no code implementations • 26 Apr 2018 • Quanshi Zhang, Yu Yang, Qian Yu, Ying Nian Wu
This paper focuses on a new task, i. e., transplanting a category-and-task-specific neural network to a generic, modular network without strong supervision.
no code implementations • CVPR 2018 • Ruiqi Gao, Yang Lu, Junpei Zhou, Song-Chun Zhu, Ying Nian Wu
Within each iteration of our learning algorithm, for each observed training image, we generate synthesized images at multiple grids by initializing the finite-step MCMC sampling from a minimal 1 x 1 version of the training image.
no code implementations • CVPR 2019 • Quanshi Zhang, Yu Yang, Haotian Ma, Ying Nian Wu
We propose to learn a decision tree, which clarifies the specific reason for each prediction made by the CNN at the semantic level.
no code implementations • 5 Aug 2017 • Quanshi Zhang, Ruiming Cao, Shengming Zhang, Mark Redmonds, Ying Nian Wu, Song-Chun Zhu
In the scenario of one/multi-shot learning, conventional end-to-end learning strategies without sufficient supervision are usually not powerful enough to learn correct patterns from noisy signals.
no code implementations • 5 Aug 2017 • Quanshi Zhang, Ruiming Cao, Feng Shi, Ying Nian Wu, Song-Chun Zhu
Considering that each filter in a conv-layer of a pre-trained CNN usually represents a mixture of object parts, we propose a simple yet efficient method to automatically disentangles different part patterns from each filter, and construct an explanatory graph.
no code implementations • 13 Aug 2017 • Quanshi Zhang, Ying Nian Wu, Hao Zhang, Song-Chun Zhu
The loss is defined for nodes in all layers of the AOG, including the generative loss (measuring the likelihood of the images) and the discriminative loss (measuring the fitness to human answers).
no code implementations • CVPR 2017 • Jianwen Xie, Song-Chun Zhu, Ying Nian Wu
We show that a spatial-temporal generative ConvNet can be used to model and synthesize dynamic patterns.
no code implementations • CVPR 2017 • Quanshi Zhang, Ruiming Cao, Ying Nian Wu, Song-Chun Zhu
We use an active human-computer communication to incrementally grow such an AOG on the pre-trained CNN as follows.
no code implementations • 14 Nov 2016 • Quanshi Zhang, Ruiming Cao, Ying Nian Wu, Song-Chun Zhu
This paper proposes a learning strategy that extracts object-part concepts from a pre-trained convolutional neural network (CNN), in an attempt to 1) explore explicit semantics hidden in CNN units and 2) gradually grow a semantically interpretable graphical model on the pre-trained CNN for hierarchical object understanding.
no code implementations • 29 Sep 2016 • Jianwen Xie, Yang Lu, Ruiqi Gao, Song-Chun Zhu, Ying Nian Wu
Specifically, within each iteration of the cooperative learning algorithm, the generator model generates initial synthesized examples to initialize a finite-step MCMC that samples and trains the energy-based descriptor model.
no code implementations • 28 Jun 2016 • Tian Han, Yang Lu, Song-Chun Zhu, Ying Nian Wu
This paper proposes an alternating back-propagation algorithm for learning the generator network model.
no code implementations • 1 Jul 2016 • Jianwen Xie, Pamela K. Douglas, Ying Nian Wu, Arthur L. Brody, Ariana E. Anderson
Spatial sparse coding algorithms ($L1$ Regularized Learning and K-SVD) would impose local specialization and a discouragement of multitasking, where the total observed activity in a single voxel originates from a restricted number of possible brain networks.
no code implementations • 10 Feb 2016 • Jianwen Xie, Yang Lu, Song-Chun Zhu, Ying Nian Wu
If we further assume that the non-linearity in the ConvNet is Rectified Linear Unit (ReLU) and the reference distribution is Gaussian white noise, then we obtain a generative ConvNet model that is unique among energy-based models: The model is piecewise Gaussian, and the means of the Gaussian pieces are defined by an auto-encoder, where the filters in the bottom-up encoding become the basis functions in the top-down decoding, and the binary activation variables detected by the filters in the bottom-up convolution process become the coefficients of the basis functions in the top-down deconvolution process.
no code implementations • 28 Sep 2015 • Yang Lu, Song-Chun Zhu, Ying Nian Wu
We explain that each learned model corresponds to a new CNN unit at a layer above the layer of filters employed by the model.
no code implementations • 2 May 2013 • Adrian Barbu, Tianfu Wu, Ying Nian Wu
Each template is a binary vector, and a template generates examples by randomly switching its binary components independently with a certain probability.
no code implementations • 1 Oct 2018 • Tianmin Shu, Caiming Xiong, Ying Nian Wu, Song-Chun Zhu
In particular, the probing agent (i. e. a learner) learns to interact with the environment and with a target agent (i. e., a demonstrator) to maximize the change in the observed behaviors of that agent.
no code implementations • 9 Oct 2018 • Ying Nian Wu, Ruiqi Gao, Tian Han, Song-Chun Zhu
In this paper, we review three families of probability models, namely, the discriminative models, the descriptive models, and the generative models.
no code implementations • 18 Dec 2018 • Quanshi Zhang, Xin Wang, Ruiming Cao, Ying Nian Wu, Feng Shi, Song-Chun Zhu
This paper introduces a graphical model, namely an explanatory graph, which reveals the knowledge hierarchy hidden inside conv-layers of a pre-trained CNN.
no code implementations • 18 Dec 2018 • Quanshi Zhang, Ruiming Cao, Ying Nian Wu, Song-Chun Zhu
The AOG associates each object part with certain neural units in feature maps of conv-layers.
no code implementations • ICCV 2015 • Quanshi Zhang, Ying Nian Wu, Song-Chun Zhu
This paper reformulates the theory of graph mining on the technical basis of graph matching, and extends its scope of applications to computer vision.
no code implementations • 27 Dec 2018 • Jianwen Xie, Ruiqi Gao, Zilong Zheng, Song-Chun Zhu, Ying Nian Wu
The non-linear transformation of this transition model can be parametrized by a feedforward neural network.
no code implementations • 8 Jan 2019 • Quanshi Zhang, Xin Wang, Ying Nian Wu, Huilin Zhou, Song-Chun Zhu
This paper proposes a generic method to learn interpretable convolutional filters in a deep convolutional neural network (CNN) for object classification, where each interpretable filter encodes features of a specific object part.
no code implementations • 21 Jan 2019 • Quanshi Zhang, Yu Yang, Qian Yu, Ying Nian Wu
This paper focuses on a new task, i. e., transplanting a category-and-task-specific neural network to a generic, modular network without strong supervision.
no code implementations • 21 Jan 2019 • Quanshi Zhang, Yu Yang, Ying Nian Wu
This paper presents an unsupervised method to learn a neural network, namely an explainer, to interpret a pre-trained convolutional neural network (CNN), i. e., the explainer uses interpretable visual concepts to explain features in middle conv-layers of a CNN.
no code implementations • CVPR 2014 • Jianwen Xie, Wenze Hu, Song-Chun Zhu, Ying Nian Wu
We investigate an inhomogeneous version of the FRAME (Filters, Random field, And Maximum Entropy) model and apply it to modeling object patterns.
no code implementations • CVPR 2014 • Jifeng Dai, Yi Hong, Wenze Hu, Song-Chun Zhu, Ying Nian Wu
Given a set of unannotated training images, a dictionary of such hierarchical templates are learned so that each training image can be represented by a small number of templates that are spatially translated, rotated and scaled versions of the templates in the learned dictionary.
no code implementations • CVPR 2017 • Jianwen Xie, Yifei Xu, Erik Nijkamp, Ying Nian Wu, Song-Chun Zhu
This paper proposes a method for generative learning of hierarchical random field models.
no code implementations • 20 Jan 2019 • Xianglei Xing, Song-Chun Zhu, Ying Nian Wu
We introduce an explainable generative model by applying sparse operation on the feature maps of the generator network.
no code implementations • 7 Feb 2019 • Jianwen Xie, Zilong Zheng, Xiaolin Fang, Song-Chun Zhu, Ying Nian Wu
This paper studies the problem of learning the conditional distribution of a high-dimensional output given an input, where the output and input may belong to two different domains, e. g., the output is a photo image and the input is a sketch image.
no code implementations • 24 Jan 2019 • Ruiqi Gao, Jianwen Xie, Siyuan Huang, Yufan Ren, Song-Chun Zhu, Ying Nian Wu
This paper proposes a representational model for image pairs such as consecutive video frames that are related by local pixel displacements, in the hope that the model may shed light on motion perception in primary visual cortex (V1).
no code implementations • 10 Apr 2019 • Yifei Xu, Jianwen Xie, Tianyang Zhao, Chris Baker, Yibiao Zhao, Ying Nian Wu
The problem of continuous inverse optimal control (over finite time horizon) is to learn the unknown cost function over the sequence of continuous control variables from expert demonstrations.
no code implementations • NeurIPS 2019 • Erik Nijkamp, Mitch Hill, Song-Chun Zhu, Ying Nian Wu
We treat this non-convergent short-run MCMC as a learned generator model or a flow model.
no code implementations • CVPR 2020 • Xianglei Xing, Tianfu Wu, Song-Chun Zhu, Ying Nian Wu
To realize this AND-OR hierarchy in image synthesis, we learn a generator network that consists of the following two components: (i) Each layer of the hierarchy is represented by an over-complete set of convolutional basis functions.
no code implementations • 26 Sep 2019 • Jianwen Xie, Song-Chun Zhu, Ying Nian Wu
We show that an energy-based spatial-temporal generative ConvNet can be used to model and synthesize dynamic patterns.
no code implementations • 19 Nov 2019 • Dandan Zhu, Tian Han, Linqi Zhou, Xiaokang Yang, Ying Nian Wu
We propose the clustered generator model for clustering which contains both continuous and discrete latent variables.
no code implementations • 26 Nov 2019 • Jianwen Xie, Ruiqi Gao, Erik Nijkamp, Song-Chun Zhu, Ying Nian Wu
Learning representations of data is an important problem in statistics and machine learning.
no code implementations • 26 Nov 2019 • Jianwen Xie, Ruiqi Gao, Zilong Zheng, Song-Chun Zhu, Ying Nian Wu
To model the motions explicitly, it is natural for the model to be based on the motions or the displacement fields of the pixels.
no code implementations • ECCV 2020 • Erik Nijkamp, Bo Pang, Tian Han, Linqi Zhou, Song-Chun Zhu, Ying Nian Wu
Learning such a generative model requires inferring the latent variables for each training example based on the posterior distribution of these latent variables.
no code implementations • 20 Apr 2020 • Yixin Zhu, Tao Gao, Lifeng Fan, Siyuan Huang, Mark Edmonds, Hangxin Liu, Feng Gao, Chi Zhang, Siyuan Qi, Ying Nian Wu, Joshua B. Tenenbaum, Song-Chun Zhu
We demonstrate the power of this perspective to develop cognitive AI systems with humanlike common sense by showing how to observe and apply FPICU with little training data to solve a wide range of challenging tasks, including tool use, planning, utility inference, and social learning.
no code implementations • CVPR 2020 • Tian Han, Erik Nijkamp, Linqi Zhou, Bo Pang, Song-Chun Zhu, Ying Nian Wu
This paper proposes a joint training method to learn both the variational auto-encoder (VAE) and the latent energy-based model (EBM).
no code implementations • 12 Jun 2020 • Erik Nijkamp, Ruiqi Gao, Pavel Sountsov, Srinivas Vasudevan, Bo Pang, Song-Chun Zhu, Ying Nian Wu
Learning energy-based model (EBM) requires MCMC sampling of the learned model as an inner loop of the learning algorithm.
no code implementations • 1 Jan 2021 • Chi Zhang, Sirui Xie, Baoxiong Jia, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu
We further show that the algebraic representation learned can be decoded by isomorphism and used to generate an answer.
no code implementations • NeurIPS Workshop ICBINB 2020 • Bo Pang, Erik Nijkamp, Jiali Cui, Tian Han, Ying Nian Wu
This paper proposes a latent space energy-based prior model for semi-supervised learning.
no code implementations • 19 Oct 2020 • Bo Pang, Tian Han, Ying Nian Wu
Deep generative models have recently been applied to molecule design.
no code implementations • 25 Dec 2020 • Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, Ying Nian Wu
3D data that contains rich geometry information of objects and scenes is valuable for understanding 3D physical world.
no code implementations • 22 Feb 2021 • Sirui Xie, Xiaojian Ma, Peiyu Yu, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu
Leveraging these concepts, they could understand the internal structure of this task, without seeing all of the problem instances.
no code implementations • 2 Mar 2021 • Qing Li, Siyuan Huang, Yining Hong, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu
We believe the HINT dataset and the experimental findings are of great interest to the learning community on systematic generalization.
no code implementations • 7 Mar 2021 • Jianwen Xie, Zilong Zheng, Xiaolin Fang, Song-Chun Zhu, Ying Nian Wu
This paper studies the unsupervised cross-domain translation problem by proposing a generative framework, in which the probability distribution of each domain is represented by a generative cooperative network that consists of an energy-based model and a latent variable model.
no code implementations • ACL 2021 • Liang Qiu, Yuan Liang, Yizhou Zhao, Pan Lu, Baolin Peng, Zhou Yu, Ying Nian Wu, Song-Chun Zhu
Inferring social relations from dialogues is vital for building emotionally intelligent robots to interpret human language better and act accordingly.
Ranked #5 on Dialog Relation Extraction on DialogRE
no code implementations • NAACL 2021 • Erik Nijkamp, Bo Pang, Ying Nian Wu, Caiming Xiong
We introduce Self-CRItic Pretraining Transformers (SCRIPT) for representation learning of text.
no code implementations • 16 Jul 2021 • Quanshi Zhang, Tian Han, Lixin Fan, Zhanxing Zhu, Hang Su, Ying Nian Wu, Jie Ren, Hao Zhang
This workshop pays a special interest in theoretic foundations, limitations, and new application trends in the scope of XAI.
no code implementations • ICLR 2022 • Erik Nijkamp, Ruiqi Gao, Pavel Sountsov, Srinivas Vasudevan, Bo Pang, Song-Chun Zhu, Ying Nian Wu
However, MCMC sampling of EBMs in high-dimensional data space is generally not mixing, because the energy function, which is usually parametrized by deep network, is highly multi-modal in the data space.
no code implementations • 25 Sep 2019 • Yangzi Guo, Yiyuan She, Ying Nian Wu, Adrian Barbu
However, in non-vision sparse datasets, especially with many irrelevant features where a standard neural network would overfit, this might not be the case and there might be many non-equivalent local optima.
no code implementations • 28 Sep 2020 • Ruiqi Gao, Jianwen Xie, Xue-Xin Wei, Song-Chun Zhu, Ying Nian Wu
The grid cells in the mammalian medial entorhinal cortex exhibit striking hexagon firing patterns when the agent navigates in the open field.
no code implementations • NeurIPS Workshop DL-IG 2020 • Tian Han, Jun Zhang, Ying Nian Wu
This paper reviews the em-projections in information geometry and the recent understanding of variational auto-encoder, and explains that they share a common formulation as joint minimization of the Kullback-Leibler divergence between two manifolds of probability distributions, and the joint minimization can be implemented by alternating projections or alternating gradient descent.
no code implementations • 25 Nov 2021 • Chi Zhang, Sirui Xie, Baoxiong Jia, Ying Nian Wu, Song-Chun Zhu, Yixin Zhu
Extensive experiments show that by incorporating an algebraic treatment, the ALANS learner outperforms various pure connectionist models in domains requiring systematic generalization.
no code implementations • 14 Jan 2022 • Feng Gao, Qing Ping, Govind Thattai, Aishwarya Reganti, Ying Nian Wu, Prem Natarajan
Outside-knowledge visual question answering (OK-VQA) requires the agent to comprehend the image, make use of relevant knowledge from the entire web, and digest all the information to answer the question.
no code implementations • CVPR 2022 • Feng Gao, Qing Ping, Govind Thattai, Aishwarya Reganti, Ying Nian Wu, Prem Natarajan
Most previous works address the problem by first fusing the image and question in the multi-modal space, which is inflexible for further fusion with a vast amount of external knowledge.
Ranked #19 on Visual Question Answering (VQA) on OK-VQA
no code implementations • 4 Oct 2022 • Qing Li, Yixin Zhu, Yitao Liang, Ying Nian Wu, Song-Chun Zhu, Siyuan Huang
We evaluate NSR's efficacy across four challenging benchmarks designed to probe systematic generalization capabilities: SCAN for semantic parsing, PCFG for string manipulation, HINT for arithmetic reasoning, and a compositional machine translation task.
no code implementations • 5th Workshop on Meta-Learning at NeurIPS 2021 2021 • Deqian Kong, Bo Pang, Ying Nian Wu
We propose to learn an energy-based model (EBM) in the latent space of a top-down generative model such that the EBM in the low dimensional latent space is able to be learned efficiently and adapt to each task rapidly.
no code implementations • 19 May 2023 • Qin Zhang, Dongsheng An, Tianjun Xiao, Tong He, Qingming Tang, Ying Nian Wu, Joseph Tighe, Yifan Xing, Stefano Soatto
In deep metric learning for visual recognition, the calibration of distance thresholds is crucial for achieving desired model performance in the true positive rates (TPR) or true negative rates (TNR).
no code implementations • CVPR 2023 • Jiali Cui, Ying Nian Wu, Tian Han
To tackle this issue and learn more expressive prior models, we propose an energy-based model (EBM) on the joint latent space over all layers of latent variables with the multi-layer generator as its backbone.
no code implementations • 8 Jul 2023 • Qin Zhang, Linghan Xu, Qingming Tang, Jun Fang, Ying Nian Wu, Joe Tighe, Yifan Xing
Existing losses used in deep metric learning (DML) for image retrieval often lead to highly non-uniform intra-class and inter-class representation structures across test classes and data distributions.
no code implementations • 5 Oct 2023 • Yilue Qian, Peiyu Yu, Ying Nian Wu, Yao Su, Wei Wang, Lifeng Fan
In this paper, we propose an interpretable and generalizable visual planning framework consisting of i) a novel Substitution-based Concept Learner (SCL) that abstracts visual inputs into disentangled concept representations, ii) symbol abstraction and reasoning that performs task planning via the self-learned symbols, and iii) a Visual Causal Transition model (ViCT) that grounds visual causal transitions to semantically similar real-world actions.
no code implementations • 5 Oct 2023 • Deqian Kong, Yuhao Huang, Jianwen Xie, Ying Nian Wu
This paper proposes a latent prompt Transformer model for solving challenging optimization problems such as molecule design, where the goal is to find molecules with optimal values of a target chemical or biological property that can be computed by an existing software.
no code implementations • ICCV 2023 • Jiali Cui, Ying Nian Wu, Tian Han
In this paper, we propose a joint latent space EBM prior model with multi-layer latent variables for effective hierarchical representation learning.
no code implementations • 29 Oct 2023 • Dehong Xu, Ruiqi Gao, Wen-Hao Zhang, Xue-Xin Wei, Ying Nian Wu
As the agent moves, the vector is transformed by an RNN that takes the velocity of the agent as input.
no code implementations • 18 Jan 2024 • Cheng Han, James C. Liang, Qifan Wang, Majid Rabbani, Sohail Dianat, Raghuveer Rao, Ying Nian Wu, Dongfang Liu
We introduce the novel Diffusion Visual Programmer (DVP), a neuro-symbolic image translation framework.
no code implementations • 7 Feb 2024 • Deqian Kong, Dehong Xu, Minglu Zhao, Bo Pang, Jianwen Xie, Andrew Lizarraga, Yuhao Huang, Sirui Xie, Ying Nian Wu
We introduce the Latent Plan Transformer (LPT), a novel model that leverages a latent space to connect a Transformer-based trajectory generator and the final return.
no code implementations • 27 Feb 2024 • Deqian Kong, Yuhao Huang, Jianwen Xie, Edouardo Honig, Ming Xu, Shuanghong Xue, Pei Lin, Sanping Zhou, Sheng Zhong, Nanning Zheng, Ying Nian Wu
Designing molecules with desirable properties, such as drug-likeliness and high binding affinities towards protein targets, is a challenging problem.
no code implementations • 10 Apr 2024 • Yasi Zhang, Peiyu Yu, Ying Nian Wu
Text-to-image diffusion models have shown great success in generating high-quality text-guided images.
1 code implementation • EACL 2021 • Bo Pang, Erik Nijkamp, Tian Han, Ying Nian Wu
It is initialized from the prior distribution of the latent variable and then runs a small number (e. g., 20) of Langevin dynamics steps guided by its posterior distribution.
1 code implementation • 1 Sep 2019 • Zijun Zhang, Linqi Zhou, Liangke Gou, Ying Nian Wu
We report a neural architecture search framework, BioNAS, that is tailored for biomedical researchers to easily build, evaluate, and uncover novel knowledge from interpretable deep learning models.
1 code implementation • NeurIPS 2021 • Luyao Yuan, Dongruo Zhou, Junhong Shen, Jingdong Gao, Jeffrey L. Chen, Quanquan Gu, Ying Nian Wu, Song-Chun Zhu
Recently, the benefits of integrating this cooperative pedagogy into machine concept learning in discrete spaces have been proved by multiple works.
1 code implementation • 10 Nov 2023 • Andrew Lizarraga, Brandon Taraku, Edouardo Honig, Ying Nian Wu, Shantanu H. Joshi
Given the complex geometry of white matter streamlines, Autoencoders have been proposed as a dimension-reduction tool to simplify the analysis streamlines in a low-dimensional latent spaces.
1 code implementation • 12 Feb 2024 • Huixin Zhan, Ying Nian Wu, Zijun Zhang
Impressively, applying these adapters on natural language foundation models matched or even exceeded the performance of DNA foundation models.
1 code implementation • 20 Nov 2022 • Yu-Zhe Shi, Manjie Xu, John E. Hopcroft, Kun He, Joshua B. Tenenbaum, Song-Chun Zhu, Ying Nian Wu, Wenjuan Han, Yixin Zhu
Specifically, at the $representational \ level$, we seek to answer how the complexity varies when a visual concept is mapped to the representation space.
1 code implementation • 14 Jun 2021 • Yifei Xu, Jingqiao Zhang, Ru He, Liangzhu Ge, Chao Yang, Cheng Yang, Ying Nian Wu
In this paper, we propose a self-augmentation strategy (SAS) where a single network is utilized for both regular pre-training and contextualized data augmentation for the training in later epochs.
1 code implementation • 6 Oct 2022 • Dehong Xu, Ruiqi Gao, Wen-Hao Zhang, Xue-Xin Wei, Ying Nian Wu
Recurrent neural networks have been proposed to explain the properties of the grid cells by updating the neural activity vector based on the velocity input of the animal.
1 code implementation • 1 Jun 2023 • Yan Xu, Deqian Kong, Dehong Xu, Ziwei Ji, Bo Pang, Pascale Fung, Ying Nian Wu
The capability to generate responses with diversity and faithfulness using factual knowledge is paramount for creating a human-like, trustworthy dialogue system.
1 code implementation • 26 Aug 2021 • Bo Pang, Ying Nian Wu
The energy term of the prior model couples a continuous latent vector and a symbolic one-hot vector, so that discrete category can be inferred from the observed example based on the continuous latent vector.
1 code implementation • 9 Jun 2023 • Deqian Kong, Bo Pang, Tian Han, Ying Nian Wu
To search for molecules with desired properties, we propose a sampling with gradual distribution shifting (SGDS) algorithm, so that after learning the model initially on the training data of existing molecules and their properties, the proposed algorithm gradually shifts the model distribution towards the region supported by molecules with desired values of properties.
1 code implementation • 28 Dec 2018 • Tian Han, Erik Nijkamp, Xiaolin Fang, Mitch Hill, Song-Chun Zhu, Ying Nian Wu
This paper proposes the divergence triangle as a framework for joint training of generator model, energy-based model and inference model.
1 code implementation • 26 Mar 2021 • Xu Xie, Chi Zhang, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu
Predicting agents' future trajectories plays a crucial role in modern AI systems, yet it is challenging due to intricate interactions exhibited in multi-agent systems, especially when it comes to collision avoidance.
1 code implementation • CVPR 2021 • Yaxuan Zhu, Ruiqi Gao, Siyuan Huang, Song-Chun Zhu, Ying Nian Wu
Specifically, the camera pose and 3D scene are represented as vectors and the local camera movement is represented as a matrix operating on the vector of the camera pose.
1 code implementation • 18 Mar 2024 • Shu Wang, Muzhi Han, Ziyuan Jiao, Zeyu Zhang, Ying Nian Wu, Song-Chun Zhu, Hangxin Liu
Through a series of simulations in a box-packing domain, we quantitatively demonstrate the effectiveness of LLM^3 in solving TAMP problems and the efficiency in selecting action parameters.
2 code implementations • 16 Jun 2018 • Xianglei Xing, Ruiqi Gao, Tian Han, Song-Chun Zhu, Ying Nian Wu
We present a deformable generator model to disentangle the appearance and geometric information for both image and video data in a purely unsupervised manner.
1 code implementation • ICLR 2019 • Ruiqi Gao, Jianwen Xie, Song-Chun Zhu, Ying Nian Wu
In this model, the 2D self-position of the agent is represented by a high-dimensional vector, and the 2D self-motion or displacement of the agent is represented by a matrix that transforms the vector.
1 code implementation • CVPR 2021 • Bo Pang, Tianyang Zhao, Xu Xie, Ying Nian Wu
Sampling from or optimizing the learned LB-EBM yields a belief vector which is used to make a path plan, which then in turn helps to predict a long-range trajectory.
1 code implementation • 22 Oct 2022 • Cristian Challu, Peihong Jiang, Ying Nian Wu, Laurent Callot
In this work, we tackle two widespread challenges in real applications for time-series forecasting that have been largely understudied: distribution shifts and missing data.
1 code implementation • 15 Feb 2022 • Cristian Challu, Peihong Jiang, Ying Nian Wu, Laurent Callot
Multivariate time series anomaly detection has become an active area of research in recent years, with Deep Learning models outperforming previous approaches on benchmark datasets.
1 code implementation • CVPR 2021 • Jianwen Xie, Yifei Xu, Zilong Zheng, Song-Chun Zhu, Ying Nian Wu
We propose a generative model of unordered point sets, such as point clouds, in the form of an energy-based model, where the energy function is parameterized by an input-permutation-invariant bottom-up neural network.
1 code implementation • NeurIPS 2020 • Bo Pang, Tian Han, Erik Nijkamp, Song-Chun Zhu, Ying Nian Wu
Due to the low dimensionality of the latent space and the expressiveness of the top-down network, a simple EBM in latent space can capture regularities in the data effectively, and MCMC sampling in latent space is efficient and mixes well.
1 code implementation • CVPR 2018 • Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, Ying Nian Wu
This paper proposes a 3D shape descriptor network, which is a deep convolutional energy-based model, for modeling volumetric shape patterns.
2 code implementations • 29 Mar 2019 • Erik Nijkamp, Mitch Hill, Tian Han, Song-Chun Zhu, Ying Nian Wu
On the other hand, ConvNet potentials learned with non-convergent MCMC do not have a valid steady-state and cannot be considered approximate unnormalized densities of the training data because long-run MCMC samples differ greatly from observed images.
2 code implementations • NeurIPS 2021 • Peiyu Yu, Sirui Xie, Xiaojian Ma, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu
Foreground extraction can be viewed as a special case of generic image segmentation that focuses on identifying and disentangling objects from the background.
1 code implementation • NeurIPS 2021 • Ruiqi Gao, Jianwen Xie, Xue-Xin Wei, Song-Chun Zhu, Ying Nian Wu
In this paper, we conduct theoretical analysis of a general representation model of path integration by grid cells, where the 2D self-position is encoded as a higher dimensional vector, and the 2D self-motion is represented by a general transformation of the vector.
2 code implementations • ICLR 2021 • Ruiqi Gao, Yang song, Ben Poole, Ying Nian Wu, Diederik P. Kingma
Inspired by recent progress on diffusion probabilistic models, we present a diffusion recovery likelihood method to tractably learn and sample from a sequence of EBMs trained on increasingly noisy versions of a dataset.
Ranked #18 on Image Generation on CelebA 64x64
1 code implementation • ICML 2020 • Qing Li, Siyuan Huang, Yining Hong, Yixin Chen, Ying Nian Wu, Song-Chun Zhu
In this paper, we address these issues and close the loop of neural-symbolic learning by (1) introducing the \textbf{grammar} model as a \textit{symbolic prior} to bridge neural perception and symbolic reasoning, and (2) proposing a novel \textbf{back-search} algorithm which mimics the top-down human-like learning procedure to propagate the error through the symbolic reasoning module efficiently.
1 code implementation • CVPR 2019 • Tianyang Zhao, Yifei Xu, Mathew Monfort, Wongun Choi, Chris Baker, Yibiao Zhao, Yizhou Wang, Ying Nian Wu
Specifically, the model encodes multiple agents' past trajectories and the scene context into a Multi-Agent Tensor, then applies convolutional fusion to capture multiagent interactions while retaining the spatial structure of agents and the scene context.
2 code implementations • 13 Jun 2022 • Peiyu Yu, Sirui Xie, Xiaojian Ma, Baoxiong Jia, Bo Pang, Ruiqi Gao, Yixin Zhu, Song-Chun Zhu, Ying Nian Wu
Latent space Energy-Based Models (EBMs), also known as energy-based priors, have drawn growing interests in generative modeling.
2 code implementations • CVPR 2020 • Ruiqi Gao, Erik Nijkamp, Diederik P. Kingma, Zhen Xu, Andrew M. Dai, Ying Nian Wu
(2) The update of the flow model approximately minimizes the Jensen-Shannon divergence between the flow model and the data distribution.
1 code implementation • NeurIPS 2018 • Siyuan Huang, Siyuan Qi, Yinxue Xiao, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu
Holistic 3D indoor scene understanding refers to jointly recovering the i) object bounding boxes, ii) room layout, and iii) camera pose, all in 3D.
Ranked #5 on Monocular 3D Object Detection on SUN RGB-D
2 code implementations • CVPR 2018 • Quanshi Zhang, Ying Nian Wu, Song-Chun Zhu
Instead, the interpretable CNN automatically assigns each filter in a high conv-layer with an object part during the learning process.
Ranked #1 on single catogory classification on ILSVRC Part
1 code implementation • NeurIPS 2023 • Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Jianfeng Gao
At the heart of Chameleon is an LLM-based planner that assembles a sequence of tools to execute to generate the final response.
2 code implementations • 29 Sep 2022 • Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, Ashwin Kalyan
However, it is unknown if the models can handle more complex problems that involve math reasoning over heterogeneous information, such as tabular data.