Search Results for author:
Found 76 papers, 33 papers with code
Where and when to look? Spatial-temporal attention for action recognition in videos
Our model is efficient, as it proposes a separable spatio-temporal mechanism for video attention, while being able to identify important parts of the video both spatially and temporally.
Action Recognition In Videos
Temporal Action Localization
+1
Tele-FLM Technical Report
Large language models (LLMs) have showcased profound capabilities in language understanding and generation, facilitating a wide array of applications.
Advances and Open Challenges in Federated Learning with Foundation Models
The integration of Foundation Models (FMs) with Federated Learning (FL) presents a transformative paradigm in Artificial Intelligence (AI), offering enhanced capabilities while addressing concerns of privacy, data decentralization, and computational efficiency.
M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models
Additionally, we propose M3D-LaMed, a versatile multi-modal large language model for 3D medical image analysis.
SynArtifact: Classifying and Alleviating Artifacts in Synthetic Images via Vision-Language Model
To alleviate artifacts and improve quality of synthetic images, we fine-tune Vision-Language Model (VLM) as artifact classifier to automatically identify and classify a wide range of artifacts and provide supervision for further optimizing generative models.
Pushing Auto-regressive Models for 3D Shape Generation at Capacity and Scalability
In this paper, we extend auto-regressive models to 3D domains, and seek a stronger ability of 3D shape generation by improving auto-regressive models at capacity and scalability simultaneously.
Efficient Multimodal Learning from Data-centric Perspective
Multimodal Large Language Models (MLLMs) have demonstrated notable capabilities in general visual understanding and reasoning tasks.
RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model
Recent advancements in Multi-Modal Large Language models (MLLMs) have shown promising potential in enhancing the explainability as a driving agent by producing control predictions along with natural language explanations.
Spin: An Efficient Secure Computation Framework with GPU Acceleration
Accuracy and efficiency remain challenges for multi-party computation (MPC) frameworks.
Learning Position-Aware Implicit Neural Network for Real-World Face Inpainting
Face inpainting requires the model to have a precise global understanding of the facial position structure.
Tenplex: Dynamic Parallelism for Deep Learning using Parallelizable Tensor Collections
After a GPU change, Scalai uses the PTC to transform the job state: the PTC repartitions the dataset state under data parallelism and exposes it to DL workers through a virtual file system; and the PTC obtains the model state as partitioned checkpoints and transforms them to reflect the new parallelization configuration.
Open-DDVM: A Reproduction and Extension of Diffusion Model for Optical Flow Estimation
Recently, Google proposes DDVM which for the first time demonstrates that a general diffusion model for image-to-image translation task works impressively well on optical flow estimation task without any specific designs like RAFT.
SegVol: Universal and Interactive Volumetric Medical Image Segmentation
Precise image segmentation provides clinical study with instructive information.
A global product of fine-scale urban building height based on spaceborne lidar
Our results revealed that the estimated method of building height samples based on the GEDI data was effective with 0. 78 of Pearson's r and 3. 67 m of RMSE in comparison to the reference data.
Real-Fake: Effective Training Data Synthesis Through Distribution Matching
Synthetic training data has gained prominence in numerous learning tasks and scenarios, offering advantages such as dataset augmentation, generalization evaluation, and privacy preservation.
Image Captions are Natural Prompts for Text-to-Image Models
With the rapid development of Artificial Intelligence Generated Content (AIGC), it has become common practice in many learning tasks to train or fine-tune large models on synthetic data due to the data-scarcity and privacy leakage problems.
SVIT: Scaling up Visual Instruction Tuning
Thanks to the emerging of foundation models, the large language and vision models are integrated to acquire the multimodal ability of visual captioning, question answering, etc.
Federated Generative Learning with Foundation Models
Existing federated learning solutions focus on transmitting features, parameters or gadients between clients and server, which suffer from serious low-efficiency and privacy-leakage problems.
Pushing the Limits of 3D Shape Generation at Scale
Through the adaptation of the Auto-Regressive model and the utilization of large language models, we have developed a remarkable model with an astounding 3. 6 billion trainable parameters, establishing it as the largest 3D shape generation model to date, named Argus-3D.
Large-scale Dataset Pruning with Dynamic Uncertainty
The state of the art of many learning tasks, e. g., image classification, is advanced by collecting larger datasets and then training larger models on them.
DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting
While diffusion models can successfully generate data and make predictions, they are predominantly designed for static images.
Improving Convergence and Generalization Using Parameter Symmetries
Finally, we show that integrating teleportation into a wide range of optimization algorithms and optimization-based meta-learning improves convergence.
Accelerated MR Fingerprinting with Low-Rank and Generative Subspace Modeling
To address this problem, we present a new image reconstruction method for MR Fingerprinting, integrating low-rank and subspace modeling with a deep generative prior.
Toward Moiré-Free and Detail-Preserving Demosaicking
In both applications, our model substantially alleviates artifacts such as Moir\'e and over-smoothness at similar or lower computational cost to currently top-performing models, as validated by diverse evaluations.
Accelerating Dataset Distillation via Model Augmentation
Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones.
The Impact of Weather on Local Government Spending
While there is a new and rapidly growing literature on the effects of climatic factors on economic and social outcomes, little research has been conducted to understand the fiscal impact of weather, especially at the sub-state level.
CR-LSO: Convex Neural Architecture Optimization in the Latent Space of Graph Variational Autoencoder with Input Convex Neural Networks
In neural architecture search (NAS) methods based on latent space optimization (LSO), a deep generative model is trained to embed discrete neural architectures into a continuous latent space.
Symmetries, flat minima, and the conserved quantities of gradient flow
Empirical studies of the loss landscape of deep networks have revealed that many local minima are connected through low-loss valleys.
Nowhere to Hide: A Lightweight Unsupervised Detector against Adversarial Examples
In this paper, we propose an AutoEncoder-based Adversarial Examples (AEAE) detector, that can guard DNN models by detecting adversarial examples with low computation in an unsupervised manner.
MSRL: Distributed Reinforcement Learning with Dataflow Fragments
Yet, current distributed RL systems tie the definition of RL algorithms to their distributed execution: they hard-code particular distribution strategies and only accelerate specific parts of the computation (e. g. policy network updates) on GPU workers.
Everyone's Preference Changes Differently: Weighted Multi-Interest Retrieval Model
In this paper, we propose the Multi-Interest Preference (MIP) model, an approach that not only produces multi-interest for users by using the user's sequential engagement more effectively but also automatically learns a set of weights to represent the preference over each embedding so that the candidates can be retrieved from each interest proportionally.
LIMO: Latent Inceptionism for Targeted Molecule Generation
We corroborate these docking-based results with more accurate molecular dynamics-based calculations of absolute binding free energy and show that one of our generated drug-like compounds has a predicted $K_D$ (a measure of binding affinity) of $6 \cdot 10^{-14}$ M against the human estrogen receptor, well beyond the affinities of typical early-stage drug candidates and most FDA-approved drugs to their respective targets.
Privacy for Free: How does Dataset Condensation Help Privacy?
In this work, we for the first time identify that dataset condensation (DC) which is originally designed for improving training efficiency is also a better solution to replace the traditional data generators for private data generation, thus providing privacy for free.
FaceMAE: Privacy-Preserving Face Recognition via Masked Autoencoders
Firstly, randomly masked face images are used to train the reconstruction module in FaceMAE.
Symmetry Teleportation for Accelerated Optimization
Experimentally, we show that teleportation improves the convergence speed of gradient descent and AdaGrad for several optimization problems including test functions, multi-layer regressions, and MNIST classification.
DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning
Coreset selection, which aims to select a subset of the most informative training samples, is a long-standing learning problem that can benefit many downstream tasks such as data-efficient learning, continual learning, neural architecture search, active learning, etc.
Synthesizing Informative Training Samples with GAN
However, traditional GANs generated images are not as informative as the real training samples when being used to train deep neural networks.
CAFE: Learning to Condense Dataset by Aligning Features
Dataset condensation aims at reducing the network training effort through condensing a cumbersome training set into a compact synthetic one.
Towards Understanding and Harnessing the Effect of Image Transformation in Adversarial Detection
Experimental results show that the contribution of image transformations to adversarial detection is significantly different, the combination of them can significantly improve the generic detection ability against state-of-the-art adversarial attacks.
Makeup216: Logo Recognition with Adversarial Attention Representations
One of the challenges of logo recognition lies in the diversity of forms, such as symbols, texts or a combination of both; further, logos tend to be extremely concise in design while similar in appearance, suggesting the difficulty of learning discriminative representations.
One-dimensional Deep Low-rank and Sparse Network for Accelerated MRI
Deep learning has shown astonishing performance in accelerated magnetic resonance imaging (MRI).
Dataset Condensation with Distribution Matching
Computational cost of training state-of-the-art deep models in many learning problems is rapidly increasing due to more sophisticated models and larger datasets.
Accelerated MRI Reconstruction with Separable and Enhanced Low-Rank Hankel Regularization
The combination of the sparse sampling and the low-rank structured matrix reconstruction has shown promising performance, enabling a significant reduction of the magnetic resonance imaging data acquisition time.
Feature-Filter: Detecting Adversarial Examples through Filtering off Recessive Features
In this paper, we reveal that imperceptible adversarial examples are the product of recessive features misleading neural networks, and an adversarial attack is essentially a kind of method to enrich these recessive features in the image.
FedCom: A Byzantine-Robust Local Model Aggregation Rule Using Data Commitment for Federated Learning
The results demonstrate its effectiveness and superior performance compared to the state-of-the-art Byzantine-robust schemes in defending against typical data poisoning and model poisoning attacks under practical Non-IID data distributions.
Concentric Spherical GNN for 3D Representation Learning
Learning 3D representations that generalize well to arbitrarily oriented inputs is a challenge of practical importance in applications varying from computer vision to physics and chemistry.
Interpretable Attention Guided Network for Fine-grained Visual Classification
We propose an Interpretable Attention Guided Network (IAGN) for fine-grained visual classification.
Dataset Condensation with Differentiable Siamese Augmentation
In many machine learning problems, large-scale datasets have become the de-facto standard to train state-of-the-art deep networks at the price of heavy computation load.
Quantum Adiabatic Doping for Atomic Fermi-Hubbard Quantum Simulations
Its feasibility has been demonstrated with numerical simulations of the adiabatic preparation for certain incommensurate particle-doping fractions, where the major problem to circumvent is the atomic localization in the incommensurate lattice.
Quantum Gases Strongly Correlated Electrons Quantum Physics
Multiple Plans are Better than One: Diverse Stochastic Planning
In planning problems, it is often challenging to fully model the desired specifications.
Active Sampling for Accelerated MRI with Low-Rank Tensors
In this paper, we introduce an active low-rank tensor model for fast MR imaging.
GreedyFool: Multi-Factor Imperceptibility and Its Application to Designing a Black-box Adversarial Attack
Adversarial examples are well-designed input samples, in which perturbations are imperceptible to the human eyes, but easily mislead the output of deep neural networks (DNNs).
Attribute-guided image generation from layout
Also, the generated images from our model have higher resolution, object classification accuracy and consistency, as compared to the previous state-of-the-art.
Augmented Bi-path Network for Few-shot Learning
Finally, the model learns to compare global and local features separately, i. e., in two paths, before merging the similarities.
PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterest
Latent user representations are widely adopted in the tech industry for powering personalized recommender systems.
Privacy-Preserving Technology to Help Millions of People: Federated Prediction Model for Stroke Prevention
It adopts federated averaging during the model training process, without patient data being taken out of the hospitals during the whole process of model training and forecasting.
Dataset Condensation with Gradient Matching
As the state-of-the-art machine learning methods in many fields rely on larger datasets, storing datasets and training models on them become significantly more expensive.
Continual Representation Learning for Biometric Identification
With the explosion of digital data in recent years, continuously learning new tasks from a stream of data without forgetting previously acquired knowledge has become increasingly important.
iDLG: Improved Deep Leakage from Gradients
Particularly, our approach can certainly extract the ground-truth labels as opposed to DLG, hence we name it Improved DLG (iDLG).
DwNet: Dense warp-based network for pose-guided human video generation
In this paper, we focus on human motion transfer - generation of a video depicting a particular subject, observed in a single image, performing a series of motions exemplified by an auxiliary (driving) video.
Image Generation from Layout
The representation of each object is disentangled into a specified/certain part (category) and an unspecified/uncertain part (appearance).
Ranked #2 on
Layout-to-Image Generation
on Visual Genome 64x64
Interpretable Spatio-temporal Attention for Video Action Recognition
Inspired by the observation that humans are able to process videos efficiently by only paying attention where and when it is needed, we propose an interpretable and easy plug-in spatial-temporal attention mechanism for video action recognition.
Guess Me if You Can: Acronym Disambiguation for Enterprises
The framework takes the enterprise corpus as input and produces a high-quality acronym disambiguation system as output.
MSplit LBI: Realizing Feature Selection and Dense Estimation Simultaneously in Few-shot and Zero-shot Learning
To solve this task, $L_{1}$ regularization is widely used for the pursuit of feature selection and avoiding overfitting, and yet the sparse estimation of features in $L_{1}$ regularization may cause the underfitting of training data.
A Large-scale Attribute Dataset for Zero-shot Learning
In classical ZSL algorithms, attributes are introduced as the intermediate semantic representation to realize the knowledge transfer from seen classes to unseen classes.
Modular Generative Adversarial Networks
Existing methods for multi-domain image-to-image translation (or generation) attempt to directly map an input image (or a random vector) to an image in one of the output domains.
Left-Right Comparative Recurrent Model for Stereo Matching
Left-right consistency check is an effective way to enhance the disparity estimation by referring to the information from the opposite view.
Zero-shot Learning via Shared-Reconstruction-Graph Pursuit
With the learned SRG, each unseen class prototype (cluster center) in the image feature space can be synthesized by the linear combination of other class prototypes, so that testing instances can be classified based on the distance to these synthesized prototypes.
AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding
Significant progress has been achieved in Computer Vision by leveraging large-scale image datasets.
Optimal Experiment Design for Magnetic Resonance Fingerprinting: Cramér-Rao Bound Meets Spin Dynamics
Magnetic resonance (MR) fingerprinting is a new quantitative imaging paradigm, which simultaneously acquires multiple MR tissue parameter maps in a single experiment.
Signal Processing
Neural Person Search Machines
We investigate the problem of person search in the wild in this work.
Ranked #4 on
Person Re-Identification
on CUHK-SYSU
Memory-Augmented Attribute Manipulation Networks for Interactive Fashion Search
We introduce a new fashion search protocol where attribute manipulation is allowed within the interaction between users and search engines, e. g. manipulating the color attribute of the clothing from red to blue.
Multi-View Image Generation from a Single-View
This paper addresses a challenging problem -- how to generate multi-view cloth images from only a single view input.
Zero-Shot Learning posed as a Missing Data Problem
This paper presents a method of zero-shot learning (ZSL) which poses ZSL as the missing data problem, rather than the missing label problem.
Diversified Visual Attention Networks for Fine-Grained Object Classification
Fine-grained object classification is a challenging task due to the subtle inter-class difference and large intra-class variation.
Fast Low-rank Representation based Spatial Pyramid Matching for Image Classification
Although the methods achieve a higher recognition rate than the traditional SPM, they consume more time to encode the local descriptors extracted from the image.
