Search Results for author:
Found 38 papers, 18 papers with code
SiGeo: Sub-One-Shot NAS via Information Theory and Geometry of Loss Landscape
Neural Architecture Search (NAS) has become a widely used tool for automating neural network design.
AutoML for Large Capacity Modeling of Meta's Ranking Systems
The large scale of models and tight production schedule requires AutoML to outperform human baselines by only using a small number of model evaluation trials (around 100).
Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale
In industry system, such as ranking system in Meta, it is unclear whether NAS algorithms from the literature can outperform production baselines because of: (1) scale - Meta ranking systems serve billions of users, (2) strong baselines - the baselines are production models optimized by hundreds to thousands of world-class engineers for years since the rise of deep learning, (3) dynamic baselines - engineers may have established new and stronger baselines during NAS search, and (4) efficiency - the search pipeline must yield results quickly in alignment with the productionization life cycle.
DistDNAS: Search Efficient Feature Interactions within 2 Hours
To optimize search efficiency, DistDNAS distributes the search and aggregates the choice of optimal interaction modules on varying data dates, achieving over 25x speed-up and reducing search cost from 2 days to 2 hours.
Farthest Greedy Path Sampling for Two-shot Recommender Search
FGPS enhances path diversity to facilitate more comprehensive supernet exploration, while emphasizing path quality to ensure the effective identification and utilization of promising architectures.
Unified and Dynamic Graph for Temporal Character Grouping in Long Videos
In this paper, we present a unified and dynamic graph (UniDG) framework for temporal character grouping.
D3G: Exploring Gaussian Prior for Temporal Sentence Grounding with Glance Annotation
Under this setup, we propose a Dynamic Gaussian prior based Grounding framework with Glance annotation (D3G), which consists of a Semantic Alignment Group Contrastive Learning module (SA-GCL) and a Dynamic Gaussian prior Adjustment module (DGA).
Ranked #10 on
Temporal Sentence Grounding
on Charades-STA
Sex Differences in 6-Year Progression of White Matter Hyperintensities in Non-Demented Older Adults: Sydney Memory and Ageing Study
Conclusion: The findings highlighted sex differences in the associations between WMH progression and cognition decline over time, suggesting sex-specific strategies in managing WMH accumulations in ageing.
See Finer, See More: Implicit Modality Alignment for Text-based Person Retrieval
To explore the fine-grained alignment, we further propose two implicit semantic alignment paradigms: multi-level alignment (MLA) and bidirectional mask modeling (BMM).
Exploiting Feature Diversity for Make-up Temporal Video Grounding
This technical report presents the 3rd winning solution for MTVG, a new task introduced in the 4-th Person in Context (PIC) Challenge at ACM MM 2022.
NASRec: Weight Sharing Neural Architecture Search for Recommender Systems
To overcome the data multi-modality and architecture heterogeneity challenges in the recommendation domain, NASRec establishes a large supernet (i. e., search space) to search the full architectures.
Hormonal Factors Moderate the Associations Between Vascular Risk Factors and White Matter Hyperintensities
Generalised linear models were applied to examine 1) the main effects of vascular (body mass index, hip to waist ratio, pulse wave velocity, hypercholesterolemia, diabetes, hypertension, smoking status) and hormonal (testosterone levels, contraceptive pill, hormone replacement therapy, menopause) factors on WMH, and 2) the moderation effects of hormonal factors on the relationship between vascular risk factors and WMH volumes.
Computer-Aided Extraction of Select MRI Markers of Cerebral Small Vessel Disease: A Systematic Review
Cerebral small vessel disease (CSVD) is a major vascular contributor to cognitive impairment in ageing, including dementias.
Network resilience in the aging brain
Degeneration and adaptation are two competing sides of the same coin called resilience in the progressive processes of brain aging or diseases.
Head and Body: Unified Detector and Graph Network for Person Search in Media
It is by far the largest dataset for person search in media.
Brain Age Estimation From MRI Using Cascade Networks with Ranking Loss
In this paper, a novel 3D convolutional network, called two-stage-age-network (TSAN), is proposed to estimate brain age from T1-weighted MRI data.
TRP: Trained Rank Pruning for Efficient Deep Neural Networks
The TRP trained network inherently has a low-rank structure, and is approximated with negligible performance loss, thus eliminating the fine-tuning process after low rank decomposition.
Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification
In this work, we propose SVD training, the first method to explicitly achieve low-rank DNNs during training without applying SVD on every step.
Neural Predictor for Neural Architecture Search
First we train N random architectures to generate N (architecture, validation accuracy) pairs and use them to train a regression model that predicts accuracy based on the architecture.
Trained Rank Pruning for Efficient Deep Neural Networks
To accelerate DNNs inference, low-rank approximation has been widely adopted because of its solid theoretical rationale and efficient implementations.
Conditional Transferring Features: Scaling GANs to Thousands of Classes with 30% Less High-quality Data for Training
As such, training our GAN architecture requires much fewer high-quality images with a small number of additional low-quality images.
DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures
Inspired by the Hoyer measure (the ratio between L1 and L2 norms) used in traditional compressed sensing problems, we present DeepHoyer, a set of sparsity-inducing regularizers that are both differentiable almost everywhere and scale-invariant.
Joint Regularization on Activations and Weights for Efficient Neural Network Pruning
With the rapid scaling up of deep neural networks (DNNs), extensive research studies on network model compression such as weight pruning have been performed for improving deployment efficiency.
Integral Pruning on Activations and Weights for Efficient Neural Networks
With the rapidly scaling up of deep neural networks (DNNs), extensive research studies on network model compression such as weight pruning have been performed for efficient deployment.
PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration
State-of-the-art convolutional neural networks (CNNs) used in vision applications have large models with numerous weights.
Trained Rank Pruning for Efficient Deep Neural Networks
We propose Trained Rank Pruning (TRP), which iterates low rank approximation and training.
Mini-batch Serialization: CNN Training with Inter-layer Data Reuse
Training convolutional neural networks (CNNs) requires intense computations and high memory bandwidth.
SmoothOut: Smoothing Out Sharp Minima to Improve Generalization in Deep Learning
It becomes an open question whether escaping sharp minima can improve the generalization.
Learning Intrinsic Sparse Structures within Long Short-Term Memory
This work aims to learn structurally-sparse Long Short-Term Memory (LSTM) by reducing the sizes of basic structures within LSTM units, including input updates, gates, hidden states, cell states and outputs.
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
We mathematically prove the convergence of TernGrad under the assumption of a bound on gradients.
Coordinating Filters for Faster Deep Neural Networks
Moreover, Force Regularization better initializes the low-rank DNNs such that the fine-tuning can converge faster toward higher accuracy.
A Compact DNN: Approaching GoogLeNet-Level Accuracy of Classification and Domain Adaptation
Our DNN has 4. 1M parameters, which is only 6. 7% of AlexNet or 59% of GoogLeNet.
Group Scissor: Scaling Neuromorphic Computing Design to Large Neural Networks
Following rank clipping, group connection deletion further reduces the routing area of LeNet and ConvNet to 8. 1\% and 52. 06\%, respectively.
Classification Accuracy Improvement for Neuromorphic Computing Systems with One-level Precision Synapses
Brain inspired neuromorphic computing has demonstrated remarkable advantages over traditional von Neumann architecture for its high energy efficiency and parallel data processing.
Learning Structured Sparsity in Deep Neural Networks
SSL can: (1) learn a compact structure from a bigger DNN to reduce computation cost; (2) obtain a hardware-friendly structured sparsity of DNN to efficiently accelerate the DNNs evaluation.
Faster CNNs with Direct Sparse Convolutions and Guided Pruning
Pruning CNNs in a way that increase inference speed often imposes specific sparsity structures, thus limiting the achievable sparsity levels.
A New Learning Method for Inference Accuracy, Core Occupation, and Performance Co-optimization on TrueNorth Chip
IBM TrueNorth chip uses digital spikes to perform neuromorphic computing and achieves ultrahigh execution parallelism and power efficiency.
