Search Results for author:
Found 61 papers, 31 papers with code
NLP-Powered Repository and Search Engine for Academic Papers: A Case Study on Cyber Risk Literature with CyLit
As the body of academic literature continues to grow, researchers face increasing difficulties in effectively searching for relevant resources.
SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding
To develop an LLM specialized in scientific literature understanding, we propose a hybrid strategy that integrates continual pre-training (CPT) and supervised fine-tuning (SFT), to simultaneously infuse scientific domain knowledge and enhance instruction-following capabilities for domain-specific tasks. cIn this process, we identify two key challenges: (1) constructing high-quality CPT corpora, and (2) generating diverse SFT instructions.
Not All Samples Should Be Utilized Equally: Towards Understanding and Improving Dataset Distillation
Our findings suggest that prioritizing the synthesis of easier samples from the original dataset can enhance the quality of distilled datasets, especially in low IPC (image-per-class) settings.
Learning local equivariant representations for quantum operators
Predicting quantum operator matrices such as Hamiltonian, overlap, and density matrices in the density functional theory (DFT) framework is crucial for understanding material properties.
Uni-ELF: A Multi-Level Representation Learning Framework for Electrolyte Formulation Design
Advancements in lithium battery technology heavily rely on the design and engineering of electrolytes.
Uni-Mol2: Exploring Molecular Pretraining Model at Scale
In recent years, pretraining models have made significant advancements in the fields of natural language processing (NLP), computer vision (CV), and life sciences.
DiTFastAttn: Attention Compression for Diffusion Transformer Models
We identify three key redundancies in the attention computation during DiT inference: 1. spatial redundancy, where many attention heads focus on local information; 2. temporal redundancy, with high similarity between neighboring steps' attention outputs; 3. conditional redundancy, where conditional and unconditional inferences exhibit significant similarity.
Uni-Mol Docking V2: Towards Realistic and Accurate Binding Pose Prediction
In recent years, machine learning (ML) methods have emerged as promising alternatives for molecular docking, offering the potential for high accuracy without incurring prohibitive computational costs.
Uni-SMART: Universal Science Multimodal Analysis and Research Transformer
Scientific literature often includes a wide range of multimodal elements, such as tables, charts, and molecule, which are hard for text-focused LLMs to understand and analyze.
SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis
Recent breakthroughs in Large Language Models (LLMs) have revolutionized natural language understanding and generation, sparking significant interest in applying them to scientific literature analysis.
Incident-Specific Cyber Insurance
In the current market practice, many cyber insurance products offer a coverage bundle for losses arising from various types of incidents, such as data breaches and ransomware attacks, and the coverage for each incident type comes with a separate limit and deductible.
Ada3D : Exploiting the Spatial Redundancy with Adaptive Inference for Efficient 3D Object Detection
One reason for this high resource consumption is the presence of a large number of redundant background points in Lidar point clouds, resulting in spatial redundancy in both 3D voxel and dense BEV map representations.
Revisiting Data Augmentation in Model Compression: An Empirical and Comprehensive Study
In this paper, we revisit the usage of data augmentation in model compression and give a comprehensive study on the relation between model sizes and their optimal data augmentation policy.
CORSD: Class-Oriented Relational Self Distillation
Knowledge distillation conducts an effective model compression method while holding some limitations:(1) the feature based distillation methods only focus on distilling the feature map but are lack of transferring the relation of data examples; (2) the relational distillation methods are either limited to the handcrafted functions for relation extraction, such as L2 norm, or weak in inter- and intra- class relation modeling.
Uni-QSAR: an Auto-ML Tool for Molecular Property Prediction
Recently deep learning based quantitative structure-activity relationship (QSAR) models has shown surpassing performance than traditional methods for property prediction tasks in drug discovery.
Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+
Uni-Mol+ first generates a raw 3D molecule conformation from inexpensive methods such as RDKit.
Multi-Frequency Representation Enhancement with Privilege Information for Video Super-Resolution
CNN's limited receptive field restricts its ability to capture long-range spatial-temporal dependencies, leading to unsatisfactory performance in video super-resolution.
Tiny Updater: Towards Efficient Neural Network-Driven Software Updating
Significant advancements have been accomplished with deep neural networks in diverse visual tasks, which have substantially elevated their deployment in edge device software.
Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?
The success of deep learning heavily relies on large-scale data with comprehensive labels, which is more expensive and time-consuming to fetch in 3D compared to 2D images or natural languages.
Ranked #6 on
Few-Shot 3D Point Cloud Classification
on ModelNet40 10-way (10-shot)
(using extra training data)
Few-Shot 3D Point Cloud Classification
Knowledge Distillation
+1
Structured Knowledge Distillation Towards Efficient and Compact Multi-View 3D Detection
Detecting 3D objects from multi-view images is a fundamental problem in 3D computer vision.
Deep Variational Free Energy Approach to Dense Hydrogen
We developed a deep generative model-based variational free energy approach to the equations of state of dense hydrogen.
Uni-Mol: A Universal 3D Molecular Representation Learning Framework
Uni-Mol is composed of two models with the same SE(3)-equivariant transformer architecture: a molecular pretraining model trained by 209M molecular conformations; a pocket pretraining model trained by 3M candidate protein pocket data.
Ranked #1 on
Molecular Property Prediction
on MUV
DPA-1: Pretraining of Attention-based Deep Potential Model for Molecular Simulation
Machine learning assisted modeling of the inter-atomic potential energy surface (PES) is revolutionizing the field of molecular simulation.
Contrastive Deep Supervision
The success of deep learning is usually accompanied by the growth in neural network depth.
DeePKS+ABACUS as a Bridge between Expensive Quantum Mechanical Models and Machine Learning Potentials
However, for high-level QM methods, such as density functional theory (DFT) at the meta-GGA level and/or with exact exchange, quantum Monte Carlo, etc., generating a sufficient amount of data for training a ML potential has remained computationally challenging due to their high cost.
Region-aware Knowledge Distillation for Efficient Image-to-Image Translation
In this paper, we propose Region-aware Knowledge Distillation ReKo to compress image-to-image translation models.
PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection
The remarkable breakthroughs in point cloud representation learning have boosted their usage in real-world applications such as self-driving cars and virtual reality.
Cyber Risk Assessment for Capital Management
This framework is demonstrated by a case study based on a historical cyber incident dataset, which shows that a comprehensive cost-benefit analysis is necessary for a budget-constrained company with competing objectives for cyber risk management.
Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translation
Instead of directly distilling the generated images of teachers, wavelet knowledge distillation first decomposes the images into different frequency bands with discrete wavelet transformation and then only distills the high frequency bands.
$m^\ast$ of two-dimensional electron gas: a neural canonical transformation study
The quasiparticle effective mass $m^\ast$ of interacting electrons is a fundamental quantity in the Fermi liquid theory.
Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networks
Besides, an efficient deployment flow for the mobile CPU is developed, achieving up to 7. 46$\times$ inference acceleration on an octa-core ARM CPU.
Not All Regions are Worthy to be Distilled: Region-aware Knowledge Distillation Towards Efficient Image-to-Image Translation
To tackle this challenge, in this paper, we propose Region-aware Knowledge Distillation which first localizes the crucial regions in the images with attention mechanism.
Ab-initio study of interacting fermions at finite temperature with neural canonical transformation
The variational density matrix is parametrized by a permutation equivariant many-body unitary transformation together with a discrete probabilistic model.
The Phase Diagram of a Deep Potential Water Model
Using the Deep Potential methodology, we construct a model that reproduces accurately the potential energy surface of the SCAN approximation of density functional theory for water, from low temperature and pressure to about 2400 K and 50 GPa, excluding the vapor stability region.
Chemical Physics
Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors
In this paper, we suggest that the failure of knowledge distillation on object detection is mainly caused by two reasons: (1) the imbalance between pixels of foreground and background and (2) lack of distillation on the relation between different pixels.
Isotope effects on molecular structures and electronic properties of liquid water via deep potential molecular dynamics based on SCAN functional
Feynman path-integral deep potential molecular dynamics (PI-DPMD) calculations have been employed to study both light (H$_2$O) and heavy water (D$_2$O) within the isothermal-isobaric ensemble.
Chemical Physics Computational Physics
Pandemic risk management: resources contingency planning and allocation
Repeated history of pandemics, such as SARS, H1N1, Ebola, Zika, and COVID-19, has shown that pandemic risk is inevitable.
Task-Oriented Feature Distillation
Moreover, an orthogonal loss is applied to the feature resizing layer in TOFD to improve the performance of knowledge distillation.
DeePKS: a comprehensive data-driven approach towards chemically accurate density functional theory
We propose a general machine learning-based framework for building an accurate and widely-applicable energy functional within the framework of generalized Kohn-Sham density functional theory.
Deep Potential generation scheme and simulation protocol for the Li10GeP2S12-type superionic conductors
It has been a challenge to accurately simulate Li-ion diffusion processes in battery materials at room temperature using {\it ab initio} molecular dynamics (AIMD) due to its high computational cost.
Computational Physics Materials Science Chemical Physics
Integrating Machine Learning with Physics-Based Modeling
Machine learning is poised as a very powerful tool that can drastically improve our ability to carry out scientific research.
Auxiliary Training: Towards Accurate and Robust Models
In the training stage, a novel distillation method named input-aware self distillation is proposed to facilitate the primary classifier to learn the robust information from auxiliary classifiers.
Pushing the limit of molecular dynamics with ab initio accuracy to 100 million atoms with machine learning
For 35 years, {\it ab initio} molecular dynamics (AIMD) has been the method of choice for modeling complex atomistic phenomena from first principles.
Computational Physics
Fine-Grained Emotion Classification of Chinese Microblogs Based on Graph Convolution Networks
In addition, a pooling method based on percentile is proposed to improve the accuracy of the model.
Deep Density: circumventing the Kohn-Sham equations via symmetry preserving neural networks
By directly targeting at the self-consistent electron density, we demonstrate that the adapted network architecture, called the Deep Density, can effectively represent the electron density as the linear combination of contributions from many local clusters.
DP-GEN: A concurrent learning platform for the generation of reliable deep learning based potential energy models
Materials 3, 023804] and is capable of generating uniformly accurate deep learning based PES models in a way that minimizes human intervention and the computational cost for data generation and model training.
Computational Physics
Neural Canonical Transformation with Symplectic Flows
We construct flexible and powerful canonical transformations as generative models using symplectic neural networks.
Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?
Based on the proposed comparison framework, with the same accuracy and quantization, the results show that non-structrued pruning is not competitive in terms of both storage and computation efficiency.
Deep neural network for Wannier function centers
We introduce a deep neural network (DNN) model that assigns the position of the centers of the electronic charge in each atomic configuration on a molecular dynamics trajectory.
Computational Physics Materials Science Chemical Physics
SCAN: A Scalable Neural Networks Framework Towards Compact and Efficient Models
Remarkable achievements have been attained by deep neural networks in various applications.
Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation
Different from traditional knowledge distillation - a knowledge transformation methodology among networks, which forces student neural networks to approximate the softmax layer outputs of pre-trained teacher neural networks, the proposed self distillation framework distills knowledge within network itself.
Monge-Amp\`ere Flow for Generative Modeling
We present a deep generative model, named Monge-Amp\`ere flow, which builds on continuous-time gradient flow arising from the Monge-Amp\`ere equation in optimal transport theory.
Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation
An active learning procedure called Deep Potential Generator (DP-GEN) is proposed for the construction of accurate and transferable machine learning-based models of the potential energy surface (PES) for the molecular modeling of materials.
Monge-Ampère Flow for Generative Modeling
We present a deep generative model, named Monge-Amp\`ere flow, which builds on continuous-time gradient flow arising from the Monge-Amp\`ere equation in optimal transport theory.
StructADMM: A Systematic, High-Efficiency Framework of Structured Weight Pruning for DNNs
Without loss of accuracy on the AlexNet model, we achieve 2. 58X and 3. 65X average measured speedup on two GPUs, clearly outperforming the prior work.
Solving Many-Electron Schrödinger Equation Using Deep Neural Networks
We introduce a new family of trial wave-functions based on deep neural networks to solve the many-electron Schr\"odinger equation.
Computational Physics Chemical Physics
End-to-end Symmetry Preserving Inter-atomic Potential Energy Model for Finite and Extended Systems
Machine learning models are changing the paradigm of molecular modeling, which is a fundamental tool for material science, chemistry, and computational biology.
Computational Physics Materials Science Chemical Physics
DeePMD-kit: A deep learning package for many-body potential energy representation and molecular dynamics
Here we describe DeePMD-kit, a package written in Python/C++ that has been designed to minimize the effort required to build deep learning based representation of potential energy and force field and to perform molecular dynamics.
Reinforced dynamics for enhanced sampling in large atomic and molecular systems
Like metadynamics, it allows for an efficient exploration of the configuration space by adding an adaptively computed biasing potential to the original dynamics.
Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics
We introduce a scheme for molecular simulations, the Deep Potential Molecular Dynamics (DeePMD) method, based on a many-body potential and interatomic forces generated by a carefully crafted deep neural network trained with ab initio data.
Deep Potential: a general representation of a many-body potential energy surface
When tested on a wide variety of examples, Deep Potential is able to reproduce the original model, whether empirical or quantum mechanics based, within chemical accuracy.
Computational Physics
