Search Results for author:
Found 53 papers, 18 papers with code
DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services
We propose DISC-LawLLM, an intelligent legal system utilizing large language models (LLMs) to provide a wide range of legal services.
Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity
Therefore, we propose Flash-LLM for enabling low-cost and highly-efficient large generative model inference with the sophisticated support of unstructured sparsity on high-performance but highly restrictive Tensor Cores.
CARE: Large Precision Matrix Estimation for Compositional Data
High-dimensional compositional data are prevalent in many applications.
TAP: Targeted Prompting for Task Adaptive Generation of Textual Training Instances for Visual Classification
Vision and Language Models (VLMs), such as CLIP, have enabled visual recognition of a potentially unlimited set of categories described by text prompts.
Accurate Prediction of Antibody Function and Structure Using Bio-Inspired Antibody Language Model
Remarkably, BALMFold outperforms those well-established methods like AlphaFold2, IgFold, ESMFold, and OmegaFold in the antibody benchmark, demonstrating significant potential to advance innovative engineering and streamline therapeutic antibody development by reducing the need for unnecessary trials.
Heterogeneous Knowledge Fusion: A Novel Approach for Personalized Recommendation via LLM
The analysis and mining of user heterogeneous behavior are of paramount importance in recommendation systems.
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
Multimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image.
Modeling Dual Period-Varying Preferences for Takeaway Recommendation
Different from traditional recommendation, takeaway recommendation faces two main challenges: (1) Dual Interaction-Aware Preference Modeling.
Sit Back and Relax: Learning to Drive Incrementally in All Weather Conditions
We propose to store these affine parameters as a memory bank for each weather condition and plug-in their weather-specific parameters during driving (i. e. test time) when the respective weather conditions are encountered.
LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collections
Recently, large-scale pre-trained Vision and Language (VL) models have set a new state-of-the-art (SOTA) in zero-shot visual classification enabling open-vocabulary recognition of potentially unlimited set of categories defined as simple language prompts.
HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level
The global-level attention can model the graphical structure of HKG using hypergraph dual-attention layers, while the local-level attention can learn the sequential structure inside H-Facts via heterogeneous self-attention layers.
Dual Intent Enhanced Graph Neural Network for Session-based New Item Recommendation
We design a dual-intent network to learn user intent from an attention mechanism and the distribution of historical data respectively, which can simulate users' decision-making process in interacting with a new item.
Neural Delay Differential Equations: System Reconstruction and Image Classification
Neural Ordinary Differential Equations (NODEs), a framework of continuous-depth neural networks, have been widely applied, showing exceptional efficacy in coping with representative datasets.
AIR-DA: Adversarial Image Reconstruction for Unsupervised Domain Adaptive Object Detection
This results in an imbalance of the adversarial training between the domain discriminator and the feature extractor.
Embedding Theory of Reservoir Computing and Reducing Reservoir Network Using Time Delays
Reservoir computing (RC), a particular form of recurrent neural network, is under explosive development due to its exceptional efficacy and high performance in reconstruction or/and prediction of complex physical systems.
MAtch, eXpand and Improve: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledge
We adapt a VL model for zero-shot and few-shot action recognition using a collection of unlabeled videos and an unpaired action dictionary.
Ranked #2 on
Zero-Shot Action Recognition
on Kinetics
TAEC: Unsupervised Action Segmentation with Temporal-Aware Embedding and Clustering
Temporal action segmentation in untrimmed videos has gained increased attention recently.
Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform
We present Rhino, a system for accelerating tensor programs with automatic parallelization on AI platform for real production environment.
Expediting Distributed DNN Training with Device Topology-Aware Graph Deployment
This paper presents TAG, an automatic system to derive optimized DNN training graph and its deployment onto any device topology, for expedited training in device- and topology- heterogeneous ML clusters.
TAP: Accelerating Large-Scale DNN Training Through Tensor Automatic Parallelisation
However, finding a suitable model parallel schedule for an arbitrary neural network is a non-trivial task due to the exploding search space.
Optimal Transport Minimization: Crowd Localization on Density Maps for Semi-Supervised Counting
In this paper, we propose the optimal transport minimization (OT-M) algorithm for crowd localization with density maps.
Scale-Prior Deformable Convolution for Exemplar-Guided Class-Agnostic Counting
Here we propose a scale-sensitive generalized loss to tackle this problem.
Ranked #3 on
Object Counting
on FSC147
SKDBERT: Compressing BERT via Stochastic Knowledge Distillation
In this paper, we propose Stochastic Knowledge Distillation (SKD) to obtain compact BERT-style language model dubbed SKDBERT.
Video Test-Time Adaptation for Action Recognition
Our proposed method demonstrates a substantial performance gain over existing test-time adaptation approaches in both evaluations of a single distribution shift and the challenging case of random distribution shifts.
ActMAD: Activation Matching to Align Distributions for Test-Time-Training
Test-Time-Training (TTT) is an approach to cope with out-of-distribution (OOD) data by adapting a trained model to distribution shifts occurring at test-time.
MATE: Masked Autoencoders are Online 3D Test-Time Learners
Our MATE is the first Test-Time-Training (TTT) method designed for 3D data, which makes deep networks trained for point cloud classification robust to distribution shifts occurring in test data.
Multi-Frequency-Aware Patch Adversarial Learning for Neural Point Cloud Rendering
The proposed approach aims to improve the rendering realness by minimizing the spectrum discrepancy between real and synthesized images, especially on the high-frequency localized sharpness information which causes image blur visually.
Heterogeneous Federated Learning on a Graph
In this work, we consider parameter estimation in federated learning with data distribution and communication heterogeneity, as well as limited computational capacity of local devices.
RAW-GNN: RAndom Walk Aggregation based Graph Neural Network
Due to the homophily assumption of Graph Convolutional Networks (GCNs) that these methods use, they are not suitable for heterophily graphs where nodes with different labels or dissimilar attributes tend to be adjacent.
CGMN: A Contrastive Graph Matching Network for Self-Supervised Graph Similarity Learning
As most of the existing graph neural networks yield effective graph representations of a single graph, little effort has been made for jointly learning two graph representations and calculating their similarity score.
Cross-View Cross-Scene Multi-View Crowd Counting
Multi-view crowd counting has been previously proposed to utilize multi-cameras to extend the field-of-view of a single camera, capturing more people in the scene, and improve counting performance for occluded people or those in low resolution.
EasyNLP: A Comprehensive and Easy-to-use Toolkit for Natural Language Processing
The success of Pre-Trained Models (PTMs) has reshaped the development of Natural Language Processing (NLP).
PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systems
However, we observe that GPU devices in training recommender systems are underutilized, and they cannot attain an expected throughput improvement as what it has achieved in CV and NLP areas.
CycDA: Unsupervised Cycle Domain Adaptation from Image to Video
To address these challenges, we propose Cycle Domain Adaptation (CycDA), a cycle-based approach for unsupervised image-to-video domain adaptation by leveraging the joint spatial information in images and videos on the one hand and, on the other hand, training an independent spatio-temporal model to bridge the modality gap.
AC-Feasible Power Transfer Regions of Virtual Power Plants: Characterization and Application
Distributed energy resources (DERs) in distribution networks can be aggregated as a virtual power plant (VPP) for transmission-level operations.
Neural Piecewise-Constant Delay Differential Equations
Continuous-depth neural networks, such as the Neural Ordinary Differential Equations (ODEs), have aroused a great deal of interest from the communities of machine learning and data science in recent years, which bridge the connection between deep neural networks and dynamical systems.
Tie-line Security Regions in High Dimension for Renewable Accommodations
However, a tie-line security region is a high-dimension polytope due to multiple time periods and border buses inherently in power system operations, leading to the considerable computational burden.
Cost Functions over Feasible Power Transfer Regions of Virtual Power Plants
To address this challenge, a characterization method is presented in this paper for the intraday operation of a VPP based on the concepts of nonanticipativity and robustness to DERs' uncertainties.
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
Recent expeditious developments in deep learning algorithms, distributed training, and even hardware design for large models have enabled training extreme-scale models, say GPT-3 and Switch Transformer possessing hundreds of billions or even trillions of parameters.
Binary Code based Hash Embedding for Web-scale Applications
In these applications, embedding learning of categorical features is crucial to the success of deep learning models.
Learning Effective and Efficient Embedding via an Adaptively-Masked Twins-based Layer
Each feature value is mapped to an embedding vector via an embedding learning process.
Boosting the Convergence of Reinforcement Learning-based Auto-pruning Using Historical Data
Reinforcement learning (RL)-based auto-pruning has been further proposed to automate the DNN pruning process to avoid expensive hand-crafted work.
Nonasymptotic theory for two-layer neural networks: Beyond the bias-variance trade-off
Large neural networks have proved remarkably effective in modern deep learning practice, even in the overparametrized regime where the number of active parameters is large relative to the sample size.
M6-T: Exploring Sparse Expert Models and Beyond
Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost, and thus it has become a trend in model scaling.
Explicit Semantic Cross Feature Learning via Pre-trained Graph Neural Networks for CTR Prediction
Most of the existing methods adopt a DNN-based model to capture the cross features in an implicit manner.
Towards a Better Tradeoff between Effectiveness and Efficiency in Pre-Ranking: A Learnable Feature Selection based Approach
In real-world search, recommendation, and advertising systems, the multi-stage ranking architecture is commonly adopted.
Joule-Thomson expansion of the torus-like black hole
Furthermore, we investigate similarities and differences between the Van der Waals fluid, the torus-like black hole and the charged AdS black holes for the expansion.
General Relativity and Quantum Cosmology
M6: A Chinese Multimodal Pretrainer
In this work, we construct the largest dataset for multimodal pretraining in Chinese, which consists of over 1. 9TB images and 292GB texts that cover a wide range of domains.
Neural Delay Differential Equations
Neural Ordinary Differential Equations (NODEs), a framework of continuous-depth neural networks, have been widely applied, showing exceptional efficacy in coping with some representative datasets.
EasyTransfer -- A Simple and Scalable Deep Transfer Learning Platform for NLP Applications
The literature has witnessed the success of leveraging Pre-trained Language Models (PLMs) and Transfer Learning (TL) algorithms to a wide range of Natural Language Processing (NLP) applications, yet it is not easy to build an easy-to-use and scalable TL toolkit for this purpose.
INT8 Winograd Acceleration for Conv1D Equipped ASR Models Deployed on Mobile Devices
The intensive computation of Automatic Speech Recognition (ASR) models obstructs them from being deployed on mobile devices.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A bi-diffusion based layer-wise sampling method for deep learning in large graphs
The Graph Convolutional Network (GCN) and its variants are powerful models for graph representation learning and have recently achieved great success on many graph-based applications.
