Search Results for author:
Found 73 papers, 55 papers with code
GLM-130B: An Open Bilingual Pre-trained Model
We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters.
Ranked #1 on
Language Modelling
on CLUE (OCNLI_50K)
Extensive Self-Contrast Enables Feedback-Free Language Model Alignment
In this work, we introduce Self-Contrast, a feedback-free large language model alignment method via exploiting extensive self-generated negatives.
Heterogeneous Graph Transformer
Recent years have witnessed the emerging success of graph neural networks (GNNs) for modeling structured data.
Ranked #25 on
Node Property Prediction
on ogbn-mag
metapath2vec: Scalable Representation Learning for Heterogeneous Networks
We study the problem of representation learning in heterogeneous networks.
Ranked #5 on
Link Prediction
on MovieLens 25M
Graph Random Neural Network for Semi-Supervised Learning on Graphs
We study the problem of semi-supervised learning on graphs, for which graph neural networks (GNNs) have been extensively explored.
OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs
Enabling effective and efficient machine learning (ML) over large-scale graph data (e. g., graphs with billions of edges) can have a great impact on both industrial and scientific applications.
Ranked #1 on
Knowledge Graphs
on WikiKG90M-LSC
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Evaluations on HumanEval-X
Large pre-trained code generation models, such as OpenAI Codex, can generate syntax- and function-correct code, making the coding of programmers more productive and our pursuit of artificial general intelligence closer.
Ranked #80 on
Code Generation
on MBPP
CogVLM: Visual Expert for Pretrained Language Models
We introduce CogVLM, a powerful open-source visual language foundation model.
Ranked #4 on
Visual Question Answering (VQA)
on InfiMM-Eval
CogAgent: A Visual Language Model for GUI Agents
People are spending an enormous amount of time on digital devices through graphical user interfaces (GUIs), e. g., computer or smartphone screens.
Ranked #14 on
Visual Question Answering
on MM-Vet
Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec
This work lays the theoretical foundation for skip-gram based network embedding methods, leading to a better understanding of latent network representation learning.
Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers
By updating only 0. 1% of the model parameters, the prompt tuning strategy can help retrieval models achieve better generalization performance than traditional methods in which all parameters are updated.
Open Graph Benchmark: Datasets for Machine Learning on Graphs
We present the Open Graph Benchmark (OGB), a diverse set of challenging and realistic benchmark datasets to facilitate scalable, robust, and reproducible graph machine learning (ML) research.
Ranked #1 on
Link Property Prediction
on ogbl-citation2
AgentBench: Evaluating LLMs as Agents
We present AgentBench, a multi-dimensional evolving benchmark that currently consists of 8 distinct environments to assess LLM-as-Agent's reasoning and decision-making abilities in a multi-turn open-ended generation setting.
CogDL: A Comprehensive Library for Graph Deep Learning
In CogDL, we propose a unified design for the training and evaluation of GNN models for various graph tasks, making it unique among existing graph learning libraries.
WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences
We present WebGLM, a web-enhanced question-answering system based on the General Language Model (GLM).
AgentTuning: Enabling Generalized Agent Abilities for LLMs
Though many prompting methods have been proposed to complete particular agent tasks, there is lack of research focusing on improving the agent capabilities of LLMs themselves without compromising their general abilities.
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
We present a comprehensive solution to learn and improve text-to-image models from human preference feedback.
GPT-GNN: Generative Pre-Training of Graph Neural Networks
Graph neural networks (GNNs) have been demonstrated to be powerful in modeling graph-structured data.
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
In this paper, we introduce LongBench, the first bilingual, multi-task benchmark for long context understanding, enabling a more rigorous evaluation of long context understanding.
GraphMAE: Self-Supervised Masked Graph Autoencoders
Despite this, contrastive learning-which heavily relies on structural data augmentation and complicated training strategies-has been the dominant approach in graph SSL, while the progress of generative SSL on graphs, especially graph autoencoders (GAEs), has thus far not reached the potential as promised in other fields.
Ranked #1 on
Node Classification
on Cora: fixed 20 node per class
GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner
Graph self-supervised learning (SSL), including contrastive and generative approaches, offers great potential to address the fundamental challenge of label scarcity in real-world graph data.
GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
Graph representation learning has emerged as a powerful technique for addressing real-world problems.
CharacterGLM: Customizing Chinese Conversational AI Characters with Large Language Models
In this paper, we present CharacterGLM, a series of models built upon ChatGLM, with model sizes ranging from 6B to 66B parameters.
DeepInf: Social Influence Prediction with Deep Learning
Inspired by the recent success of deep neural networks in a wide range of computing applications, we design an end-to-end framework, DeepInf, to learn users' latent feature representation for predicting social influence.
Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understanding of their advancements.
Black-Box Prompt Optimization: Aligning Large Language Models without Model Training
However, these models are often not well aligned with human intents, which calls for additional treatments on them, that is, the alignment problem.
Graph Random Neural Networks for Semi-Supervised Learning on Graphs
We study the problem of semi-supervised learning on graphs, for which graph neural networks (GNNs) have been extensively explored.
AlignBench: Benchmarking Chinese Alignment of Large Language Models
We will provide public APIs for evaluating AlignBench with CritiqueLLM to facilitate the evaluation of LLMs' Chinese alignment.
CritiqueLLM: Scaling LLM-as-Critic for Effective and Explainable Evaluation of Large Language Model Generation
Since the natural language processing (NLP) community started to make large language models (LLMs), such as GPT-4, act as a critic to evaluate the quality of generated texts, most of them only train a critique generation model of a specific scale on specific datasets.
AutoWebGLM: Bootstrap And Reinforce A Large Language Model-based Web Navigating Agent
Large language models (LLMs) have fueled many intelligent agent tasks, such as web navigation -- but most existing agents perform far from satisfying in real-world webpages due to three factors: (1) the versatility of actions on webpages, (2) HTML text exceeding model processing capacity, and (3) the complexity of decision-making due to the open-domain nature of web.
MATCH: Metadata-Aware Text Classification in A Large Hierarchy
Multi-label text classification refers to the problem of assigning each given document its most relevant labels from the label set.
NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization
Previous research shows that 1) popular network embedding benchmarks, such as DeepWalk, are in essence implicitly factorizing a matrix with a closed form, and 2)the explicit factorization of such matrix generates more powerful embeddings than existing methods.
CogCoM: Train Large Vision-Language Models Diving into Details through Chain of Manipulations
Vision-Language Models (VLMs) have demonstrated their widespread viability thanks to extensive training in aligning visual instructions to answers.
LongAlign: A Recipe for Long Context Alignment of Large Language Models
Extending large language models to effectively handle long contexts requires instruction fine-tuning on input sequences of similar length.
Graph Robustness Benchmark: Benchmarking the Adversarial Robustness of Graph Machine Learning
To bridge this gap, we present the Graph Robustness Benchmark (GRB) with the goal of providing a scalable, unified, modular, and reproducible evaluation for the adversarial robustness of GML models.
Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries
In this work, we present the Knowledge Graph Transformer (kgTransformer) with masked pre-training and fine-tuning strategies.
OAG-BERT: Towards A Unified Backbone Language Model For Academic Knowledge Services
Academic knowledge services have substantially facilitated the development of the science enterprise by providing a plenitude of efficient research tools.
A Self-supervised Method for Entity Alignment
We present SelfKG by leveraging this discovery to design a contrastive learning strategy across two KGs.
SelfKG: Self-Supervised Entity Alignment in Knowledge Graphs
We present SelfKG with efficient strategies to optimize this objective for aligning entities without label supervision.
SciGLM: Training Scientific Language Models with Self-Reflective Instruction Annotation and Tuning
To bridge these gaps, we introduce SciGLM, a suite of scientific language models able to conduct college-level scientific reasoning.
A Large-Scale Database for Graph Representation Learning
With the rapid emergence of graph representation learning, the construction of new large-scale datasets is necessary to distinguish model capabilities and accurately assess the strengths and weaknesses of each technique.
EvoKG: Jointly Modeling Event Time and Network Structure for Reasoning over Temporal Knowledge Graphs
How can we perform knowledge reasoning over temporal knowledge graphs (TKGs)?
SCR: Training Graph Neural Networks with Consistency Regularization
However, it is unclear how to best design the generalization strategies in GNNs, as it works in a semi-supervised setting for graph data.
Ranked #3 on
Node Property Prediction
on ogbn-papers100M
ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline
Large language models (LLMs) have shown excellent mastering of human language, but still struggle in real-world applications that require mathematical problem-solving.
Graph Contrastive Learning for Anomaly Detection
To achieve the contrastive objective, we design a graph neural network encoder that can infer and further remove suspicious links during message passing, as well as learn the global context of the input graph.
GRAND+: Scalable Graph Random Neural Networks
In this work, we present a scalable and high-performance GNN framework GRAND+ for semi-supervised graph learning.
Ranked #1 on
Node Classification
on MAG-scholar-C
Web-Scale Academic Name Disambiguation: the WhoIsWho Benchmark, Leaderboard, and Toolkit
The toolkit is at https://github. com/THUDM/WhoIsWho.
Understanding WeChat User Preferences and "Wow" Diffusion
"Top Stories" is a novel friend-enhanced recommendation engine in WeChat, in which users can read articles based on preferences of both their own and their friends.
Graph Representation Learning
Social and Information Networks
TDGIA:Effective Injection Attacks on Graph Neural Networks
In the GIA scenario, the adversary is not able to modify the existing link structure and node attributes of the input graph, instead the attack is performed by injecting adversarial nodes into it.
BatchSampler: Sampling Mini-Batches for Contrastive Learning in Vision, Language, and Graphs
To make each mini-batch have fewer false negatives, we design the proximity graph of randomly-selected instances.
RecDCL: Dual Contrastive Learning for Recommendation
In this work, we investigate how to employ both batch-wise CL (BCL) and feature-wise CL (FCL) for recommendation.
GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation
Despite the recent emergence of video captioning models, how to generate vivid, fine-grained video descriptions based on the background knowledge (i. e., long and informative commentary about the domain-specific scenes with appropriate reasoning) is still far from being solved, which however has great applications such as automatic sports narrative.
Open-World Semi-Supervised Learning for Node Classification
Open-world semi-supervised learning (Open-world SSL) for node classification, that classifies unlabeled nodes into seen classes or multiple novel classes, is a practical but under-explored problem in the graph community.
Will This Paper Increase Your h-index? Scientific Impact Prediction
The effectiveness of such predictions, however, is fundamentally limited by the power-law distribution of citations, whereby publications with few citations are extremely common and publications with many citations are relatively rare.
Social and Information Networks Digital Libraries Physics and Society H.2.8; H.3.7
Neural Tensor Factorization
Neural collaborative filtering (NCF) and recurrent recommender systems (RRN) have been successful in modeling user-item relational data.
Inferring Social Status and Rich Club Effects in Enterprise Communication Networks
Our model can predict social status of individuals with 93% accuracy.
Revisiting Knowledge Base Embedding as Tensor Decomposition
We study the problem of knowledge base (KB) embedding, which is usually addressed through two frameworks---neural KB embedding and tensor decomposition.
A Century of Science: Globalization of Scientific Collaborations, Citations, and Innovations
We find that science has benefited from the shift from individual work to collaborative effort, with over 90% of the world-leading innovations generated by collaborations in this century, nearly four times higher than they were in the 1900s.
Digital Libraries Social and Information Networks Physics and Society
Generalizing Graph Convolutional Networks
Graph convolutional networks (GCNs) have emerged as a powerful framework for mining and learning with graphs.
P-INT: A Path-based Interaction Model for Few-shot Knowledge Graph Completion
Its general process is to first encode the implicit relation of an entity pair and then match the relation of a query entity pair with the relations of the reference entity pairs.
Adaptive Diffusion in Graph Neural Networks
Notably, message passing based GNNs, e. g., graph convolutional networks, leverage the immediate neighbors of each node during the aggregation process, and recently, graph diffusion convolution (GDC) is proposed to expand the propagation neighborhood by leveraging generalized graph diffusion.
Diagonal Graph Convolutional Networks with Adaptive Neighborhood Aggregation
In this paper, we revisit the mathematical foundation of GCNs and study how to extend their representation capacity.
IDPG: An Instance-Dependent Prompt Generation Method
Prompt tuning is a new, efficient NLP transfer learning paradigm that adds a task-specific prompt in each input instance during the model training stage.
Revisiting Parallel Context Windows: A Frustratingly Simple Alternative and Chain-of-Thought Deterioration
We identify two crucial limitations in the evaluation of recent parallel-integrated method Parallel Context Windows (PCW), which extends the maximum context lengths of language models, e. g., 2048 for LLaMA, by harnessing window-wise attention and positional embedding techniques.
APAR: LLMs Can Do Auto-Parallel Auto-Regressive Decoding
The massive adoption of large language models (LLMs) demands efficient deployment strategies.
xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein
We propose a unified protein language model, xTrimoPGLM, to address these two types of tasks simultaneously through an innovative pre-training framework.
TriSampler: A Better Negative Sampling Principle for Dense Retrieval
Negative sampling stands as a pivotal technique in dense retrieval, essential for training effective retrieval models and significantly impacting retrieval performance.
Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments
The applications of large language models (LLMs) have expanded well beyond the confines of text processing, signaling a new era where LLMs are envisioned as generalist language agents capable of operating within complex real-world environments.
OAG-Bench: A Human-Curated Benchmark for Academic Graph Mining
We envisage that OAG-Bench can serve as a common ground for the community to evaluate and compare algorithms in academic graph mining, thereby accelerating algorithm development and advancement in this field.
Does Negative Sampling Matter? A Review with Insights into its Theory and Applications
Is there a general framework that can incorporate all existing negative sampling methods?
CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusion
Recent advancements in text-to-image generative systems have been largely driven by diffusion models.
Understanding Emergent Abilities of Language Models from the Loss Perspective
Recent studies have put into question the belief that emergent abilities in language models are exclusive to large models.
ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedback
The work presents our practices of aligning LLMs with human preferences, offering insights into the challenges and solutions in RLHF implementations.
