Search Results for author:
Found 45 papers, 18 papers with code
Fewer Truncations Improve Language Modeling
In large language model training, input documents are typically concatenated together and then split into sequences of equal length to avoid padding tokens.
Token Alignment via Character Matching for Subword Completion
Generative models, widely utilized in various applications, can often struggle with prompts corresponding to partial tokens.
Unifying Generation and Compression: Ultra-low bitrate Image Coding Via Multi-stage Transformer
Recent progress in generative compression technology has significantly improved the perceptual quality of compressed data.
StarCoder 2 and The Stack v2: The Next Generation
Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size.
Ranked #25 on
Code Generation
on MBPP
Learning Efficient Unsupervised Satellite Image-based Building Damage Detection
Existing Building Damage Detection (BDD) methods always require labour-intensive pixel-level annotations of buildings and their conditions, hence largely limiting their applications.
Towards Open World Active Learning for 3D Object Detection
To seek effective solutions, we investigate a more practical yet challenging research task: Open World Active Learning for 3D Object Detection (OWAL-3D), aiming at selecting a small number of 3D boxes to annotate while maximizing detection performance on both known and unknown classes.
CIFAR-10-Warehouse: Broad and More Realistic Testbeds in Model Generalization Analysis
Analyzing model performance in various unseen environments is a critical research problem in the machine learning community.
The AI Revolution: Opportunities and Challenges for the Finance Sector
A further issue identified in this report is the systemic risk that AI can introduce to the financial sector.
Project Aria: A New Tool for Egocentric Multi-Modal AI Research
Egocentric, multi-modal data as available on future augmented reality (AR) devices provides unique challenges and opportunities for machine perception.
UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understanding
However, existing advanced algorithms are limited to effectively utilizing the immense representation capabilities and rich world knowledge inherent to these large pre-trained models, and the beneficial connections among tasks within the context of text-rich scenarios have not been sufficiently explored.
Extreme Image Compression using Fine-tuned VQGANs
Remarkably, even with the loss of up to $20\%$ of indices, the images can be effectively restored with minimal perceptual loss.
A Static Evaluation of Code Completion by Large Language Models
Large language models trained on code have shown great potential to increase productivity of software developers.
Energy-Latency Attacks to On-Device Neural Networks via Sponge Poisoning
In recent years, on-device deep learning has gained attention as a means of developing affordable deep learning applications for mobile devices.
Greener yet Powerful: Taming Large Code Generation Models with Quantization
Such large models incur significant resource usage (in terms of memory, latency, and dollars) as well as carbon footprint.
Exploring Active 3D Object Detection from a Generalization Perspective
To alleviate the high annotation cost in LiDAR-based 3D object detection, active learning is a promising solution that learns to select only a small portion of unlabeled data to annotate, without compromising model performance.
How Far Pre-trained Models Are from Neural Collapse on the Target Dataset Informs their Transferability
This paper focuses on model transferability estimation, i. e., assessing the performance of pre-trained models on a downstream task without performing fine-tuning.
CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context
While pre-trained language models (LM) for code have achieved great success in code completion, they generate code conditioned only on the contents within the file, i. e., in-file context, but ignore the rich semantics in other files within the same project, i. e., cross-file context, a critical source of information that is especially useful in modern modular software development.
ReCode: Robustness Evaluation of Code Generation Models
Most existing works on robustness in text or code tasks have focused on classification, while robustness in generation tasks is an uncharted area and to date there is no comprehensive benchmark for robustness in code generation.
Multi-lingual Evaluation of Code Generation Models
Using these benchmarks, we are able to assess the performance of code generation models in a multi-lingual fashion, and discovered generalization ability of language models on out-of-domain languages, advantages of multi-lingual models over mono-lingual, the ability of few-shot prompting to teach the model new languages, and zero-shot translation abilities even on mono-lingual settings.
ContraCLM: Contrastive Learning For Causal Language Model
Specifically, we attain $44\%$ relative improvement on the Semantic Textual Similarity tasks and $34\%$ on Code-to-Code Search tasks.
Knowing Where and What: Unified Word Block Pretraining for Document Understanding
In this paper, we focus on the embedding learning of word blocks containing text and layout information, and propose UTel, a language model with Unified TExt and Layout pre-training.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Debiasing Neural Retrieval via In-batch Balancing Regularization
People frequently interact with information retrieval (IR) systems, however, IR models exhibit biases and discrimination towards various demographics.
ShowFace: Coordinated Face Inpainting with Memory-Disentangled Refinement Networks
Furthermore, to better improve the inter-coordination between the corrupted and non-corrupted regions and enhance the intra-coordination in corrupted regions, we design InCo2 Loss, a pair of similarity based losses to constrain the feature consistency.
Towards Differential Relational Privacy and its use in Question Answering
While bounding general memorization can have detrimental effects on the performance of a trained model, bounding RM does not prevent effective learning.
DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization
Empirical analyses show that, despite the challenging nature of generative tasks, we were able to achieve a 16. 5x model footprint compression ratio with little performance drop relative to the full-precision counterparts on multiple summarization and QA datasets.
Source-Free Progressive Graph Learning for Open-Set Domain Adaptation
However, most existing OSDA approaches are limited due to three main reasons, including: (1) the lack of essential theoretical analysis of generalization bound, (2) the reliance on the coexistence of source and target data during adaptation, and (3) failing to accurately estimate the uncertainty of model predictions.
Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network with Graph Representation Learning
To preserve the generated faces being more structure-coordinated, the IRSG models inter-class structural relations among every facial component by graph representation learning.
Generative Adversarial Network
Graph Representation Learning
+1
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained on.
Self-supervised Correlation Mining Network for Person Image Generation
However, such methods fail to exploit the spatial correlation between the disentangled features.
Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation
In this paper, both empirical and theoretical investigations of this representation degeneration problem are first provided, based on which a novel recommender model DuoRec is proposed to improve the item embeddings distribution.
RoadAtlas: Intelligent Platform for Automated Road Defect Detection and Asset Management
With the rapid development of intelligent detection algorithms based on deep learning, much progress has been made in automatic road defect recognition and road marking parsing.
PAENet: A Progressive Attention-Enhanced Network for 3D to 2D Retinal Vessel Segmentation
In the three-dimensional feature learning path, we design a novel Adaptive Pooling Module (APM) and propose a new Quadruple Attention Module (QAM).
Learning to Diversify for Single Domain Generalization
Domain generalization (DG) aims to generalize a model trained on multiple source (i. e., training) domains to a distributionally different target (i. e., test) domain.
Back to Prior Knowledge: Joint Event Causality Extraction via Convolutional Semantic Infusion
Joint event and causality extraction is a challenging yet essential task in information retrieval and data mining.
Adversarial Bipartite Graph Learning for Video Domain Adaptation
To further enhance the model capacity and testify the robustness of the proposed architecture on difficult transfer tasks, we extend our model to work in a semi-supervised setting using an additional video-level bipartite graph.
Ranked #2 on
Domain Adaptation
on HMDB --> UCF (full)
Clustering and Power Optimization for NOMA Multi-Objective Problems
This paper considers uplink multiple access (MA) transmissions, where the MA technique is adaptively selected between Non Orthogonal Multiple Access (NOMA) and Orthogonal Multiple Access (OMA).
Progressive Graph Learning for Open-Set Domain Adaptation
The existing domain adaptation approaches which tackle this problem work in the closed-set setting with the assumption that the source and the target data share exactly the same classes of objects.
Modeling Subjective Assessments of Guilt in Newspaper Crime Narratives
Crime reporting is a prevalent form of journalism with the power to shape public perceptions and social policies.
Deep Collaborative Discrete Hashing with Semantic-Invariant Structure
Existing deep hashing approaches fail to fully explore semantic correlations and neglect the effect of linguistic context on visual attention learning, leading to inferior performance.
Answering Complex Open-domain Questions Through Iterative Query Generation
It is challenging for current one-step retrieve-and-read question answering (QA) systems to answer questions like "Which novel by the author of 'Armada' will be adapted as a feature film by Steven Spielberg?"
Ranked #47 on
Question Answering
on HotpotQA
TalkDown: A Corpus for Condescension Detection in Context
Condescending language use is caustic; it can bring dialogues to an end and bifurcate communities.
Demographic Inference and Representative Population Estimates from Multilingual Social Media Data
In a large experiment over multilingual heterogeneous European regions, we show that our demographic inference and bias correction together allow for more accurate estimates of populations and make a significant step towards representative social sensing in downstream applications with multilingual social media.
It's going to be okay: Measuring Access to Support in Online Communities
People use online platforms to seek out support for their informational and emotional needs.
Out-of-focus: Learning Depth from Image Bokeh for Robotic Perception
Comparative studies were conducted on both a standard RGBD data set and our own data set (learning from both single and multiple images), and results verified that stacked focal images yield better depth estimation than using just single RGB image.
