Automatic Language Identification in Texts: A Survey

1 code implementation • 22 Apr 2018 • Tommi Jauhiainen, Marco Lui, Marcos Zampieri, Timothy Baldwin, Krister Lindén

Language identification (LI) is the problem of determining the natural language that a document or part thereof is written in.

Language Identification

Exploring Methods and Resources for Discriminating Similar Languages

no code implementations • WS 2014 • Marco Lui, Ned Letcher, Oliver Adams, Long Duong, Paul Cook, Timothy Baldwin

Language Identification

Accurate Language Identification of Twitter Messages

no code implementations • WS 2014 • Marco Lui, Timothy Baldwin

Language Identification Part-Of-Speech Tagging

Automatic Detection and Language Identification of Multilingual Documents

no code implementations • TACL 2014 • Marco Lui, Jey Han Lau, Timothy Baldwin

Language identification is the task of automatically detecting the language(s) present in a document based on the content of the document.

Language Identification Machine Translation

Classifying English Documents by National Dialect

no code implementations • ALTA 2013 • Marco Lui, Paul Cook

Text Classification

Recovering Casing and Punctuation using Conditional Random Fields

no code implementations • ALTA 2013 • Marco Lui, Li Wang

How Noisy Social Media Text, How Diffrnt Social Media Sources?

no code implementations • IJCNLP 2013 • Timothy Baldwin, Paul Cook, Marco Lui, Andrew MacKinlay, Li Wang

Time Series Analysis

UniMelb\_NLP-CORE: Integrating predictions from multiple domains and feature sets for estimating semantic textual similarity

no code implementations • SEMEVAL 2013 • Sp Gella, ana, Bahar Salehi, Marco Lui, Karl Grieser, Paul Cook, Timothy Baldwin

Information Retrieval Machine Translation +3

langid.py for better language modelling

no code implementations • ALTA 2012 • Paul Cook, Marco Lui

Language Modelling

Unsupervised Estimation of Word Usage Similarity

no code implementations • ALTA 2012 • Marco Lui, Timothy Baldwin, Diana McCarthy

Topic Models Word Sense Induction

Feature Stacking for Sentence Classification in Evidence-Based Medicine

no code implementations • ALTA 2012 • Marco Lui

Classification General Classification +2

langid.py: An Off-the-shelf Language Identification Tool

no code implementations • ACL 2012 • Marco Lui, Timothy Baldwin

Language Identification Machine Translation +1

Cross-domain Feature Selection for Language Identification

no code implementations • IJCNLP 2011 • Marco Lui, Timothy Baldwin

We show that transductive (cross-domain) learning is an important consideration in building a general-purpose language identification system, and develop a feature selection method that generalizes across domains.

feature selection Language Identification +1