Qudata.com - Abstracts for Machine Learning

Library -> Machine Learning

Machine Learning

Classifying Cue Phrases in Text and Speech Using Machine Learning
Author: Diane J. Litman (AT&T Bell Laboratories, Murray Hill, NJ)
Comments: 8 pages, PostScript File, to appear in the Proceedings of AAAI-94
Subj-class: Computation and Language
Cue phrases may be used in a discourse sense to explicitly signal discourse structure, but also in a sentential sense to convey semantic rather than structural information. This paper explores the use of machine learning for classifying cue phrases as discourse or sentential. Two machine learning programs (Cgrendel and C4.5) are used to induce classification rules from sets of pre-classified cue phrases and their features. Machine learning is shown to be an effective technique for not only automating the generation of classification rules, but also for improving upon previous results.
Full-text: PostScript, PDF, or Other formats
Rule-based Machine Learning Methods for Functional Prediction
Authors: S. M. Weiss, N. Indurkhya
Comments: See this http URL for any accompanying files
Subj-class: Artificial Intelligence
Journal-ref: Journal of Artificial Intelligence Research, Vol 3, (1995), 383-403
We describe a machine learning method for predicting the value of a real-valued function, given the values of multiple input variables. The method induces solutions from samples in the form of ordered disjunctive normal form (DNF) decision rules. A central objective of the method and representation is the induction of compact, easily interpretable solutions. This rule-based decision model can be extended to search efficiently for similar cases prior to approximating function values. Experimental results on real-world data demonstrate that the new techniques are competitive with existing machine learning and statistical methods and can sometimes yield superior regression performance.
Full-text: PostScript, PDF
A Machine Learning Approach to the Classification of Dialogue Utterances
Author: Toine Andernach (Parlevink Group, Department of Computer Science, University of Twente, The Netherlands)
Comments: 12 pages, using nemlap.sty, harvard.sty and agsm.bst, to appear in Proceedings of NeMLaP-2, Bilkent University, Ankara, Turkey
Subj-class: Computation and Language
The purpose of this paper is to present a method for automatic classification of dialogue utterances and the results of applying that method to a corpus. Superficial features of a set of training utterances (which we will call cues) are taken as the basis for finding relevant utterance classes and for extracting rules for assigning these classes to new utterances. Each cue is assumed to partially contribute to the communicative function of an utterance. Instead of relying on subjective judgments for the tasks of finding classes and rules, we opt for using machine learning techniques to guarantee objectivity.
Full-text: PostScript, PDF, or Other formats
Cue Phrase Classification Using Machine Learning
Author: Diane J. Litman (AT&T Labs - Research)
Comments: 42 pages, uses jair.sty, theapa.bst, theapa.sty
Subj-class: Computation and Language
Journal-ref: Journal of Artificial Intelligence Research 5 (1996) 53-94
Cue phrases may be used in a discourse sense to explicitly signal discourse structure, but also in a sentential sense to convey semantic rather than structural information. Correctly classifying cue phrases as discourse or sentential is critical in natural language processing systems that exploit discourse structure, e.g., for performing tasks such as anaphora resolution and plan recognition. This paper explores the use of machine learning for classifying cue phrases as discourse or sentential. Two machine learning programs (Cgrendel and C4.5) are used to induce classification models from sets of pre-classified cue phrases and their features in text and speech. Machine learning is shown to be an effective technique for not only automating the generation of classification models, but also for improving upon previous results. When compared to manually derived classification models already in the literature, the learned models often perform with higher accuracy and contain new linguistic insights into the data. In addition, the ability to automatically construct classification models makes it easier to comparatively analyze the utility of alternative feature representations of the data. Finally, the ease of retraining makes the learning approach more scalable and flexible than manual methods.
Full-text: PostScript, PDF, or Other formats
Comparative Experiments on Disambiguating Word Senses: An Illustration of the Role of Bias in Machine Learning
Author: Raymond J. Mooney (University of Texas at Austin)
Comments: 10 pages
Subj-class: Computation and Language
This paper describes an experimental comparison of seven different learning algorithms on the problem of learning to disambiguate the meaning of a word from context. The algorithms tested include statistical, neural-network, decision-tree, rule-based, and case-based classification techniques. The specific problem tested involves disambiguating six senses of the word ``line'' using the words in the current and proceeding sentence as context. The statistical and neural-network methods perform the best on this particular problem and we discuss a potential reason for this observed difference. We also discuss the role of bias in machine learning and its importance in explaining performance differences observed on specific problems.
Full-text: PostScript, PDF
Machine Learning of User Profiles: Representational Issues
Author: Eric Bloedorn (MITRE Corporation and George Mason University), Inderjeet Mani (MITRE Corporation), T. Richard MacMillan (MITRE Corporation)
Comments: 6 pages
Subj-class: Computation and Language; Learning
As more information becomes available electronically, tools for finding information of interest to users becomes increasingly important. The goal of the research described here is to build a system for generating comprehensible user profiles that accurately capture user interest with minimum user interaction. The research described here focuses on the importance of a suitable generalization hierarchy and representation for learning profiles which are predictively accurate and comprehensible. In our experiments we evaluated both traditional features based on weighted term vectors as well as subject features corresponding to categories which could be drawn from a thesaurus. Our experiments, conducted in the context of a content-based profiling system for on-line newspapers on the World Wide Web (the IDD News Browser), demonstrate the importance of a generalization hierarchy and the promise of combining natural language processing techniques with machine learning (ML) to address an information retrieval (IR) problem.
Full-text: PostScript, PDF
Cached Sufficient Statistics for Efficient Machine Learning with Large Datasets
Authors: A. Moore, M. S. Lee
Comments: See this http URL for any accompanying files
Subj-class: Artificial Intelligence
Journal-ref: Journal of Artificial Intelligence Research, Vol 8, (1998), 67-91
This paper introduces new algorithms and data structures for quick counting for machine learning datasets. We focus on the counting task of constructing contingency tables, but our approach is also applicable to counting the number of records in a dataset that match conjunctive queries. Subject to certain assumptions, the costs of these operations can be shown to be independent of the number of records in the dataset and loglinear in the number of non-zero entries in the contingency table. We provide a very sparse data structure, the ADtree, to minimize memory use. We provide analytical worst-case bounds for this structure for several models of data distribution. We empirically demonstrate that tractably-sized data structures can be produced for large real-world datasets by (a) using a sparse tree structure that never allocates memory for counts of zero, (b) never allocating memory for counts that can be deduced from other counts, and (c) not bothering to expand the tree fully near its leaves. We show how the ADtree can be used to accelerate Bayes net structure finding algorithms, rule learning algorithms, and feature selection algorithms, and we provide a number of empirical results comparing ADtree methods against traditional direct counting approaches. We also discuss the possible uses of ADtrees in other machine learning methods, and discuss the merits of ADtrees in comparison with alternative representations such as kd-trees, R-trees and Frequent Sets.
Full-text: PostScript, PDF
Machine Learning of Generic and User-Focused Summarization
Authors: Inderjeet Mani, Eric Bloedorn
Comments: In Proceedings of the Fifteenth National Conference on AI (AAAI-98), p. 821-826
Subj-class: Computation and Language; Learning
ACM-class: I.2.6; I.2.7
A key problem in text summarization is finding a salience function which determines what information in the source should be included in the summary. This paper describes the use of machine learning on a training corpus of documents and their abstracts to discover salience functions which describe what combination of features is optimal for a given summarization task. The method addresses both "generic" and user-focused summaries.
Full-text: PostScript, PDF, or Other formats
Explanation-based Learning for Machine Translation
Authors: Janine Toole, Fred Popowich, Devlan Nicholson, Davide Turcato, Paul McFetridge
Comments: 12 pages, 3 figures, To appear in Proceedings of the 8th International Conference on Theoretical and Methodological Issues in Machine Translation
Subj-class: Computation and Language
ACM-class: J.5
In this paper we present an application of explanation-based learning (EBL) in the parsing module of a real-time English-Spanish machine translation system designed to translate closed captions. We discuss the efficiency/coverage trade-offs available in EBL and introduce the techniques we use to increase coverage while maintaining a high level of space and time efficiency. Our performance results indicate that this approach is effective.
Full-text: PostScript, PDF, or Other formats
A Machine-Learning Approach to Estimating the Referential Properties of Japanese Noun Phrases
Authors: Masaki Murata, Kiyotaka Uchimoto, Qing Ma, Hitoshi Isahara
Comments: 9 pages. Computation and Language. This paper is included in the book entitled by "Computational Linguistics and Intelligent Text Processing, Second International Conference, CICLing 2001, Mexico City, February 2001 Proceedings", Alexander Gelbukh (Ed.), Springer Publisher, ISSN 0302-9743, ISBN 3-540-41687-0
Subj-class: Computation and Language
ACM-class: H.3.3; I.2.7
Journal-ref: CICLing'2001, Mexico City, February 2001
The referential properties of noun phrases in the Japanese language, which has no articles, are useful for article generation in Japanese-English machine translation and for anaphora resolution in Japanese noun phrases. They are generally classified as generic noun phrases, definite noun phrases, and indefinite noun phrases. In the previous work, referential properties were estimated by developing rules that used clue words. If two or more rules were in conflict with each other, the category having the maximum total score given by the rules was selected as the desired category. The score given by each rule was established by hand, so the manpower cost was high. In this work, we automatically adjusted these scores by using a machine-learning method and succeeded in reducing the amount of manpower needed to adjust these scores.
Full-text: PostScript, PDF, or Other formats
Correction of Errors in a Modality Corpus Used for Machine Translation by Using Machine-learning Method
Authors: Masaki Murata, Masao Utiyama, Kiyotaka Uchimoto, Qing Ma, Hitoshi Isahara
Comments: 9 pages. Computation and Language. This paper is the English translation of our Japanese papar
Subj-class: Computation and Language
ACM-class: H.3.3; I.2.7
We performed corpus correction on a modality corpus for machine translation by using such machine-learning methods as the maximum-entropy method. We thus constructed a high-quality modality corpus based on corpus correction. We compared several kinds of methods for corpus correction in our experiments and developed a good method for corpus correction.
Full-text: PostScript, PDF, or Other formats
Man [and Woman] vs. Machine: A Case Study in Base Noun Phrase Learning
Authors: Eric Brill, Grace Ngai
Comments: 8 pages, 2 figures, presented at ACL 1999
Subj-class: Computation and Language
ACM-class: I.2.7
Journal-ref: Proceedings of the 37th Annual Meeting of the Association of Computational Linguistics, pages 65-72, College Park, MD, USA (1999)
A great deal of work has been done demonstrating the ability of machine learning algorithms to automatically extract linguistic knowledge from annotated corpora. Very little work has gone into quantifying the difference in ability at this task between a person and a machine. This paper is a first step in that direction.
Full-text: PostScript, PDF, or Other formats
Machine Learning in Automated Text Categorization
Authors: Fabrizio Sebastiani
Comments: Accepted for publication on ACM Computing Surveys
Subj-class: Information Retrieval; Learning
ACM-class: H.3.1;H.3.3;I.2.3
The automated categorization (or classification) of texts into predefined categories has witnessed a booming interest in the last ten years, due to the increased availability of documents in digital form and the ensuing need to organize them. In the research community the dominant approach to this problem is based on machine learning techniques: a general inductive process automatically builds a classifier by learning, from a set of preclassified documents, the characteristics of the categories. The advantages of this approach over the knowledge engineering approach (consisting in the manual definition of a classifier by domain experts) are a very good effectiveness, considerable savings in terms of expert manpower, and straightforward portability to different domains. This survey discusses the main approaches to text categorization that fall within the machine learning paradigm. We will discuss in detail issues pertaining to three different problems, namely document representation, classifier construction, and classifier evaluation.
Full-text: PostScript, PDF, or Other formats
Machine Learning with Lexical Features: The Duluth Approach to Senseval-2
Authors: Ted Pedersen
Comments: Appears in the Proceedings of SENSEVAL-2: Second International Workshop on Evaluating Word Sense Disambiguation Systems July 5-6, 2001, Toulouse, France
Subj-class: Computation and Language
ACM-class: I.2.7
This paper describes the sixteen Duluth entries in the Senseval-2 comparative exercise among word sense disambiguation systems. There were eight pairs of Duluth systems entered in the Spanish and English lexical sample tasks. These are all based on standard machine learning algorithms that induce classifiers from sense-tagged training text where the context in which ambiguous words occur are represented by simple lexical features. These are highly portable, robust methods that can serve as a foundation for more tailored approaches.
Full-text: PostScript, PDF, or Other formats
Thumbs up? Sentiment Classification using Machine Learning Techniques
Authors: Bo Pang, Lillian Lee, Shivakumar Vaithyanathan
Comments: To appear in EMNLP-2002
Subj-class: Computation and Language; Learning
ACM-class: I.2.7; I.2.6
We consider the problem of classifying documents not by topic, but by overall sentiment, e.g., determining whether a review is positive or negative. Using movie reviews as data, we find that standard machine learning techniques definitively outperform human-produced baselines. However, the three machine learning methods we employed (Naive Bayes, maximum entropy classification, and support vector machines) do not perform as well on sentiment classification as on traditional topic-based categorization. We conclude by examining factors that make the sentiment classification problem more challenging.
Full-text: PostScript, PDF, or Other formats
Comparing Machine Learning and Knowledge Discovery in DataBases : An Application to Knowledge Discovery in Texts
Author: Yves Kodratoff
CNRS, LRI Bât. 490 Univ. Paris-Sud, F - 91405 Orsay Cedex yk@lri.fr
This presentation has two goals. The first goal is to compare ML and Knowledge Discovery in Data (KDD, also often called Data Mining, DM) in order to insist on how much they actually differ In order to make my ideas somewhat easier to understand, and as an illustration, I will include a description of several research topics that I find relevant to KDD and to KDD only. The second goal is to show that the definition I give of KDD can be almost directly applied to text analysis, and that will lead us to a very restrictive definition of Knowledge Discovery in Texts (KDT). I will provide a compelling example of a real-life set of rules obtained by what I call KDT techniques.
Full-text: PDF
Machine Learning
Author: Luc De Raedt
Cours of Lectures
Foundations of AI
Chapter 10: Machine Learning.
Full-text: PostScript, PDF
A Simpler Look at Consistency
Authors: Spears, William M. and Diana F. Gordon (1994)
Technical Report AIC-94-018). Washington, DC: Naval Research Laboratory, Navy Center for Applied Research on Artificial Intelligence.
Keywords: Consistency bias, simplicity bias, supervised concept learning
One of the major goals of most early concept learners was to find hypotheses that were perfectly consistent with the training data. It was believed that this goal would indirectly achieve a high degree of predictive accuracy on a set of test data. Later research has partially disproved this belief. However, the issue of consistency has not yet been resolved completely. We examine the issue of consistency from a new perspective. To avoid overfitting the training data, a considerable number of current systems have sacrificed the goal of learning hypotheses that are perfectly consistent with the training instances by setting a goal of hypothesis simplicity (Occam s razor). Instead of using simplicity as a goal, we have developed a novel approach that addresses consistency directly. In other words, our concept learner has the explicit goal of selecting the most appropriate degree of consistency with the training data. We begin this paper by exploring concept learning with less than perfect consistency. Next, we describe a system that can adapt its degree of consistency in response to feedback about predictive accuracy on test data. Finally, we present the results of initial experiments that begin to address the question of how tightly hypotheses should fit the training data for different problems.
Full-text: PostScript, PDF, or PS compressed
Competition-Based Learning
Authors: John J. Grefenstette, Kenneth A. De Jong, William M. Spears
chapter 6 in Foundations of Knowledge Acquisition: Machine Learning, 203-225. Alan Meyrowitz and Susan Chipman (editors), Kluwer Academic Publishers
This paper summarizes recent research on competition-based learning procedures performed by the Navy Center for Applied Research in Artificial Intelligence at the Naval Research Laboratory. We have focused on a particularly interesting class of competition-based techniques called genetic algorithms. Genetic algorithms are adaptive search algorithms based on principles derived from the mechanisms of biological evolution. Recent results on the analysis of the implicit parallelism of alternative selection algorithms are summarized, along with an analysis of alternative crossover operators. Applications of these results in practical learning systems for sequential decision problems and for concept classification are also presented.
Full-text: PostScript, PDF, or PS compressed
Residual Algorithms: Reinforcement Learning with Function Approximation
Author: Leemon Baird
Machine Learning: Proceedings of the Twelfth International Conference, 9-12 July, Morgan Kaufman Publishers, San Francisco, CA.(22 Nov 95 errata corrects errors in the published version).

Full-text: GZiped PostScript, PDF, RTF, or HTML