Freno / Trentin Hybrid Random Fields

A Scalable Approach to Structure and Parameter Learning in Probabilistic Graphical Models
1. Auflage 2011
ISBN: 978-3-642-20308-4
Verlag: Springer
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, Band 15, 210 Seiten

Reihe: Intelligent Systems Reference Library

This book presents an exciting new synthesis of directed and undirected, discrete and continuous graphical models. Combining elements of Bayesian networks and Markov random fields, the newly introduced hybrid random fields are an interesting approach to get the best of both these worlds, with an added promise of modularity and scalability. The authors have written an enjoyable book---rigorous in the treatment of the mathematical background, but also enlivened by interesting and original historical and philosophical perspectives.
-- Manfred Jaeger, Aalborg Universitet

The book not only marks an effective direction of investigation with significant experimental advances, but it is also---and perhaps primarily---a guide for the reader through an original trip in the space of probabilistic modeling. While digesting the book, one is enriched with a very open view of the field, with full of stimulating connections. [...] Everyone specifically interested in Bayesian networks and Markov random fields should not miss it.
-- Marco Gori, Università degli Studi di Siena

Graphical models are sometimes regarded---incorrectly---as an impractical approach to machine learning, assuming that they only work well for low-dimensional applications and discrete-valued domains. While guiding the reader through the major achievements of this research area in a technically detailed yet accessible way, the book is concerned with the presentation and thorough (mathematical and experimental) investigation of a novel paradigm for probabilistic graphical modeling, the hybrid random field. This model subsumes and extends both Bayesian networks and Markov random fields. Moreover, it comes with well-defined learning algorithms, both for discrete and continuous-valued domains, which fit the needs of real-world applications involving large-scale, high-dimensional data.

Autoren/Hrsg.

Freno, Antonino

Trentin, Edmondo

Inhaltsverzeichnis

1;Title;2
2;Contents;10
3;Introduction;17
3.1;Manifesto;17
3.2;Statistics, Graphs, and Beyond;19
3.3;Probabilistic Graphical Models;22
3.4;A Piece of History;25
3.5;Overview of the Book;26
4;Bayesian Networks;31
4.1;Introduction;31
4.2;Representation of Probabilities;37
4.2.1;d-Separation;38
4.2.2;Markov Blankets;40
4.3;Parameter Learning;42
4.4;Structure Learning;44
4.4.1;Evaluation Function;44
4.4.2;Search Strategy;47
4.5;The Naive Bayes Classifier;49
4.6;Final Remarks;51
4.6.1;Extensions of Bayesian Networks;52
4.6.2;Applications of Bayesian Networks;56
4.6.3;Strengths and Weaknesses of Bayesian Networks;56
5;Markov Random Fields;58
5.1;Introduction;58
5.2;Representation of Probabilities;63
5.2.1;The Hammersley-Clifford Theorem;64
5.2.2;A Pseudo-Likelihood Measure;66
5.2.3;Markov Blankets;67
5.3;Parameter Learning;68
5.3.1;Optimizing the Weights;69
5.3.2;Finding the Maximal Cliques;73
5.4;Structure Learning;74
5.5;Final Remarks;75
5.5.1;Generalizations and Variations on the Theme;77
5.5.2;Applications of Markov Random Fields;81
5.5.3;Points of Strength and Limitationsof Markov Random Fields;82
6;Introducing Hybrid Random Fields: Discrete-Valued Variables;84
6.1;Introduction;84
6.2;Representation of Probabilities;86
6.3;Formal Properties;89
6.4;Inference;92
6.5;Parameter Learning;94
6.6;Structure Learning;94
6.6.1;Model Initialization;95
6.6.2;Search Operator;95
6.6.3;Evaluation Function;96
6.6.4;Discussion;97
6.7;Related Work;99
6.8;Final Remarks;100
7;Extending Hybrid Random Fields: Continuous-Valued Variables;102
7.1;Introduction;102
7.1.1;An Abiding Debate: Discrete vs Continuous Variables;103
7.1.2;Optimal Convergence to the Real Probability Density Function: The Parzen Window Estimator;105
7.1.3;How the Parzen Window Throws Light on the Debate;106
7.1.4;Chapter Outline;107
7.2;Conditional Density Estimation;108
7.3;Parametric Hybrid Random Fields;109
7.3.1;Normal Distributions;109
7.3.2;Gaussian Mixture Models;115
7.4;Semiparametric Hybrid Random Fields;119
7.4.1;Change of Variables;119
7.4.2;The Nonparanormal;119
7.5;Nonparametric Hybrid Random Fields;123
7.5.1;Kernel-Based Conditional Density Estimation;124
7.5.2;Bandwidth Selection;124
7.5.3;Dual-Tree Recursion;125
7.6;Structure Learning;129
7.6.1;Model Initialization;130
7.6.2;Learning the Local Structures;131
7.6.3;Learning the Global Structure;132
7.7;Final Remarks;132
8;Applications;135
8.1;Introduction;135
8.2;Selecting Features by Learning Markov Blankets;137
8.2.1;A Feature Selection Technique;137
8.2.2;Related Work;138
8.2.3;Results;139
8.3;Application to Discrete Domains;141
8.3.1;Setting Up the Markov Random Fieldand Dependency Network Models;141
8.3.2;Computational Burden of Structure Learning;142
8.3.3;Pattern Classification;144
8.3.4;Link Prediction;149
8.4;Pattern Classification in Continuous Domains;157
8.4.1;Random Data Generation;157
8.4.2;Results;160
8.5;Final Remarks;164
9;Probabilistic Graphical Models: Cognitive Science or Cognitive Technology?;165
9.1;Introduction;165
9.2;A Philosophical View of Artificial Intelligence;166
9.2.1;The Argument from Authority;166
9.2.2;The Argument from Scientific Practice;170
9.3;From Cognitive Science to Cognitive Technology;170
9.4;Statistical Machine Learning and the Philosophy of Science;172
9.4.1;Machine Learning as a Logic of Scientific Discovery;173
9.4.2;Simplicity Reconsidered;174
9.4.3;Scalability as an Epistemic Virtue;175
9.5;Final Remarks;176
10;Conclusions;177
10.1;Hybrid Random Fields: Where Are We Now?;177
10.2;Future Research: Where Do We Go from Here?;178
10.2.1;Statistical Relational Learning: Some Open Questions;178
10.2.2;Nonparametric Density Estimation:Beyond Kernel Machines;181
11;Probability Theory;182
12;Graph Theory;189
13;References;195
14;Index;211