David J. Hand

A well-known statistician presents his theory that extraordinary and rare events are actually commonplace and cites stories of two-time lottery winners and other bizarre coincidences to support his theory that unlikely events statistically must happen. 50,000 first printing.

Why Coincidences, Miracles, and Rare Events Happen Every Day

The Improbability Principle

"Data describe and represent the world. However, no matter how big they may be, data sets don't - indeed cannot - capture everything. Data are measurements - and, as such, they represent only what has been measured. They don't necessarily capture all the information that is relevant to the questions we may want to ask. If we do not take into account what may be missing/unknown in the data we have, we may find ourselves unwittingly asking questions that our data cannot actually address, come to mistaken conclusions, and make disastrous decisions. In this book, David Hand looks at the ubiquitous phenomenon of "missing data." He calls this "dark data" (making a comparison to "dark matter" - i.e., matter in the universe that we know is there, but which is invisible to direct measurement). He reveals how we can detect when data is missing, the types of settings in which missing data are likely to be found, and what to do about it. It can arise for many reasons, which themselves may not be obvious - for example, asymmetric information in wars; time delays in financial trading; dropouts in clinical trials; deliberate selection to enhance apparent performance in hospitals, policing, and schools; etc. What becomes clear is that measuring and collecting more and more data (big data) will not necessarily lead us to better understanding or to better decisions. We need to be vigilant to what is missing or unknown in our data, so that we can try to control for it. How do we do that? We can be alert to the causes of dark data, design better data-collection strategies that sidestep some of these causes - and, we can ask better questions of our data, which will lead us to deeper insights and better decisions"--

Why What You Don’t Know Matters

Dark Data

Statistics has evolved into an exciting discipline which uses deep theory and powerful software to shed light on the world around us: from clinical trials in medicine, to economics, sociology, and countless other subjects vital to understanding modern life. This Very Short Introduction explores and explains how statistics works today.

Statistics: A Very Short Introduction

Heikki Mannila

Padhraic Smyth

The first truly interdisciplinary text on data mining, blending the contributions of information science, computer science, and statistics. The growing interest in data mining is motivated by a common problem across disciplines: how does one store, access, model, and ultimately describe and understand very large data sets? Historically, different aspects of data mining have been addressed independently by different disciplines. This is the first truly interdisciplinary text on data mining, blending the contributions of information science, computer science, and statistics. The book consists of three sections. The first, foundations, provides a tutorial overview of the principles underlying data mining algorithms and their application. The presentation emphasizes intuition rather than rigor. The second section, data mining algorithms, shows how algorithms are constructed to solve specific problems in a principled manner. The algorithms covered include trees and rules for classification and regression, association rules, belief networks, classical statistical models, nonlinear models such as neural networks, and local "memory-based" models. The third section shows how all of the preceding analysis fits together when applied to real-world data mining problems. Topics include the role of metadata, how to handle missing data, and data preprocessing.

Principles of Data Mining

Improbability Principle

Measurement is a fundamental concept that underpins almost every aspect of the modern world. It is central to the sciences, social sciences, medicine, and economics, but it affects everyday life. We measure everything - from the distance of far-off galaxies to the temperature of the air, levels of risk, political majorities, taxes, blood pressure, IQ, and weight. The history of measurement goes back to the ancient world, and its story has been one of gradual standardization. Today there are different types of measurement, levels of accuracy, and systems of units, applied in different contexts. Measurement involves notions of variability, accuracy, reliability, and error, and challenges such as the measurement of extreme values. In this Very Short Introduction, David Hand explains the common mathematical framework underlying all measurement, the main approaches to measurement, and the challenges involved. Following a brief historical account of measurement, he discusses measurement as used in the physical sciences and engineering, the life sciences and medicine, the social and behavioural sciences, economics, business, and public policy. ABOUT THE SERIES: The Very Short Introductions series from Oxford University Press contains hundreds of titles in almost every subject area. These pocket-sized books are the perfect way to get ahead in a new subject quickly. Our expert authors combine facts, analysis, perspective, new ideas, and enthusiasm to make interesting and challenging topics highly readable.

Measurement

Wojtek J. Krzanowski

Since ROC curves have become ubiquitous in many application areas, the various advances have been scattered across disparate articles and texts. ROC Curves for Continuous Data is the first book solely devoted to the subject, bringing together all the relevant material to provide a clear understanding of how to analyze ROC curves. The fundamental theory of ROC curves The book first discusses the relationship between the ROC curve and numerous performance measures and then extends the theory into practice by describing how ROC curves are estimated. Further building on the theory, the authors present statistical tests for ROC curves and their summary statistics. They consider the impact of covariates on ROC curves, examine the important special problem of comparing two ROC curves, and cover Bayesian methods for ROC analysis. Special topics The text then moves on to extensions of the basic analysis to cope with more complex situations, such as the combination of multiple ROC curves and problems induced by the presence of more than two classes. Focusing on design and interpretation issues, it covers missing data, verification bias, sample size determination, the design of ROC studies, and the choice of optimum threshold from the ROC curve. The final chapter explores applications that not only illustrate some of the techniques but also demonstrate the very wide applicability of these techniques across different disciplines. With nearly 5,000 articles published to date relating to ROC analysis, the explosive interest in ROC curves and their analysis will continue in the foreseeable future. Embracing this growth of interest, this timely book will undoubtedly guide present and future users of ROC analysis.

ROC Curves for Continuous Data

Measurement: A Very Short Introduction

Olasiliksizlik

Martin J. Crowder

Repeated measures data arise when the same characteristic is measured on each case or subject at several times or under several conditions. There is a multitude of techniques available for analysing such data and in the past this has led to some confusion. This book describes the whole spectrum of approaches, beginning with very simple and crude methods, working through intermediate techniques commonly used by consultant statisticians, and concluding with more recent and advanced methods. Those covered include multiple testing, response feature analysis, univariate analysis of variance approaches, multivariate analysis of variance approaches, regression models, two-stage line models, approaches to categorical data and techniques for analysing crossover designs. The theory is illustrated with examples, using real data brought to the authors during their work as statistical consultants.

Analysis of Repeated Measures

Modern statistics is very different from the dry and dusty discipline of the popular imagination. In its place is an exciting subject which uses deep theory and powerful software tools to shed light and enable understanding. And it sheds this light on all aspects of our lives, enabling astronomers to explore the origins of the universe, archaeologists to investigate ancient civilisations, governments to understand how to benefit and improve society, and businesses to learn how best to provide goods and services. Aimed at readers with no prior mathematical knowledge, this Very Short Introduction explores and explains how statistics work, and how we can decipher them. ABOUT THE SERIES: The Very Short Introductions series from Oxford University Press contains hundreds of titles in almost every subject area. These pocket-sized books are the perfect way to get ahead in a new subject quickly. Our expert authors combine facts, analysis, perspective, new ideas, and enthusiasm to make interesting and challenging topics highly readable.

E. Ostrowski

K. McConway

D. Lunn

Fergus Daly

This book should be of interest to statistics lecturers who want ready-made data sets complete with notes for teaching.

David J. Hand

About

Books

Join the club