This talk overviews probability and uncertainty in big data analysis. See more @ Microsoft.
The last forty years of the information revolution have been driven by one simple fact: the number of transistors on a silicon chip doubles every couple of years. The resulting exponential growth in the power of processors distinguishes computing from any previous technology in history. Today we are witnessing a second form of exponential growth: in the quantity of data being collected and stored. It is driving a transformation in information technology, away from solutions that are explicitly hand-crafted to those which are learned from data. Real-world data, however, is full of complexity, ambiguity and uncertainty and so the data revolution is driving a shift from computing with logic to computing with probabilities. This talk will introduce the key ideas of computing with uncertainty, and will be illustrated with a variety of large-scale case studies.
Video describes Microsoft’s Infer.NET project found here.
Infer.NET is a framework for running Bayesian inference in graphical models. It can also be used for probabilistic programming as shown in this video.
Alan Tatourian 