These are the latest articles and videos I found most interesting.
- Connections between physics and deep learning
- My Neurons, My Self
- Why December Has the Longest Days
- To Scale: The Solar System
- Brain and Mental Health
Connections between physics and deep learning
Max Tegmark at the Center for Brains, Minds and Machines (CBMM)
We show how the success of deep learning depends not only on mathematics but also on physics: although well-known mathematical theorems guarantee that neural networks can approximate arbitrary functions well, the class of functions of practical interest can be approximated through “cheap learning” with exponentially fewer parameters than generic ones, because they have simplifying properties tracing back to the laws of physics. The exceptional simplicity of physics-based functions hinges on properties such as symmetry, locality, compositionality and polynomial log-probability, and we explore how these properties translate into exceptionally simple neural networks approximating both natural phenomena such as images and abstract representations thereof such as drawings. We further argue that when the statistical process generating the data is of a certain hierarchical form prevalent in physics and machine-learning, a deep neural network can be more efficient than a shallow one. We formalize these claims using information theory and discuss the relation to renormalization group procedures. We prove various “no-flattening theorems” showing when such efficient deep networks cannot be accurately approximated by shallow ones without efficiency loss: flattening even linear functions can be costly, and flattening polynomials is exponentially expensive; we use group theoretic techniques to show that n variables cannot be multiplied using fewer than 2^n neurons in a single hidden layer.
Two papers described in the video:
- Critical Behavior from Deep Dynamics: A Hidden Dimension in Natural Language
- Why does deep and cheap learning work so well?
My Neurons, My Self
With ever more refined techniques for measuring complex brain activity, scientists are challenging the understanding of thought, memory and emotion–what we have traditionally called “the self.” How do electrical and chemical currents translate to self-awareness? And why does the brain produce consciousness at all? Join a discussion among eminent neuroscientists, philosophers and psychologists who are redefining what it means to be human.
Why December Has the Longest Days
December has the longest solar days (noon-to-noon) because of the weird way a combination of the axial tilt of the earth and the eccentricity of its elliptical orbit conspire in December. Perihelion + Solstices = Long Days.
Here’s some more info about:
To Scale: The Solar System
A film by Wylie Overstreet and Alex Gorosh
On a dry lakebed in Nevada, a group of friends build the first scale model of the solar system with complete planetary orbits: a true illustration of our place in the universe.
Brain and Mental Health
This 3D medical animation shows the parts of the brain associated with mental health and function. The process of neurons passing messages to each other via neurotransmitters is shown. A few examples of mental illnesses are listed, along with the neurotransmitters associated with them.
Alan Tatourian 