Connections between physics and deep learning

These are the latest articles and videos I found most interesting.

  1. Connections between physics and deep learning
  2. My Neurons, My Self
  3. Why December Has the Longest Days
  4. To Scale: The Solar System
  5. Brain and Mental Health

Connections between physics and deep learning

Max Tegmark at the Center for Brains, Minds and Machines (CBMM)

We show how the success of deep learning depends not only on mathematics but also on physics: although well-known mathematical theorems guarantee that neural networks can approximate arbitrary functions well, the class of functions of practical interest can be approximated through “cheap learning” with exponentially fewer parameters than generic ones, because they have simplifying properties tracing back to the laws of physics. The exceptional simplicity of physics-based functions hinges on properties such as symmetry, locality, compositionality and polynomial log-probability, and we explore how these properties translate into exceptionally simple neural networks approximating both natural phenomena such as images and abstract representations thereof such as drawings. We further argue that when the statistical process generating the data is of a certain hierarchical form prevalent in physics and machine-learning, a deep neural network can be more efficient than a shallow one. We formalize these claims using information theory and discuss the relation to renormalization group procedures. We prove various “no-flattening theorems” showing when such efficient deep networks cannot be accurately approximated by shallow ones without efficiency loss: flattening even linear functions can be costly, and flattening polynomials is exponentially expensive; we use group theoretic techniques to show that n variables cannot be multiplied using fewer than 2^n neurons in a single hidden layer.

Two papers described in the video:

My Neurons, My Self

World Science Festival

With ever more refined techniques for measuring complex brain activity, scientists are challenging the understanding of thought, memory and emotion–what we have traditionally called “the self.” How do electrical and chemical currents translate to self-awareness? And why does the brain produce consciousness at all? Join a discussion among eminent neuroscientists, philosophers and psychologists who are redefining what it means to be human.

Neural vs Galactic Networks. Brain cells on the left and Intergalactic structure of the universe on the right.
Neural vs Galactic Networks. Brain cells on the left and Intergalactic structure of the universe on the right.

Why December Has the Longest Days

Minute Physics

December has the longest solar days (noon-to-noon) because of the weird way a combination of the axial tilt of the earth and the eccentricity of its elliptical orbit conspire in December. Perihelion + Solstices = Long Days.
Here’s some more info about:

To Scale: The Solar System

A film by Wylie Overstreet and Alex Gorosh

On a dry lakebed in Nevada, a group of friends build the first scale model of the solar system with complete planetary orbits: a true illustration of our place in the universe.

Brain and Mental Health

Nucleus Medical Media

This 3D medical animation shows the parts of the brain associated with mental health and function. The process of neurons passing messages to each other via neurotransmitters is shown. A few examples of mental illnesses are listed, along with the neurotransmitters associated with them.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: