Interpretable neural nets

Enforce sparsity or nonnegativity.

• [NTPV13] Learning Parts-based Representations with Nonnegative Restricted Boltzmann Machine paper
  • Enforce nonnegativity in RBM by having a regularizer that’s a quadratic barrier function \(\min(0,x)^2\).
• [MNCG14] Interpretable Sparse High-Order Boltzmann Machines paper
• [KV16] Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models paper

link to an article on extracting interpretable features out of deep nets? I forget the authors. Maybe Salakhutdinov was one of them.