Weekly summary 2016-09-10

Threads

Continue from last week.

• Work out optimality conditions for 2-layer net for DL.

(Threshold PMI’s first?)

PCANet.

Conversation with Arora

• PMI experiments
  • Is PMI the right measure? Are the feature vectors close to sparse? if the average image there is \(>20\%\) activation, then no.
  • Look for discriminative features, those that occur with one label and rarely for others.
  • Do SVM coefficients have negative weights?
• Things to read
  • [HRS16] Train faster, generalize better: stability of stochastic gradient descent paper. See also blog post.
  • Teacher-training net