Neural net experiments

Done

• MNIST basics
• CIFAR-10 basics

Todo

See Convnets ideas.

• LSTM: generate text
  • Character by character
  • Word by word, using word embeddings
  • With restrictions
  • With parse trees
• LSTM for algorithms
  • Thresholding
  • Neural net Turing machines
• Rotational invariance, etc.
• Smoothness regularization in convolution kernels
• Get familiar with Tensorboard.
• Visualizing neural nets.
• Sparsification
• Reimplementations
  • Deep-fried convnets
  • Fooling neural nets
  • Deep reinforcement learning with Atari games
• “Train twice and put together”: independent ways of doing things (cf. boosting)
• Check claims of suspiciously good performance (ex. learning to execute…)