Neural net learns dictionaries: run experiments. I found nothing unexpected: closeby initialization does converge. The farther away, the farther away the final \(A\) is. (But I think it should be a _/ shape—where is the cusp?)
Are the different dictionaries related by a linear transformation? It seems not.