Types of Neural Network Architectures
In this video we are going to describe various kinds of architectures for neural networks. What I mean by an architecture, is the way in which the neurons are connected together. By far the commonest type of architecture in practical applications is a feet forward neural network where the information comes into the input units and flows in one direction through hidden layers until each reaches the output units. A much more interesting kind architecture is a recurrent neural network in which information can flow round in cycles. These networks can remember information for a long time. They can exhibit all sorts of interesting oscillations but they are much more difficult to train in part because they are so much more complicated in what they can do. Recently, however, people have made a lot of progress in training recurrent neural networks, and they can now do some fairly impressive things. The last kind of architecture that I'll describe is a symmetrically-connected network, one in which the weights are the same in both directions between two units...