Spatial Transformer Networks In Deep Learning | Attention Model Arbitrary Region by Dr Arshad Afridi
In this video we discuss Spatial Transformer Networks In Deep Learning. The attention model for arbitrary region is explained.
Attention mechanism from Show, Attend, and Tell only lets us softly attend to fixed grid positions … can we do better?
- Read text, generate handwriting using an RNN
- Attend to arbitrary regions of the output by Which are real and which are generated? predicting params of a mixture model.
Attending to Arbitrary Regions: DRAW
Classify images by attending to arbitrary regions of the input.
Generate images by attending to arbitrary regions of the output
Attending to Arbitrary Regions: Spatial Transformer Networks
Attention mechanism similar to DRAW, but easier to explain
Idea: Function mapping
pixel coordinates (xt, yt) of output to pixel coordinates
(xs, ys) of input.
Repeat for all pixels in output to get a sampling grid
Then use bilinear interpolation to compute output. Network attends to input by predicting 𝜃.
Grid generator uses 𝜃 to compute sampling grid
A small Localization network predicts transform 𝜃.
Sampler uses bilinear interpolation to produce output
Insert spatial transformers into a classification network and it learns to attend and transform the input
Differentiable “attention /
transformation” module
Soft attention:
Easy to implement: produce distribution over input locations, reweight features and feed as input
Attend to arbitrary input locations using spatial transformer networks
Hard attention:
Attend to a single input location Can’t use gradient descent!
Need reinforcement learning!
Spatial Transformer Networks, Spatial Transformer Networks deep learning, Spatial Transformer Networks rnn, Attention Model Arbitrary region, Attending to Arbitrary Regions DRAW, Function mapping Spatial Transformer Networks, sampling grid, How Spatial Transformer Networks works, what is Spatial Transformer Networks, Spatial Transformer Networks grid generator, Soft attention gradient descent, Hard attention Need reinforcement learning, bilinear interpolation, spatial transformers classification
#SpatialTransformerNetworks, #attentionModel #rnn