Recurrent Neural Networks

focuses on sequences of information and identifying missing gaps in said sequences
all forms of data have dependency of sequence
processes sequential data: order matters in data
feed back in values in hidden layers as their inputs to find cause and effect relationships
do not have a fixed amount of computational steps (no. of layers) but rather RNNs operate or identify non-linear relationships over multiple sequences
Many to Many Relationships:
if training neural nets is optimizing functions, training recurrent neural nets is optimizing over programs
countless relations for sequencing over inputs and outputs

One to Many: used when you have one input and want to map it to multiple outputs (image description captioning)
Many to One: input mapped to single output or binary outputs (sentiment analysis)
Many to Many Mode: sequence to sequence takes in input sequence and mapped to an output sequence (text translation from different languages)

some common applications include NLP as text and language is sequential data
question answering, document classification, machine translation, text summarization, and much more
- important to note that original RNNs suffered from vanishing gradients where the partial derivative of the error function with respect to the current weight was equivalent to the parameter updates despite the learning rates
  - This means that the importance of sequential data is ultimately lost since the gradients do not update accordingly to the data style
variation of RNN known as Long Short Term Memory networks (LSTM) is more effective