The data distribution itself changes over time, requiring continuous re-adaptation of the program, such as predicting the trend of merchandise sales

The complexity of the rules is low, and the problem is small

Task rules will change over time, such as defect detection on the production line

The rules are very complicated or cannot be described, such as face recognition and voice recognition

Backpropagation can only be used in feedforward neural networks

Backpropagation can be combined with gradient descent algorithm to update network weights

Backpropagation passes through the activation function

Back propagation refers to the back propagation of errors through the network

print

Unicode

import

xrange

Overfitting

May be overfitting may be underfitting

Just fit

Underfitting

P(WIX) is a prior probability

P(XIW) 1s a conditional probability

P(W) is the posterior probability

P(X) is the posterior probability

Increase the amount of data

Feature Engineering

Reduce regularization parameters

Add features

Can be used to process sequence data.

Cannot process variable length sequence data.

Unlike convolutional neural networks. parameters of RNN cannot be shared

The units above the hidden layer are not associated With each other.

L1 Regularization can do feature selection

L1 with L2 Regularization can be used for feature selection

L2 Regularization can do feature selection

L1 with L2 Regularization cannot be used for feature selection

Convolutional neural network

Recurrent neural network

Fully connected neural network

All of the above

Different 910 processor

Different 310 processor

GPU

FPGA