Monday, December 11, 2017

Rectified Linear Unit Activation Function In Deep Learning


Sigmoid Activation function provides S shape curve graph and the values are always between 0 and 1. Sigmoid function converts the large negative values to 0 and high positive values to 1.

But with Sigmoid Activation function, we have major drawback is that it’s value is 0 near 0 and 1 which means during the back propagation method the weights values will never ever change and we will not get any output. Secondly, Sigmoid Activation Function faces the gradient vanishing problem at 0 and 1.

Below is the image of Sigmoid Curve:


To get rid from the above issues, we can use Rectified Linear Unit Activation Function which is also known as RELU. RELU function has range between 0 and Infinity. Hence Sigmoid Activation function can be used to predict the values between 0 and 1 whereas can be used to model real positive number. The best of RELU is that whenever we increase the input values of X, the gradient also changes.

RELU can be defined with the simple below mentioned Mathematical notation:
RELU(x) = MAX (0, x)
The functions says if the input value is 0 or less than 0, RELU will return 0 else it will return the input value of x.

Below is the image of Sigmoid Curve:


Click Here To Read Rest Of The Post...

Saturday, December 9, 2017

What is Data Pre-Processing In Machine Learning?


Data Pre-Processing means adding some type of mathematical operation without losing the content of the data. Let’s take example, we want to do the dimensionality reduction so that we can visualize the data more efficiently in 2D graph. It means we need to have some kind of pre-processing of on same data so that we can drop some data without losing it’s actual meaning. Let’s take another example from the previous post and get deeper understanding of data pre-processing. Below is the matrix or dataset which has price of pizza in INR which is fully dependent on it’s size. In India, the price is in INR, but for United States it will be Dollars and for Dubai it must be in Dirham. For size in India it is in inches, but in United States it might be in Centimeters. But if we are developing some kind of relation in that case we can pre-process the data and fit everything between 0 to 1. By doing this, our dependency on INR, Dollars and Inches is completely gone off.
In this case, we can take the maximum and minimum value of every column and apply the below formula on the existing data. By doing this we get the new form of data whose values will be lying between 0 and 1 without losing its actual importance.

New Pre-Process data will look alike below


The advantage of pre-processing of data is that now we can fit anything between 0 and 1 which means unit square. Below is the comparison of before pre-processing and after pre-processing of that, now we can have good visibility and everything is well fitted in unit square.

Click Here To Read Rest Of The Post...

Wednesday, December 6, 2017

Architecture Of Predicting Hand Written Digits


Logistic regression and Softmax Activation functions are the most important functions which really help us to predict the handwritten digits from the MNIST dataset. I am using Keras to predict the numbers but for this post, I have created a high level diagram which helps everyone to understand which function is required at which layer to predict the MNIST hand written digit numbers.

Click Here To Read Rest Of The Post...