What is a Neuron in Artificial Neural Networks?
This article covers the basic definition of an artificial neuron and further gives a refresher on the related topics.
What is a neuron?
- Neuron is a mathematical function that computes the weighted average of its inputs and then this sum is passed through a non-linear function (often called as activation function) together with the bias. Few activation functions to mention are sigmoid, ReLU, Tanh etc.
- A single neuron is basically a logistic regression.
- Neurons can be arranged in a multiple layer to form a neural network. Each neuron serves as an input to the neuron of next layer.
What is weight and bias in a neural network?
- In Neural network, some inputs are provided to an artificial neuron, and with each input a weight is associated. Weight increases the steepness of activation function. This means weight decide how fast the activation function will trigger whereas bias is used to delay the triggering of the activation function.
- The values of weights are bias are chosen in a way that the predicted target value is as close to the actual target value. Generally, this is achieved after several iterations.
Mathematical functions
- Linear functions are those whose graph is a straight line. A linear function has the following form y = f(x) = a + bx. A linear function has one independent variable and one dependent variable. The independent variable is x and the dependent variable is y.
- Non-Linear Functions Often a linear function cannot explain the relationship between variables. In such cases a non-linear function must be used. These could be power, quadratic, polynomial, rational, exponential, logarithmic, and sinusoidal. Non-linear means the graph is not a straight line. The graph of a non-linear function is a curved line.
Linear and logistic regression
- What is a linear regression?
Linear regression attempts to model the relationship between two variables by fitting a linear function to observed data. It is able to predict continuous values. In the example below, body weight can be predicted when the height is know and vice versa.
- What is a logistic regression?
Logistic regression is a classification algorithm used to assign observations to a discrete set of classes. Unlike linear regression which outputs continuous number values, logistic regression transforms its output using the logistic sigmoid function to return a probability value which can then be mapped to two or more discrete classes. In the example below, cholesterol levels can actually state the probability of having a heart disease or not.
Can a neural network predict a linear function?
- A neural network can predict linear function but that will be an overkill. Linear regression should be used for it.
References -
http://www.columbia.edu/itc/sipa/math/linear.html
https://www.ck12.org/algebra/identify-types-of-linear-and-nonlinear-functions/lesson/Linear-and-Nonlinear-Function-Distinction-MSM7/
https://stats.stackexchange.com/users/61836/matteo
http://www.stat.yale.edu/Courses/1997-98/101/linreg.htm
http://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704-EP713_MultivariableMethods/
https://towardsdatascience.com/multi-layer-neural-networks-with-sigmoid-function-deep-learning-for-rookies-2-bf464f09eb7f
https://towardsdatascience.com/understanding-the-roc-and-auc-curves-a05b68550b69
https://www.geeksforgeeks.org/effect-of-bias-in-neural-network/
https://www.quantamagazine.org/neural-dendrites-reveal-their-computational-power-20200114/
