Cursos /
Introduction to TensorFlow
Creating a Neural Network Layer
Single Neural Network Layer
In a basic feed-forward neural network, the output of a neuron in a layer is calculated using the weighted sum of its inputs, passed through an activation function. This can be represented as:
output = activation(inputs * weights + bias)
Where:
- weights: A matrix representing the weights associated with connections to the neuron;
- inputs: A column matrix (or vector) representing the input values to the neuron;
- bias: A scalar value;
- activation: An activation function, such as the sigmoid function.
To achieve the best performance, all calculations are performed using matrices. We will cope with this task in the same way.
Tarea
Given weights, inputs, and bias for a single neuron layer, compute its output using matrix multiplication and the sigmoid activation function. Consider a layer with 3 inputs and 2 neurons, taking a single batch containing just one sample.
- Determining Shapes:
- The shape of the input matrix
Ishould have its first dimension representing the number of samples in the batch. Given one sample with 3 inputs, its size will be1x3. - The weight matrix
Wshould have its columns representing input weights for each neuron. So for 2 neurons with 3 inputs, the expected shape is3x2. This isn't the case, so you need to transpose the weight matrix to achieve the required shape.
- The shape of the input matrix
- Matrix Multiplication:
- With the matrices in the correct shape, perform the matrix multiplication.
- Recall that in matrix multiplication, the output is derived from the dot product of each row of the first matrix with each column of the second matrix. Ensure you multiply in the right order.
- Bias Addition:
- Simply perform an element-wise addition of the result from the matrix multiplication with the bias.
- Applying Activation:
- Use the sigmoid activation function on the outcome from the bias addition to get the output of the neuron.
- TensorFlow provides the sigmoid function as
tf.sigmoid().
Note
In the end of the course we'll delve into implementing a complete feed-forward network using TensorFlow. This exercise is setting the foundation for that.
Code Description
# Define layer weights, layer inputs and layer biasHere, we are defining the weights, inputs, and bias for the layer. We are using
tf.Variable() for weights and bias as these are trainable parameters in neural networks, and tf.constant() for the input as it remains unchanged during training. dtype=tf.float64 is set for the best precision.W = tf.Variable([[0.5, 0.4, 0.7], [0.1, 0.9, 0.5]], dtype=tf.float64)This is the weight matrix for the neural layer. Each row represents the weights for one neuron with respect to each input.
I = tf.constant([[0.5, 0.6, 0.1]], dtype=tf.float64)This is the input matrix, representing a single sample with 3 features.
b = tf.Variable(0.1, dtype=tf.float64)This is the bias value. It's a scalar that's added to the weighted sum before passing through the activation function.
W = tf.transpose(W)The weights are organized in rows, but for matrix multiplication, they should be in columns. Therefore, we transpose the matrix. After this, the first column corresponds to the weights of the first neuron, and the second column to those of the second neuron.
weighted_sum = tf.matmul(I, W)Here, we compute the matrix multiplication of the input and the transposed weight matrix. This gives us the weighted sum of the inputs.
Let's break down this matrix multiplication step by step:
Determine the Shape of Resultant Matrix:
The resulting matrix will have the number of rows from the first matrix (matrix
I with shape 1x3) and the number of columns from the second matrix (matrix W with shape 3x2). Thus, the resulting matrix will be of shape 1x2.Matrix Multiplication Process:
For the first element of the resulting matrix (let's call it
S[0][0] for "weighted sum"):I by the first element of the first column of matrix W: 0.5 * 0.5 = 0.25;I by the second element of the first column of matrix W: 0.6 * 0.4 = 0.24;I by the third element of the first column of matrix W: 0.1 * 0.7 = 0.07;0.25 + 0.24 + 0.07 = 0.56.S[0][0] is 0.56.For the second element of the resulting matrix (
S[0][1]):I by the first element of the second column of matrix W: 0.5 * 0.1 = 0.05;I by the second element of the second column of matrix W: 0.6 * 0.9 = 0.54;I by the third element of the second column of matrix W: 0.1 * 0.5 = 0.05;0.05 + 0.54 + 0.05 = 0.64.S[0][1] is 0.64.Thus, the resulting matrix
S from multiplying matrix I and matrix W is:[ [0.56, 0.64] ]biased_sum = weighted_sum + bThe bias is added to the weighted sum. This is a form of affine transformation which shifts the result to achieve better fitting during training.
neuron_output = tf.sigmoid(biased_sum)The sigmoid function squashes the value between 0 and 1, ensuring our output is in a normalized range. It's commonly used in binary classification problems.
¿Todo estuvo claro?
Sección 1. Capítulo 10
Contenido del Curso
Introduction to TensorFlow
2. Basics of TensorFlow
Introduction to TensorFlow
Creating a Neural Network Layer
Single Neural Network Layer
In a basic feed-forward neural network, the output of a neuron in a layer is calculated using the weighted sum of its inputs, passed through an activation function. This can be represented as:
output = activation(inputs * weights + bias)
Where:
- weights: A matrix representing the weights associated with connections to the neuron;
- inputs: A column matrix (or vector) representing the input values to the neuron;
- bias: A scalar value;
- activation: An activation function, such as the sigmoid function.
To achieve the best performance, all calculations are performed using matrices. We will cope with this task in the same way.
Tarea
Given weights, inputs, and bias for a single neuron layer, compute its output using matrix multiplication and the sigmoid activation function. Consider a layer with 3 inputs and 2 neurons, taking a single batch containing just one sample.
- Determining Shapes:
- The shape of the input matrix
Ishould have its first dimension representing the number of samples in the batch. Given one sample with 3 inputs, its size will be1x3. - The weight matrix
Wshould have its columns representing input weights for each neuron. So for 2 neurons with 3 inputs, the expected shape is3x2. This isn't the case, so you need to transpose the weight matrix to achieve the required shape.
- The shape of the input matrix
- Matrix Multiplication:
- With the matrices in the correct shape, perform the matrix multiplication.
- Recall that in matrix multiplication, the output is derived from the dot product of each row of the first matrix with each column of the second matrix. Ensure you multiply in the right order.
- Bias Addition:
- Simply perform an element-wise addition of the result from the matrix multiplication with the bias.
- Applying Activation:
- Use the sigmoid activation function on the outcome from the bias addition to get the output of the neuron.
- TensorFlow provides the sigmoid function as
tf.sigmoid().
Note
In the end of the course we'll delve into implementing a complete feed-forward network using TensorFlow. This exercise is setting the foundation for that.
Code Description
# Define layer weights, layer inputs and layer biasHere, we are defining the weights, inputs, and bias for the layer. We are using
tf.Variable() for weights and bias as these are trainable parameters in neural networks, and tf.constant() for the input as it remains unchanged during training. dtype=tf.float64 is set for the best precision.W = tf.Variable([[0.5, 0.4, 0.7], [0.1, 0.9, 0.5]], dtype=tf.float64)This is the weight matrix for the neural layer. Each row represents the weights for one neuron with respect to each input.
I = tf.constant([[0.5, 0.6, 0.1]], dtype=tf.float64)This is the input matrix, representing a single sample with 3 features.
b = tf.Variable(0.1, dtype=tf.float64)This is the bias value. It's a scalar that's added to the weighted sum before passing through the activation function.
W = tf.transpose(W)The weights are organized in rows, but for matrix multiplication, they should be in columns. Therefore, we transpose the matrix. After this, the first column corresponds to the weights of the first neuron, and the second column to those of the second neuron.
weighted_sum = tf.matmul(I, W)Here, we compute the matrix multiplication of the input and the transposed weight matrix. This gives us the weighted sum of the inputs.
Let's break down this matrix multiplication step by step:
Determine the Shape of Resultant Matrix:
The resulting matrix will have the number of rows from the first matrix (matrix
I with shape 1x3) and the number of columns from the second matrix (matrix W with shape 3x2). Thus, the resulting matrix will be of shape 1x2.Matrix Multiplication Process:
For the first element of the resulting matrix (let's call it
S[0][0] for "weighted sum"):I by the first element of the first column of matrix W: 0.5 * 0.5 = 0.25;I by the second element of the first column of matrix W: 0.6 * 0.4 = 0.24;I by the third element of the first column of matrix W: 0.1 * 0.7 = 0.07;0.25 + 0.24 + 0.07 = 0.56.S[0][0] is 0.56.For the second element of the resulting matrix (
S[0][1]):I by the first element of the second column of matrix W: 0.5 * 0.1 = 0.05;I by the second element of the second column of matrix W: 0.6 * 0.9 = 0.54;I by the third element of the second column of matrix W: 0.1 * 0.5 = 0.05;0.05 + 0.54 + 0.05 = 0.64.S[0][1] is 0.64.Thus, the resulting matrix
S from multiplying matrix I and matrix W is:[ [0.56, 0.64] ]biased_sum = weighted_sum + bThe bias is added to the weighted sum. This is a form of affine transformation which shifts the result to achieve better fitting during training.
neuron_output = tf.sigmoid(biased_sum)The sigmoid function squashes the value between 0 and 1, ensuring our output is in a normalized range. It's commonly used in binary classification problems.
¿Todo estuvo claro?
Sección 1. Capítulo 10
