Summary  
This chapter covers how to implement the forward pass of a single neural network layer by defining weight and bias variables, computing the weighted sum via matrix multiplication, adding the bias, and applying an activation function.  

General domain of usage  
Deep learning model implementation

# Single Neural Network Layer

In a basic feed-forward neural network, the output of a neuron in a layer is calculated using the **weighted sum of its inputs**, passed through an **activation function**. This can be represented as:

$$y=\sigma(W \cdot x + b)$$

Where:
- $$y$$: output of the neuron;
- $$W$$: a matrix representing the weights associated with connections to the neuron;
- $$x$$: a column matrix (or vector) representing the input values to the neuron;
- $$b$$: a scalar value;
- $$\sigma$$: an activation function, such as the sigmoid, ReLU or softmax function.

To achieve the best performance, all calculations are performed using **matrices**. We will cope with this task in the same way.

import unittest
import tensorflow as tf
import numpy as np


# Dynamic test helper required by Codefinity
def _dynamic_test(test_case, condition, ok_msg, fail_msg):
    if condition:
        test_case._testMethodName = ok_msg
        test_case.assertTrue(True)
    else:
        test_case._testMethodName = fail_msg
        test_case.fail(fail_msg)


class TestUserCode(unittest.TestCase):

    # 1. Test transpose of W
    def test_weight_transpose(self):
        import user_code

        W_original = np.array([[0.5, 0.4, 0.7],
                               [0.1, 0.9, 0.5]], dtype=np.float64)
        expected = W_original.T

        cond = False
        try:
            W = user_code.W
            cond = isinstance(W, tf.Tensor) and np.allclose(W.numpy(), expected)
        except Exception:
            cond = False

        _dynamic_test(
            self,
            cond,
            "Weight matrix W is transposed correctly.",
            "Expected W to be transposed to shape (3,2)."
        )

    # 2. Test weighted sum = I @ W
    def test_weighted_sum(self):
        import user_code

        I = np.array([[0.5, 0.6, 0.1]], dtype=np.float64)
        W = np.array([[0.5, 0.4, 0.7],
                      [0.1, 0.9, 0.5]], dtype=np.float64).T

        expected = I @ W

        cond = False
        try:
            ws = user_code.weighted_sum.numpy()
            cond = ws.shape == (1, 2) and np.allclose(ws, expected)
        except Exception:
            cond = False

        _dynamic_test(
            self,
            cond,
            "Weighted sum is computed correctly.",
            "Expected weighted_sum = I @ W with correct shape (1,2)."
        )

    # 3. Test biased sum = weighted_sum + b
    def test_biased_sum(self):
        import user_code

        I = np.array([[0.5, 0.6, 0.1]], dtype=np.float64)
        W = np.array([[0.5, 0.4, 0.7],
                      [0.1, 0.9, 0.5]], dtype=np.float64).T
        weighted = I @ W
        expected = weighted + 0.1  # bias is scalar

        cond = False
        try:
            bs = user_code.biased_sum.numpy()
            cond = bs.shape == (1, 2) and np.allclose(bs, expected)
        except Exception:
            cond = False

        _dynamic_test(
            self,
            cond,
            "Biased sum is computed correctly by adding bias.",
            "Expected biased_sum = weighted_sum + bias with shape (1,2)."
        )

    # 4. Test sigmoid activation
    def test_neuron_output(self):
        import user_code

        I = np.array([[0.5, 0.6, 0.1]], dtype=np.float64)
        W = np.array([[0.5, 0.4, 0.7],
                      [0.1, 0.9, 0.5]], dtype=np.float64).T
        weighted = I @ W
        biased = weighted + 0.1
        expected = 1 / (1 + np.exp(-biased))

        cond = False
        try:
            out = user_code.neuron_output.numpy()
            cond = out.shape == (1, 2) and np.allclose(out, expected)
        except Exception:
            cond = False

        _dynamic_test(
            self,
            cond,
            "Sigmoid activation applied correctly.",
            "Expected neuron_output = sigmoid(biased_sum)."
        )

    # Verify tf.sigmoid is actually used in code
    def test_uses_sigmoid(self):
        cond = False
        try:
            with open("user_code.py", "r") as f:
                src = f.read()
            cond = "tf.sigmoid" in src
        except Exception:
            cond = False

        _dynamic_test(
            self,
            cond,
            "The code uses tf.sigmoid() as required.",
            "Expected to use tf.sigmoid() activation function."
        )


if __name__ == "__main__":
    unittest.main()


test_code.py

Dive deep into the world of TensorFlow with our course, designed to give you a robust understanding of its core components. Begin with an exploration of tensors and the basics of TensorFlow framework. By the course's end, you'll have honed the skills to build tensor-driven systems, including crafting a basic neural network. Equip yourself with the knowledge to harness TensorFlow's full potential and set the foundation for advanced deep learning pursuits.

You will gain a foundational understanding of TensorFlow's primary components - Tensors. You'll delve into the nature and applications of tensors, familiarize yourself with tensor properties, and acquire knowledge in essential mathematical operations.

You'll learn how TensorFlow operates and the ways to improve its performance. By the end of this module, you'll be well-equipped to implement basic neural networks or other tensor calculations, using only the TensorFlow library without any extras.

Creating a Neural Network Layer

Single Neural Network Layer

解答