Course Content

Introduction to TensorFlow

1. Tensors

Welcome to TensorFlow Introduction to Tensors Tensor Properties Applications of Tensors Batches Creating Tensors Data Types Basic Operations: Arithmetic Basic Operations: Linear Algebra Challenge: Creating a Neural Network Layer Transformations Reduction Operations Quiz: TensorFlow Basics

2. Basics of TensorFlow

Gradient Tape Graph Execution Neural Network Implementation Quiz: Conclusion Summary

Graph Execution

Function Decorator

A Function Decorator is a tool that 'wraps' around a function to modify its behavior. In TensorFlow, the most commonly used decorator is @tf.function, which converts a Python function into a TensorFlow graph.

Purpose of @tf.function

The primary purpose of using decorators like @tf.function is to optimize computations. When a function is decorated with @tf.function, TensorFlow converts the function into a highly efficient graph that can be executed much faster, particularly for complex operations. This conversion enables TensorFlow to apply optimizations and exploit parallelism, which is crucial for performance in machine learning tasks.

Example

Let's go through an example to understand better.


              1234567891011
            
import tensorflow as tf

# Define a simple function and decorate it with `@tf.function`
@tf.function
def compute_area(radius):
    return 3.1415 * radius ** 2

# Call the function
area = compute_area(tf.constant(3.0))

print(f"The area is: {area.numpy()}")

In this code, compute_area() is converted into a TensorFlow graph, making it run faster and more efficiently.

How Graph Execution Works?

TensorFlow operates in two modes: Eager Execution and Graph Execution. By default, TensorFlow runs in Eager Execution mode, which means operations are executed as they are defined, providing a flexible and intuitive interface. However, Eager Execution can be less efficient for complex computations and large-scale models.

This is where @tf.function and Graph Execution come into play. When you use the @tf.function decorator on a function, TensorFlow converts that function into a static computation graph of operations.

Optimization Techniques

Graph Optimization: TensorFlow optimizes the graph by pruning unused nodes, merging duplicate subgraphs, and performing other graph-level optimizations. This results in faster execution and reduced memory usage;
Faster Execution: Graphs are executed faster than eager operations because they reduce the Python overhead. Python is not involved in the execution of the graph, which eliminates the overhead of Python interpreter calls;
Parallelism and Distribution: Graphs enable TensorFlow to easily identify opportunities for parallelism and distribute computations across multiple devices, such as CPUs and GPUs;
Caching and Reuse: When a function decorated with @tf.function is called with the same input signature, TensorFlow reuses the previously created graph, avoiding the need to recreate the graph, which saves time.

Example with Gradient Tape


              1234567891011
            
import tensorflow as tf

@tf.function
def compute_gradient(x):
    with tf.GradientTape() as tape:
        y = x * x * x
    return tape.gradient(y, x)

x = tf.Variable(3.0)
grad = compute_gradient(x)
print(f"The gradient at x = {x.numpy()} is {grad.numpy()}")

In this example, compute_gradient is a function that calculates the gradient of y = x^3 at a given point x. The @tf.function decorator ensures that the function is executed as a TensorFlow graph.

Example with Conditional Logic


              1234567891011121314
            
import tensorflow as tf

@tf.function
def compute_gradient_conditional(x):
    with tf.GradientTape() as tape:
        if tf.reduce_sum(x) > 0:
            y = x * x
        else:
            y = x * x * x
    return tape.gradient(y, x)

x = tf.Variable([-2.0, 2.0])
grad = compute_gradient_conditional(x)
print(f"The gradient at x = {x.numpy()} is {grad.numpy()}")

In this example, the function computes different gradients based on a condition. TensorFlow's @tf.function not only converts the static computation graph but also handles dynamic elements like conditionals and loops effectively.

Task

Swipe to show code editor

In this task, you will compare the execution times of two TensorFlow functions that perform matrix multiplication: one with the @tf.function decorator and one without it.

Steps

Define matrix_multiply_optimized function ensuring that it includes the @tf.function decorator.
Complete both functions by calculating the mean of the resulting matrices.
Generate two uniformly distributed random matrices using TensorFlow's random matrix generation functions.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Section 2. Chapter 2

Graph Execution

Function Decorator

Purpose of @tf.function

Example

Let's go through an example to understand better.


              1234567891011
            
import tensorflow as tf

# Define a simple function and decorate it with `@tf.function`
@tf.function
def compute_area(radius):
    return 3.1415 * radius ** 2

# Call the function
area = compute_area(tf.constant(3.0))

print(f"The area is: {area.numpy()}")

In this code, compute_area() is converted into a TensorFlow graph, making it run faster and more efficiently.

How Graph Execution Works?

Optimization Techniques

Graph Optimization: TensorFlow optimizes the graph by pruning unused nodes, merging duplicate subgraphs, and performing other graph-level optimizations. This results in faster execution and reduced memory usage;
Faster Execution: Graphs are executed faster than eager operations because they reduce the Python overhead. Python is not involved in the execution of the graph, which eliminates the overhead of Python interpreter calls;
Parallelism and Distribution: Graphs enable TensorFlow to easily identify opportunities for parallelism and distribute computations across multiple devices, such as CPUs and GPUs;
Caching and Reuse: When a function decorated with @tf.function is called with the same input signature, TensorFlow reuses the previously created graph, avoiding the need to recreate the graph, which saves time.

Example with Gradient Tape


              1234567891011
            
import tensorflow as tf

@tf.function
def compute_gradient(x):
    with tf.GradientTape() as tape:
        y = x * x * x
    return tape.gradient(y, x)

x = tf.Variable(3.0)
grad = compute_gradient(x)
print(f"The gradient at x = {x.numpy()} is {grad.numpy()}")

Example with Conditional Logic


              1234567891011121314
            
import tensorflow as tf

@tf.function
def compute_gradient_conditional(x):
    with tf.GradientTape() as tape:
        if tf.reduce_sum(x) > 0:
            y = x * x
        else:
            y = x * x * x
    return tape.gradient(y, x)

x = tf.Variable([-2.0, 2.0])
grad = compute_gradient_conditional(x)
print(f"The gradient at x = {x.numpy()} is {grad.numpy()}")

Task

Swipe to show code editor

In this task, you will compare the execution times of two TensorFlow functions that perform matrix multiplication: one with the @tf.function decorator and one without it.

Steps

Define matrix_multiply_optimized function ensuring that it includes the @tf.function decorator.
Complete both functions by calculating the mean of the resulting matrices.
Generate two uniformly distributed random matrices using TensorFlow's random matrix generation functions.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Section 2. Chapter 2

Graph Execution

Function Decorator

Purpose of @tf.function

Example

Let's go through an example to understand better.


              1234567891011
            
import tensorflow as tf

# Define a simple function and decorate it with `@tf.function`
@tf.function
def compute_area(radius):
    return 3.1415 * radius ** 2

# Call the function
area = compute_area(tf.constant(3.0))

print(f"The area is: {area.numpy()}")

In this code, compute_area() is converted into a TensorFlow graph, making it run faster and more efficiently.

How Graph Execution Works?

Optimization Techniques

Graph Optimization: TensorFlow optimizes the graph by pruning unused nodes, merging duplicate subgraphs, and performing other graph-level optimizations. This results in faster execution and reduced memory usage;
Faster Execution: Graphs are executed faster than eager operations because they reduce the Python overhead. Python is not involved in the execution of the graph, which eliminates the overhead of Python interpreter calls;
Parallelism and Distribution: Graphs enable TensorFlow to easily identify opportunities for parallelism and distribute computations across multiple devices, such as CPUs and GPUs;
Caching and Reuse: When a function decorated with @tf.function is called with the same input signature, TensorFlow reuses the previously created graph, avoiding the need to recreate the graph, which saves time.

Example with Gradient Tape


              1234567891011
            
import tensorflow as tf

@tf.function
def compute_gradient(x):
    with tf.GradientTape() as tape:
        y = x * x * x
    return tape.gradient(y, x)

x = tf.Variable(3.0)
grad = compute_gradient(x)
print(f"The gradient at x = {x.numpy()} is {grad.numpy()}")

Example with Conditional Logic


              1234567891011121314
            
import tensorflow as tf

@tf.function
def compute_gradient_conditional(x):
    with tf.GradientTape() as tape:
        if tf.reduce_sum(x) > 0:
            y = x * x
        else:
            y = x * x * x
    return tape.gradient(y, x)

x = tf.Variable([-2.0, 2.0])
grad = compute_gradient_conditional(x)
print(f"The gradient at x = {x.numpy()} is {grad.numpy()}")

Task

Swipe to show code editor

In this task, you will compare the execution times of two TensorFlow functions that perform matrix multiplication: one with the @tf.function decorator and one without it.

Steps

Define matrix_multiply_optimized function ensuring that it includes the @tf.function decorator.
Complete both functions by calculating the mean of the resulting matrices.
Generate two uniformly distributed random matrices using TensorFlow's random matrix generation functions.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Function Decorator

Purpose of @tf.function

Example

Let's go through an example to understand better.


              1234567891011
            
import tensorflow as tf

# Define a simple function and decorate it with `@tf.function`
@tf.function
def compute_area(radius):
    return 3.1415 * radius ** 2

# Call the function
area = compute_area(tf.constant(3.0))

print(f"The area is: {area.numpy()}")

In this code, compute_area() is converted into a TensorFlow graph, making it run faster and more efficiently.

How Graph Execution Works?

Optimization Techniques

Graph Optimization: TensorFlow optimizes the graph by pruning unused nodes, merging duplicate subgraphs, and performing other graph-level optimizations. This results in faster execution and reduced memory usage;
Faster Execution: Graphs are executed faster than eager operations because they reduce the Python overhead. Python is not involved in the execution of the graph, which eliminates the overhead of Python interpreter calls;
Parallelism and Distribution: Graphs enable TensorFlow to easily identify opportunities for parallelism and distribute computations across multiple devices, such as CPUs and GPUs;
Caching and Reuse: When a function decorated with @tf.function is called with the same input signature, TensorFlow reuses the previously created graph, avoiding the need to recreate the graph, which saves time.

Example with Gradient Tape


              1234567891011
            
import tensorflow as tf

@tf.function
def compute_gradient(x):
    with tf.GradientTape() as tape:
        y = x * x * x
    return tape.gradient(y, x)

x = tf.Variable(3.0)
grad = compute_gradient(x)
print(f"The gradient at x = {x.numpy()} is {grad.numpy()}")

Example with Conditional Logic


              1234567891011121314
            
import tensorflow as tf

@tf.function
def compute_gradient_conditional(x):
    with tf.GradientTape() as tape:
        if tf.reduce_sum(x) > 0:
            y = x * x
        else:
            y = x * x * x
    return tape.gradient(y, x)

x = tf.Variable([-2.0, 2.0])
grad = compute_gradient_conditional(x)
print(f"The gradient at x = {x.numpy()} is {grad.numpy()}")

Task

Swipe to show code editor

In this task, you will compare the execution times of two TensorFlow functions that perform matrix multiplication: one with the @tf.function decorator and one without it.

Steps

Define matrix_multiply_optimized function ensuring that it includes the @tf.function decorator.
Complete both functions by calculating the mean of the resulting matrices.
Generate two uniformly distributed random matrices using TensorFlow's random matrix generation functions.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Section 2. Chapter 2

Switch to desktop for real-world practiceContinue from where you are using one of the options below