When working with transfer learning in computer vision, you will often choose between feature extraction and fine-tuning.

• Feature Extraction: use the pre-trained model as a fixed feature extractor. Freeze all convolutional layers and only train new output layers;
• Fine-Tuning: unfreeze some top layers of the pre-trained model and train them along with the new head. This allows the model to adapt to the new dataset.

Layer freezing is essential for controlling which parts of the model are updated during training.

# Load the pre-trained VGG16 model without its top classification layers
from tensorflow.keras.applications import VGG16
base_model = VGG16(weights='imagenet', include_top=False, input_shape=(224, 224, 3))

Load the VGG16 architecture with weights pre-trained on ImageNet. Setting include_top=False removes the default dense layers, leaving only the convolutional base that acts as a powerful feature extractor.

# Freeze all layers to prevent them from being updated during training
for layer in base_model.layers:
    layer.trainable = False

In feature extraction, we keep the pre-trained convolutional filters unchanged. By setting layer.trainable = False, we tell Keras to freeze the layers — the network will reuse existing features like edges, shapes, and textures.

from tensorflow.keras import layers, models

# Build a new model with a custom classification head
model = models.Sequential([
    base_model,
    layers.Flatten(),
    layers.Dense(256, activation='relu'),
    layers.Dense(10, activation='softmax')  # Example: 10 output classes
])

Attaching fully connected head to the frozen base model.

• Flatten() converts the feature maps into a 1D vector;
• Dense(256, activation='relu') learns new task-specific patterns;
• The final Dense(10, activation='softmax') outputs probabilities for 10 classes.

from tensorflow.keras import optimizers

# Compile the model with a standard learning rate
model.compile(optimizer='adam', 
              loss='categorical_crossentropy', 
              metrics=['accuracy'])

Compiling the model with the Adam optimizer and a typical loss for multi-class problems — categorical crossentropy. At this stage, only the newly added dense layers are trainable.

# Unfreeze the top 4 layers of the convolutional base for fine-tuning
for layer in base_model.layers[-4:]:
    layer.trainable = True

Once the new head is trained, we can unfreeze a few of the last convolutional layers. This allows the network to adjust higher-level features to the new dataset, improving performance without retraining everything.

# Recompile with a smaller learning rate to avoid destroying pre-trained weights
model.compile(optimizer=optimizers.Adam(learning_rate=1e-5),
              loss='categorical_crossentropy', 
              metrics=['accuracy'])

Fine-tuning requires a smaller learning rate (1e-5) — this makes weight updates subtle and prevents overwriting pre-trained representations. Now the model is ready for refined training.

In the code above, you first freeze all layers for feature extraction, then selectively unfreeze the last 4 layers for fine-tuning. Using a lower learning rate helps prevent large weight updates in the pre-trained layers.