tf.keras.losses.CategoricalCrossentropy() measures the difference between a predicted probability distribution and a true one-hot encoded label, the standard loss for multi-class classification.

Syntax

tf.keras.losses.CategoricalCrossentropy()

Deep Dive Course

Categorical crossentropy computes the negative log of the predicted probability assigned to the true class, expecting labels in one-hot encoded form, a vector of all zeros except a single 1 at the true class's position, and predictions as a probability distribution, typically the output of a softmax activation. Because of the negative log, a prediction that's confidently correct produces a very small loss, while a prediction that's confidently wrong produces a very large one, which is exactly the strong, well-calibrated training signal a classifier needs. When labels are given as plain integer class indices instead of one-hot vectors, SparseCategoricalCrossentropy computes the mathematically identical loss without requiring the one-hot conversion step.

1Understanding losses.CategoricalCrossentropy()

💡

Use SparseCategoricalCrossentropy instead of CategoricalCrossentropy whenever your labels are plain integers, like 3 for the fourth class, rather than one-hot vectors — the underlying math is identical, it just saves you from manually one-hot encoding the labels first.

editor.html

import tensorflow as tf

cce = tf.keras.losses.CategoricalCrossentropy()
y_true = [[0, 1, 0]]
y_pred = [[0.05, 0.9, 0.05]]
print(round(cce(y_true, y_pred).numpy(), 4))

localhost:3000

2Practical Example

Here is a real-world application of losses.CategoricalCrossentropy() showing how it is used in production TensorFlow code.

editor.html

import tensorflow as tf

scce = tf.keras.losses.SparseCategoricalCrossentropy()
y_true = [1]
y_pred = [[0.05, 0.9, 0.05]]
print(round(scce(y_true, y_pred).numpy(), 4))

localhost:3000

3Best Practices

Follow these guidelines when working with losses.CategoricalCrossentropy():

1. Use CategoricalCrossentropy with one-hot encoded labels, or SparseCategoricalCrossentropy with plain integer labels — never mix the two label formats with the wrong loss

2. Make sure the model's final layer uses a softmax activation so its outputs form a valid probability distribution before this loss is applied

3. Pass from_logits=True instead if your model's final layer has no activation and outputs raw logits directly, which is often more numerically stable than applying softmax separately first

⚠️

Tip: Use SparseCategoricalCrossentropy instead of CategoricalCrossentropy whenever your labels are plain integers, like 3 for the fourth class, rather than one-hot vectors — the underlying math is identical, it just saves you from manually one-hot encoding the labels first.

editor.html

import tensorflow as tf

cce = tf.keras.losses.CategoricalCrossentropy()
y_true = [[0, 1, 0]]
y_pred = [[0.05, 0.9, 0.05]]
print(round(cce(y_true, y_pred).numpy(), 4))

localhost:3000

Examples

Example 01Basic Usage

import tensorflow as tf

cce = tf.keras.losses.CategoricalCrossentropy()
y_true = [[0, 1, 0]]
y_pred = [[0.05, 0.9, 0.05]]
print(round(cce(y_true, y_pred).numpy(), 4))

Example 02Advanced Example

import tensorflow as tf

scce = tf.keras.losses.SparseCategoricalCrossentropy()
y_true = [1]
y_pred = [[0.05, 0.9, 0.05]]
print(round(scce(y_true, y_pred).numpy(), 4))

Best Practices

Use CategoricalCrossentropy with one-hot encoded labels, or SparseCategoricalCrossentropy with plain integer labels — never mix the two label formats with the wrong loss
Make sure the model's final layer uses a softmax activation so its outputs form a valid probability distribution before this loss is applied
Pass from_logits=True instead if your model's final layer has no activation and outputs raw logits directly, which is often more numerically stable than applying softmax separately first

Interview Question

Why does CategoricalCrossentropy loss only depend on the predicted probability assigned to the true class, ignoring the predicted probabilities assigned to every other class?▼

Hint: Think about how the one-hot encoded true label interacts with the cross-entropy formula's sum over classes.

The full cross-entropy formula sums, across every class, the true label's value at that class multiplied by the negative log of the predicted probability for that class. Because the true label is one-hot encoded, exactly one class has a true value of 1 and every other class has a true value of 0 — multiplying by those zero values makes every term in the sum vanish except the single term corresponding to the true class, leaving just the negative log of the predicted probability for that one correct class. This means the loss genuinely doesn't care how the remaining probability was distributed among the other, incorrect classes, only how much probability mass the model assigned specifically to the correct answer — though in practice, since a softmax output's probabilities must sum to 1, giving more probability to the true class necessarily takes probability away from the others too.

Exercises

MediumPractice using losses.CategoricalCrossentropy() in a real scenario.

View Solution

import tensorflow as tf

cce = tf.keras.losses.CategoricalCrossentropy()
y_true = [[0, 1, 0]]
y_pred = [[0.05, 0.9, 0.05]]
print(round(cce(y_true, y_pred).numpy(), 4))

Frequently Asked Questions

Why does CategoricalCrossentropy loss only depend on the predicted probability assigned to the true class, ignoring the predicted probabilities assigned to every other class?▼

Related Functions

losses-binarycrossentropylosses-meansquarederrormodel-compile