tape.gradient() computes the derivative of a target value with respect to one or more source tensors, using the operations recorded by a GradientTape.

Syntax

tape.gradient(target, sources)

Deep Dive Course

Called after a GradientTape's with block has closed, gradient(target, sources) walks backward through the recorded operations, applying the chain rule automatically to compute how much target would change for a small change in each tensor in sources — passing a single tensor as sources returns a single gradient tensor, while passing a list returns a matching list of gradients, one per source. If target doesn't actually depend on a given source through any recorded operation, the returned gradient for that source is None, which is a common and important signal that something in the computation graph isn't connected the way you expected.

1Understanding tape.gradient()

💡

A gradient() result of None for a source you expected a real gradient for is a strong signal of a bug — it means target never actually depended on that source through any operation recorded on the tape, often caused by accidentally breaking the tape's tracking with a non-TensorFlow operation, like a raw NumPy call, somewhere in between.

editor.html

import tensorflow as tf

x = tf.Variable(2.0)
y = tf.Variable(3.0)
with tf.GradientTape() as tape:
    z = x * y + y ** 2
grads = tape.gradient(z, [x, y])
print([g.numpy() for g in grads])

localhost:3000

2Practical Example

Here is a real-world application of tape.gradient() showing how it is used in production TensorFlow code.

editor.html

import tensorflow as tf

x = tf.Variable(2.0)
unrelated = tf.Variable(5.0)
with tf.GradientTape() as tape:
    y = x ** 2
grad = tape.gradient(y, unrelated)
print(grad)

localhost:3000

3Best Practices

Follow these guidelines when working with tape.gradient():

1. Check for an unexpected None result from gradient() as a first debugging step whenever a custom training loop's gradients look wrong or missing

2. Pass a list of source tensors to compute multiple gradients in a single gradient() call, rather than calling it separately per tensor, which is both more efficient and the standard pattern

3. Call gradient() only once per tape by default, since a GradientTape releases its recorded resources after a single call unless persistent=True is passed to its constructor

⚠️

Tip: A gradient() result of None for a source you expected a real gradient for is a strong signal of a bug — it means target never actually depended on that source through any operation recorded on the tape, often caused by accidentally breaking the tape's tracking with a non-TensorFlow operation, like a raw NumPy call, somewhere in between.

editor.html

import tensorflow as tf

x = tf.Variable(2.0)
y = tf.Variable(3.0)
with tf.GradientTape() as tape:
    z = x * y + y ** 2
grads = tape.gradient(z, [x, y])
print([g.numpy() for g in grads])

localhost:3000

Examples

Example 01Basic Usage

import tensorflow as tf

x = tf.Variable(2.0)
y = tf.Variable(3.0)
with tf.GradientTape() as tape:
    z = x * y + y ** 2
grads = tape.gradient(z, [x, y])
print([g.numpy() for g in grads])

Example 02Advanced Example

import tensorflow as tf

x = tf.Variable(2.0)
unrelated = tf.Variable(5.0)
with tf.GradientTape() as tape:
    y = x ** 2
grad = tape.gradient(y, unrelated)
print(grad)

Best Practices

Check for an unexpected None result from gradient() as a first debugging step whenever a custom training loop's gradients look wrong or missing
Pass a list of source tensors to compute multiple gradients in a single gradient() call, rather than calling it separately per tensor, which is both more efficient and the standard pattern
Call gradient() only once per tape by default, since a GradientTape releases its recorded resources after a single call unless persistent=True is passed to its constructor

Interview Question

Why does tape.gradient() return None for a source tensor rather than raising an error, when that source wasn't actually used to compute the target?▼

Hint: Think about whether 'this value has no effect on the output' and 'this is a genuine error' are really the same thing from TensorFlow's perspective.

From a purely mathematical standpoint, if target genuinely doesn't depend on a given source tensor at all, computing the derivative of target with respect to that source, in the sense of how target changes as source changes, is a perfectly well-defined question with a perfectly well-defined answer: it doesn't change at all, so the derivative is conceptually zero, or more precisely, undefined-because-disconnected, which TensorFlow represents as None rather than an arithmetic zero to distinguish it from a genuine zero-valued gradient on a tensor that was connected but happened to have a zero derivative at that point. Raising a hard error instead would actually be surprising and unhelpful in legitimate situations, like passing a list of several possible source tensors to a shared gradient() call, where you know in advance that only some of them actually influenced a particular target — treating an unused source as a normal, expected outcome, represented by None, rather than a fatal error, keeps that valid pattern from breaking, while still leaving None as a clear, checkable signal you can use for debugging genuinely unexpected disconnections.

Exercises

MediumPractice using tape.gradient() in a real scenario.

View Solution

import tensorflow as tf

x = tf.Variable(2.0)
y = tf.Variable(3.0)
with tf.GradientTape() as tape:
    z = x * y + y ** 2
grads = tape.gradient(z, [x, y])
print([g.numpy() for g in grads])

Frequently Asked Questions

Why does tape.gradient() return None for a source tensor rather than raising an error, when that source wasn't actually used to compute the target?▼

Related Functions

tf-gradienttapeoptimizer-apply-gradientsmodel-fit