df.shift(periods) shifts a column's values down (or up) by a given number of positions, commonly used to compare each row against a previous (or future) one.

Syntax

df.shift(periods=1, freq=None)

Deep Dive Course

shift(1) moves every value down by one position, introducing NaN at the top since there's no longer a value to fill the first row, while leaving the index unchanged — this makes subtracting a shifted column from the original a standard, concise way to compute the difference between each row and the one before it, like day-over-day change. A negative periods argument shifts values upward instead, which is useful for comparing each row against a future one, like computing next-day changes.

1Understanding df.shift()

💡

Subtracting a shifted column from the original, like the current price minus the price shifted by one, is the standard idiom for computing period-over-period change in a time series — much simpler and faster than writing a manual loop comparing each row to the previous one.

editor.html

import pandas as pd

df = pd.DataFrame({"price": [100, 105, 103, 110]})
df["prev_price"] = df["price"].shift(1)
print(df)

localhost:3000

2Practical Example

Here is a real-world application of df.shift() showing how it is used in production Pandas code.

editor.html

import pandas as pd

df = pd.DataFrame({"price": [100, 105, 103, 110]})
df["daily_change"] = df["price"] - df["price"].shift(1)
print(df)

localhost:3000

3Best Practices

Follow these guidelines when working with df.shift():

1. Use shift() combined with subtraction or division to compute period-over-period changes, instead of writing a manual loop over rows

2. Use a negative periods value when you need to compare against a future row instead of a past one

3. Remember shift() introduces NaN at the boundary, top or bottom, it shifts away from — handle that expected missing value deliberately rather than as a surprise

⚠️

Tip: Subtracting a shifted column from the original, like the current price minus the price shifted by one, is the standard idiom for computing period-over-period change in a time series — much simpler and faster than writing a manual loop comparing each row to the previous one.

editor.html

import pandas as pd

df = pd.DataFrame({"price": [100, 105, 103, 110]})
df["prev_price"] = df["price"].shift(1)
print(df)

localhost:3000

Examples

Example 01Basic Usage

import pandas as pd

df = pd.DataFrame({"price": [100, 105, 103, 110]})
df["prev_price"] = df["price"].shift(1)
print(df)

Example 02Advanced Example

import pandas as pd

df = pd.DataFrame({"price": [100, 105, 103, 110]})
df["daily_change"] = df["price"] - df["price"].shift(1)
print(df)

Best Practices

Use shift() combined with subtraction or division to compute period-over-period changes, instead of writing a manual loop over rows
Use a negative periods value when you need to compare against a future row instead of a past one
Remember shift() introduces NaN at the boundary, top or bottom, it shifts away from — handle that expected missing value deliberately rather than as a surprise

Interview Question

Why does the first row of a shifted column, shift(1), always contain NaN?▼

Hint: Think about what value would need to exist before the very first row for the shift to have something to fill it with.

shift(1) moves every existing value down by one position, so the value that would need to occupy the very first row is whatever value came before the original first row — but there is no such row, since the first row is the beginning of the data. With nothing to shift into that top position, pandas fills it with NaN, representing the fact that there's genuinely no prior value available for that specific row, which is exactly the correct, meaningful signal that a period-over-period comparison can't be computed at the very start of the series.

Exercises

MediumPractice using df.shift() in a real scenario.

View Solution

import pandas as pd

df = pd.DataFrame({"price": [100, 105, 103, 110]})
df["prev_price"] = df["price"].shift(1)
print(df)

Frequently Asked Questions

Why does the first row of a shifted column, shift(1), always contain NaN?▼

Related Functions

df-rollingdf-resampledf-fillna