PYTHON / MATPLOTLIB
Scatter plots and correlation
Plot paired measurements as a cloud of markers, encode extra variables with size and colour, and read a Pearson r off the shape.
What you will learn
- Use ax.scatter(x, y) for unordered paired data instead of ax.plot, which joins rows
- Compute Pearson r from mean-centred deviation products to quantify a straight-line trend
- Encode a third variable with c= plus a colorbar and a fourth with s= marker areas
- Reduce alpha and marker size so dense regions of the cloud stay readable
Understanding Scatter plots and correlation
A scatter plot draws one marker per observation, and the two coordinates are two different measurements taken from the same subject: hours practised and test score for one student, income and life expectancy for one country. Nothing joins the markers, because the rows have no order along the x axis; whatever order they sit in is an accident of how the data was collected. That is exactly the difference from a line plot: ax.plot connects consecutive rows and so invents a path through the data, while ax.scatter leaves every observation standing on its own.
Correlation is a number that summarises the shape of that cloud. Pearson r subtracts each variable's mean, multiplies the paired deviations, and divides by the spread of both variables: points that sit above average on both axes contribute a positive product, points that are above average on one and below on the other contribute a negative one. Dividing by the spreads strips the units away, so r always lands between -1 and 1, reaching exactly ±1 only when every point lies on one straight line.
Because r only measures the straight-line component, it can be badly misleading on its own. A perfectly deterministic U-shaped relationship averages its positive and negative products to zero and reports r = 0, and a single far-off point can drag r from 0.2 to 0.8. Plot the cloud first, then quote the number, and use scatter's s= and c= arguments when you want a third or fourth variable riding along in marker area and colour.
import math
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
hours = [1, 2, 3, 4, 5]
score = [2, 4, 5, 4, 5]
def pearson(xs, ys):
n = len(xs)
mx = sum(xs) / n
my = sum(ys) / n
sxy = sum((a - mx) * (b - my) for a, b in zip(xs, ys))
sxx = sum((a - mx) ** 2 for a in xs)
syy = sum((b - my) ** 2 for b in ys)
return sxy / math.sqrt(sxx * syy)
r = pearson(hours, score)
fig, ax = plt.subplots()
ax.scatter(hours, score, s=60, color="tab:blue", edgecolor="black")
ax.set_xlabel("hours practised")
ax.set_ylabel("test score")
ax.set_title(f"r = {r:.3f}")
fig.savefig("practice.png")
print(f"points plotted: {len(hours)}")
print(f"pearson r: {r:.3f}")A scatter plot shows the shape of the relationship between two measurements, while Pearson r summarises only the straight-line part of that shape.
Worked examples
Least-squares line over the cloud
Fits a straight line by hand and draws it on top of the markers so the trend r describes becomes visible.
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
x = [1, 2, 3, 4, 5]
y = [2, 4, 5, 4, 5]
n = len(x)
mx, my = sum(x) / n, sum(y) / n
sxy = sum((a - mx) * (b - my) for a, b in zip(x, y))
sxx = sum((a - mx) ** 2 for a in x)
slope = sxy / sxx
intercept = my - slope * mx
fig, ax = plt.subplots()
ax.scatter(x, y, zorder=3, label="observations")
line_x = [0.5, 5.5]
ax.plot(line_x, [slope * v + intercept for v in line_x],
color="crimson",
label=f"y = {slope:.2f}x + {intercept:.2f}")
ax.legend()
fig.savefig("fit.png")
print(f"slope: {slope:.2f}")
print(f"intercept: {intercept:.2f}")Example explained
Line 1slope = sxy / sxx reuses the same cross-product that r is built from, so the fit line and r always agree on sign.
Line 2intercept = my - slope * mx forces the line through the point of means, which is where every least-squares line passes.
Line 3line_x holds only two x values because a straight line needs no more; ax.plot interpolates between them.
Line 4zorder=3 on the markers keeps them drawn above the line instead of being hidden under it.
Curved data with r = 0
Shows a relationship that is completely deterministic yet has zero Pearson correlation.
import math
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
x = [-2, -1, 0, 1, 2]
y = [4, 1, 0, 1, 4] # y is exactly x ** 2
n = len(x)
mx, my = sum(x) / n, sum(y) / n
sxy = sum((a - mx) * (b - my) for a, b in zip(x, y))
sxx = sum((a - mx) ** 2 for a in x)
syy = sum((b - my) ** 2 for b in y)
r = sxy / math.sqrt(sxx * syy)
fig, ax = plt.subplots()
ax.scatter(x, y, color="tab:orange")
ax.axhline(my, linestyle=":", color="grey")
ax.set_title(f"perfect dependence, r = {r:.2f}")
fig.savefig("curved.png")
print(f"sxy: {sxy}")
print(f"pearson r: {r:.2f}")Example explained
Line 1The left arm contributes negative deviation products and the right arm positive ones, so sxy cancels to exactly 0.0.
Line 2r is 0 even though y is a pure function of x, because r asks only how well one straight line fits.
Line 3ax.axhline(my) marks the mean of y, making the symmetry above and below it visible.
Line 4Knowing x here tells you y exactly, which is why the plot, not the number, is the honest summary.
Third and fourth variables in colour and size
Maps a category to marker colour through a colormap and a magnitude to marker area.
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
income = [4.2, 6.8, 9.1, 12.4, 15.0]
life = [62, 68, 71, 76, 79]
population = [8, 34, 120, 51, 210]
region = [0, 0, 1, 1, 2]
areas = [p * 3 for p in population]
fig, ax = plt.subplots()
sc = ax.scatter(income, life, s=areas, c=region,
cmap="viridis", alpha=0.7, edgecolor="black")
ax.set_xlabel("income (thousands)")
ax.set_ylabel("life expectancy (years)")
fig.colorbar(sc, ax=ax, label="region code")
fig.savefig("bubbles.png")
print("marker areas:", areas)Example explained
Line 1c=region is a list of numbers, so Matplotlib runs it through the viridis colormap rather than treating it as colours.
Line 2ax.scatter returns the mappable that fig.colorbar needs; without capturing it you cannot label the colour scale.
Line 3s=areas sets marker area in points squared, so the 210 value looks about five times wider than the 8 value, not 26 times.
Line 4alpha=0.7 with a black edge keeps overlapping bubbles distinguishable.
Important notes
The s argument is marker area in points squared, so s=400 is about a 20-point-wide marker and doubling s widens a marker by only about 1.4 times.
Pearson r is unaffected by units or axis limits: converting metres to centimetres, or zooming with set_ylim, changes how steep the cloud looks while leaving r identical.
Common mistakes
Calling ax.plot(x, y) for paired data: the points are joined in row order, producing a zigzag that suggests a path through the data that does not exist, and with no marker argument the observations themselves are invisible.
Writing c=[1, 0, 0] to ask for red: a list of numbers is treated as values to be mapped through the colormap, so the three points come out in three different colours. Use color="red" for one fixed colour.
Reporting r = 0.9 as proof that x causes y, or r = 0.02 as proof the variables are unrelated; r measures straight-line agreement only, and says nothing about direction of causation or about curvature.
Try it yourself
Change, predict, then run
Scatter x = [1, 2, 3, 4, 5, 6] against y = [3, 2, 5, 4, 7, 6], compute Pearson r and put it in the title. Then append the point (20, 60) to both lists, recompute r, and note how far one distant observation moves the number.
Open the Python workspaceCheck your understanding
A scatter plot of two variables forms a clean symmetric U shape, and Pearson r comes out as 0.00. What does that tell you?
- The two variables are independent of each other
- There is no straight-line trend, but the variables can still be strongly related
- The correlation could not be computed and fell back to zero
- One of the two variables has no variation in it
Show answer
r near zero only rules out a straight-line trend: in a symmetric U the negative deviation products on one arm cancel the positive ones on the other, so r is 0 even when y is fully determined by x. Concluding independence is the tempting error, and it is wrong because the plot shows a clear rule linking the two. A genuine failure or a zero-variance variable would make the denominator zero and raise an error rather than return 0.00.