All types of phenomena, whether its home runs, traffic accidents, PhDs granted, cancer, and so on vary in number from year to year (and place to place). That is, the number of home runs hit by the Yankees, the rate of traffic accidents in California, the number of PhD awarded by Stanford's Sociology Department, and the incidents of kidney cancer is not constant from year to year.Moreover, the level of variance is greater with small numbers of events than with large numbers. To illustrate why this is true imagine a large urn that contains 1,000 marbles, 500 that are red and 500 that are white. If you repeatedly take 5 marbles out of the urn until it is empty, most of the time you will get a mix of red and white marbles, but every so often you'll remove 5 marbles all of the same color. However, if you repeatedly take 50 marbles out of the urn each time, you'll seldom, if ever pull out 50 marbles of the same color. This may all seem obvious, but it illustrates why with small numbers of events (in this case, the number of marbles pulled) you are more likely to experience an extreme event (in this case, drawing marbles of the same color) than you are with large numbers of events. That is why surveys that interview 1,200 people are usually more accurate than those that interview 600.
Which brings us back to the drop in San Jose's murder rate. The average number of murders in San Jose is relatively small (33), which means that extreme outcomes (i.e., extreme spikes and drops in the murder rate) will be the norm, not the exception. Moreover, after a wide swing in one direction, it is quite common for rates to move back toward the average, a phenomenon I discussed in my previous post ("Regression to the Mean and the 49ers' Next Coach"). This means that the spike in the murder rate in 2012 was likely to be followed by a drop in the rate close to the average. I expected the drop to occur in 2013, but it didn't happen until 2014 when it fell to 32 (see the chart below), just below its historical average. And as I argued back then, this drop probably has more to do with the natural variation of rare events than it does with the efforts of the San Jose Police Department.