In past studies, there are two theories explaining the process of decline in fertility. The first is the socioeconomic adjustment theory, which indicates that change in fertility is related to motivational forces stemming from changes in socioeconomic conditions; the second is the innovation diffusion theory, which admits the importance of motivational forces as well, but argues that social networks and interpersonal connections, rather than socioeconomic factors, are the reasons for making people accept contraception. In this paper, by using the methods of spatial econometrics, we consider both the factors of socioeconomic structure and spatial diffusion to explain the decline in fertility rates in Taiwan across 358 townships and within three years, 1980, 1990 and 2001. The spatial lag dependent variable, representing neighborhood effects, is used to verify the application of the innovation diffusion theory. The findings suggest that socioeconomic structure factors could explain about half of the variance in fertility rates; in addition, ethnic factors are significant, showing that some cultural reasons played an important role in the process leading to a low fertility rate. Finally, the existence of neighborhood effects in our regression models implies that the innovation diffusion theory has been verified, and its influence has been increasing with time.