In 1994, the Taiwan government launched a program to promote community empowerment in the countryside, providing resources and knowledge to help rural communities become more sustainable and livable. However, despite the efforts of the government and community workers, many rural places continue to decline and lose their unique features. To address causes of rural communities' decline, it is important to understand how to effectively promote social innovation. This paper is based on fieldwork and participation in community empowerment projects in low elevation mountain communities in Dongshan District in Tainan City (Qiandapu) to investigate why this region continues to decline despite community empowerment projects. Because of geographical conditions, ethnic identities and a history of immigration, local society created the tradition of publicity and locality that, influences how community members participate in public affairs and subjectivity. At the same time, broad changes in Taiwanese society and the influx of external categories have weakened the community's ability to solve problems, leading to a loss of energy in local society and community decline. Although community empowerment and social innovation have created new possibilities in Qiandapu in recent years, the situation of local society still cannot be transformed because of both subjective and objective factors. Taking tradition of locality and publicity seriously can help social innovation projects rethink their approaches in order to more effectively promote the transformation of local societies.