This study is to examine whether social exclusion discourses are sufficient for the analysis of Taiwanese indigenous peoples’ disadvantages. Social exclusion often refers to the process of an individual being economically and non-economically deprived, emphasizing its multiple and dynamic nature, and often being used to measure the gaps of people’s living standards. The concept of social exclusion could inform social policies in relation to indigenous peoples, as it could be used to measure the welfare gaps between the mainstream and the indigenous society. However, this study finds that the framework of social exclusion analysis assumes homogenous citizenship, applying the same criteria of social exclusion to each individual, disregarding the impacts of collective experience on people’s living standards. The focus group data of this study shows that the extent of indigenous peoples’ languages, cultures and ways of life being recognized by the mainstream society has a profound impact on indigenous people’s quality of life. With regard to specific aspects of quality of life, indigenous people do not prioritize individual and household-level well-being, such as income, employment, social network, social participation, education, and health, etc. Instead, they pay more attention to collective issues of natural disasters and transportation infrastructure. This study suggests that in analyzing indigenous people’s extent of social exclusion and quality of life, one must take their collective experience and special needs into account.