Whether the idea of Corporate Social Responsibility (CSR) developed byWestern brand name companies can improve work conditions or not in the third world factory run by their suppliers is still under debate. This paper tries to explore the conflictual interests of different stakeholders arisen from the implementation of CSR imposed by the brand name companies on suppliers by analyzing the cases of Taiwanese shoes manufacturing companies in Vietnam. It is found that effective CSR implementation is not able to reach unless there is an independent worker’s representative organization, ie., the free factory trade union, to mediate different conflicts.We contend that without democratic participation of local workers to represent their interests, current top-down CSR implementation from Western brand name companies and Asian suppliers has its limitations, and will not be able to change the current unequal power relations between management and workers, which can’t channel workers’ grievances.