The study is aimed to study the cultural difference of physical education between that of Chinese and the West. Issues including the differences in the culture, the thought, and the organization of the physical education were discussed based on the contrastive analysis method. The steps of the study were consisted of reviewing, analyzing, and inducting the materials in the collected papers. Form the study, the Chinese-West cultural differences in physical education are found as follows. The first difference comes from the Chinese-West cultural difference. The Chinese culture is categorized as the continental culture that emphasizes on the integration between nature and human beings. Furthermore, everything is family oriented. In the contrast, the West is categorized as the oceanic culture. It emphasizes on the personal and believes that everything even the nature can be conquered. The second difference is the physical education thought. People in Chinese emphasize on core spirit of their race and highlight politeness during sport competition, while, in the West, people emphasize on competition fairness and devote all the life to triumph over their own limits. Thirdly, the difference is the organization of the physical education. Many private organizations having their own rules may exist simultaneously in Chinese. However, only a common organization with common rule can be established in the West for a specific item of sports. Finally, the physical education facilities used in Chinese are quite flexible and the athletes pay more attention to the artistry of their own action in sport. On the other hand, most of the apparatuses and action in the West have been standardized.