Issues relating to the elegant gathering attended by Su Shi (1037-1101) in a place called Xiyuan (Western Garden), the painting of such a gathering supposedly produced by Li Gonglin (ca. 1041-1106) and a record of the painting said to have been written by Mi Fu (1052-1107), have been debated by art historians like Ellen Johnston Laing and Xu Jianrong based on different textual interpretations and methodologies. This paper attempts to resolve the argument by tracing the origin of the name Xiyuan to Cao Pi (187-226) and Cao Zhi (192-232) of the Wei dynasty. Gatherings held in this garden by the Cao Brothers and their circle of eminent scholars were hallowed by time so that Xiyuan became synonymous with literary gatherings in later centuries. Accordingly the Xiyuan in the famous ci poem by Su Shi, 'To the Tune of Shui Long Yin--To the Rhyme of Yanghua ci by Zhang Zhifu," is generally annotated as an allusion to the Western Garden of the Cao Brothers, while the ci poem, written by Su Shi under the guise of Zhang Zhifu's wife or son, is often interpreted as an expression of longings for Zhang by his family and sadness in the passing of spring, especially in the line "Lamenting the fallen flowers of Western Garden forever gone." Yet this paper provides evidence that Su Shi had named his own gardens in Fengxiang and Mizhou Western Garden. Moreover, there was a Western Garden in Kaifeng where Su Shi, Wang Shen (ca. 1046-after 1100) and other scholars and artists could have gathered. Similarly, Xuzhou also had a garden bearing this name, which could possibly be the venue for literary gatherings of Su Shi and other scholars, particularly his good friend Wang Gong (?-ca. 1073) and younger brother Su Che (1039-1112). In other words, when Su Shi mentioned Xiyuan in the ci poem he was possibly referring to his friends and their gatherings who had all scattered. Understood in this light, the ci poem, composed during the poet's banishment in Huangzhou, has a deeper meaning as a self-statement of the poet, and his reason for instructing Zhang Zhifu not to show this ci poem to anyone else becomes clear.