Along with the era of big data oncoming, the public sector cannot out of the trend, the public sector has many priceless big data, the government can make good use of benefits arising from the big data analysis, and the proper response to the challenges of it, there must be help to improve the efficiency of government operations. So far, all the major countries has already announced its policy of big data, this paper describes the development of big data and related policies of the US, UK and Australia, and then describes how to make the public sector cost down and to improve decision-making quality, the establishment of transparent government and the development of new products, the creation of economic value and other benefits. Although big data for the public sector can bring many benefits, but it also brings challenges, such as: data collection difficulties, the data attribute difference between public and private sector, legitimacy, security and compliance obstacles and the willing to promote is lower. In response to these challenges, this paper will present findings and provides the policy maker reference and recommended that the government should give priority to promoting the big data plan, set up a dedicated analyst, instant analysis and cross-border collaboration with public and private sector.