In this paper, we propose a method that can search moving objects, track and segment these objects, and record trajectories of them, form sports video automatically. In order to search the target objects in the badminton games effectively, we first process the first frame of input video with Canny edge detector and Hough transformation to find the position of Region of Interest (ROI) area. We use the ROI area to find out the dominant color. Then target objects are segmented with the dominant color and object motion information. The eight parameters of perspective transformation are also calculated by the border of ROI area. The object tracking task is done by the Mean Shift - Particle Filter approach. The result object trajectory is converted from camera coordinate to world coordinate by perspective transformation, and then store in database. The system also provides interface to user to query the trajectories constructed. The object trajectories can be applied to several applications in different area, such as Video Database, Surveillance System, Military System. In the sport science area, players trajectories can be used for tactics analysis or players training.