To solve the problem that the overbooking model is rough and the oversold quantity is too large, overbooking of airline tickets is studied by the computer simulation method of Monte Carlo simulation. Firstly, the mathematical model of newsvendor problem in operational research is improved, and the cost in the objective function is refined. More specifically, the cost of overbooking is determined as 2 parts: overestimate cost and underestimate cost. In terms of constraints, because the oversold quantity given by the previous classical methods is too large, it is transformed into the oversold quantity constraint. Thus, a mathematical model is established to determine the oversold quantity of air tickets. Secondly, the stochastic process of booking tickets and overbooking tickets is established, and 1000 simulations are carried out respectively from three aspects: the revenue of airlines under different overbooking seats, the optimal overbooking strategy under different compensation limits and the impact of passenger return probability on overbooking of airlines. Finally, according to the results of Monte Carlo simulation, the paper gives some suggestions on the strategy of computer oversold in different situations.