With the development of big data and artificial intelligence, distributed optimization has emerged as an indispensable tool for solving large-scale problems. In particular, the multi-agent system based on distributed information processing can be elaborately designed for distributed optimization, in which the agents collaboratively minimize a global objective function made up of a sum of local objective cost functions subject to some local and/or global constraints. Inspired by the applications involving resource allocation, machine learning, power systems, sensor networks and cloud computing, a variety of distributed optimization models and algorithms have been investigated and developed. The optimization models include unconstrained and constrained problems in continuous and discontinuous systems with undirected and directed communication topology graphs. The constraints include bounded constraint, separable and inseparable equality and inequality constraints. Meanwhile, in distributed algorithms, every agent executes its local computation and updating on basis of its own data information and that exchanging with its neighboring agents by means of the underlying communication networks, in order to deal with the optimization problems in a distributed way. This paper is designed to provide a comprehensive overview of extant distributed models and algorithms for distributed optimization.