Image smoothing techniques are widely used in computer vision and graphics applications. In this paper, we present an $\ell_p$-nonconvex regularization method for image smoothing. To induce sparsity prior of the smoothed images more strongly than the $\ell_1$ norm regularization, we take the nonconvex arctangent penalty function of the image gradient as the regularization term. To make the proposed model more flexible and effective for different image smoothing applications, we use the $\ell_p$ norm function as the fidelity term, instead of the $\ell_2$ norm function. The powerful majorization-minimization (MM) algorithm is employed for the proposed nonconvex optimization model. The convergence of the resulting MM algorithm is discussed. Comprehensive experiments and comparisons show that the proposed method is effective in various image processing tasks such as texture smoothing, detail enhancement, artifact removal, image denoising, high dynamic range (HDR) tone mapping, edge detection, and image composition.