In this paper, we propose a high order semi-implicit well-balanced finite difference scheme for all-Mach Euler equations with a gravitational source. We start with the conservative form of full compressible Euler equations including conservation of total energy, for shock capturing in the high Mach regime. For asymptotic preserving in the low Mach regime and to address the difficulty of strong coupling between the stiff gravitational source and conservative variables, we add the evolution equation of the perturbation of potential temperature, which corresponds to weak potential temperature stratification under a hydrostatic background potential temperature. The resulting system is then split into a (non-stiff) nonlinear low dynamic material wave to be treated explicitly, and (stiff) fast acoustic and gravity waves to be treated implicitly. With the aid of explicit time evolution for the perturbation of potential temperature, we design a novel well-balanced finite difference weighted essentially non-oscillatory (WENO) scheme, which can be proven to be both asymptotic preserving and asymptotically accurate in the incompressible limit. Numerical experiments are provided to validate these properties.