This paper presents ALM-PINN, an adaptive physical informed neural network algorithm optimized by Levenberg-Marquardt. ALM-PINN is tailored to overcome challenges for solving singular perturbation problems (SPP). Traditional neural
network-based solvers reframe solving differential equations task as a multi-objective
optimization problem involving residual or Ritz error. However, significant disparities
in the magnitudes of loss functions and their gradients frequency result in suboptimal training and convergence challenges. Addressing these issues, ALM-PINN introduces a learnable parameter for the perturbation parameter and constructs a two-terms
loss function. The first loss term emphasizes approximating the governing equation,
while the second term minimizes the difference between perturbation and learnable
parameters. This adaptive learning strategy not only mitigates convergence issues in
directly solving SPP but also alleviates the computational burden with asymptotic iteration from a large initial value. For one-dimensional tasks, ALM-PINN enhances
training efficiency and reduces complexity by enforcing hard constraints on boundary
conditions, streamlining the loss function sub-terms. The efficacy of ALM-PINN is
evaluated on five SPPs, demonstrating its ability to capture sharp changes in physical
quantities within the boundary layer, even with small perturbation coefficients. Furthermore, ALM-PINN exhibits reduced errors in both $L_ {\infty}$ and $L_2$ norms, coupled with
improved convergence speed and stability.