Stochastic gradient descent (SGD) methods have gained widespread popularity for solving large-scale optimization problems. However, the inherent variance in SGD often leads to slow convergence rates. We introduce a family of unbiased stochastic gradient estimators that encompasses existing estimators from the literature and identify a gradient estimator that not only maintains unbiasedness but also achieves minimal variance. Compared with the existing estimator used in SGD algorithms, the proposed estimator demonstrates a significant reduction in variance. By utilizing this stochastic gradient estimator to approximate the full gradient, we propose two mini-batch stochastic conjugate gradient algorithms with minimal variance. Under the assumptions of strong convexity and smoothness on the objective function, we prove that the two algorithms achieve linear convergence rates. Numerical experiments validate the effectiveness of the proposed gradient estimator in reducing variance and demonstrate that the two stochastic conjugate gradient algorithms exhibit accelerated convergence rates and enhanced stability.

This paper presents various acceleration techniques tailored for the traditional 3D topology optimization problem. Firstly, the adoption of the finite difference method leads to a sparser stiffness matrix, resulting in more efficient matrix-vector multiplication. Additionally, a fully matrix-free technique is proposed, which only assembles stiffness matrices at the coarsest grid level and does not require complex node numbering. Moreover, an innovative N-cycle multigrid (MG) algorithm is proposed to act as a preconditioner within conjugate gradient (CG) iterations. Finally, to further enhance the optimization process on high-resolution grids, a progressive strategy is implemented. The numerical results confirm that these acceleration techniques are not only efficient, but also capable of achieving lower compliance and reducing memory consumption. MATLAB codes complementing the article can be downloaded from Github.

In this paper, we present quantum algorithms for a class of highly-oscillatory transport equations, which arise in semi-classical computation of surface hopping problems and other related non-adiabatic quantum dynamics, based on the Born-Oppenheimer approximation. Our method relies on the classical nonlinear geometric optics method, and the recently developed Schrödingerisation approach for quantum simulation of partial differential equations. The Schrödingerisation technique can transform any linear ordinary and partial differential equations into Hamiltonian systems evolving under unitary dynamics, via a warped phase transformation that maps these equations to one higher dimension. We study possible paths for better recoveries of the solution to the original problem by shifting the bad eigenvalues in the Schrödingerized system. Our method ensures the uniform error estimates independent of the wave length, thus allowing numerical accuracy, in maximum norm, even without numerically resolving the physical oscillations. Various numerical experiments are performed to demonstrate the validity of this approach.

The recently developed DOC kernels technique has been successful in the stability and convergence analysis for variable time-step BDF2 schemes. However, it may not be readily applicable to problems exhibiting an initial singularity. In the numerical simulations of solutions with initial singularity, variable time-step schemes like the graded mesh are always adopted to achieve the optimal convergence, whose first adjacent time-step ratio may become pretty large so that the acquired restriction is not satisfied. In this paper, we revisit the variable time-step implicit-explicit two-step backward differentiation formula (IMEX BDF2) scheme to solve the parabolic integro-differential equations (PIDEs) with initial singularity. We obtain the sharp error estimate under a mild restriction condition of adjacent time-step ratios $r_k := τ_k/τ_{k−1}<r_{\rm{max}}= 4.8645$ $(k ≥ 3)$ and a much mild requirement on the first ratio, i.e. $r_2> 0.$ This leads to the validation of our analysis of the variable time-step IMEX BDF2 scheme when the initial singularity is dealt by a simple strategy, i.e. the graded mesh $t_k= T (k/N)^γ.$ In this situation, the convergence order of $\mathcal{O}(N^{− min\{2,γα\}})$ is achieved, where $N$ denotes the total number of mesh points and $α$ indicates the regularity of the exact solution. This is, the optimal convergence will be achieved by taking $γ_{{\rm opt}} = 2/α.$ Numerical examples are provided to demonstrate our theoretical analysis.

In this paper, we present a novel Douglas-Rachford-splitting-based path following (DRS-PF) method that rapidly obtains the solution of linear programming (LP) with high accuracy. It originates from the fixed-point mapping associated with DRS on the log-barrier penalized LP. A path-following scheme is then proposed to simultaneously update the iterates and the penalty parameter for accelerating the overall procedure. Its global convergence towards an optimal solution to the original problem is established under mild assumptions. Numerical experiments show that DRS-PF outperforms the simplex and interior point methods implemented in the academic software (CLP, HiGHS, etc.) in terms of the geometric mean of the running time on a few typical benchmark data sets. In some cases, it is even reasonably competitive to the interior point method implemented in Gurobi, one of the most powerful software for LP.

In this paper, we propose an accelerated stochastic variance reduction gradient method with a trust-region-like framework, referred as the NMSVRG-TR method. Based on NMSVRG, we incorporate a Katyusha-like acceleration step into the stochastic trust region scheme, which improves the convergence rate of the SVRG methods. Under appropriate assumptions, the linear convergence of the algorithm is provided for strongly convex objective functions. Numerical experiment results show that our algorithm is generally superior to some existing stochastic gradient methods.

An explicit numerical method is developed for a class of non-autonomous time-changed stochastic differential equations, whose coefficients obey Hölder’s continuity in terms of the time variables and are allowed to grow super-linearly in terms of the state variables. The strong convergence of the method in the finite time interval is proved and the convergence rate is obtained. Numerical simulations are provided.

An analysis of logistic stochastic differential equations (SDEs) with general power-law and driven by a Wiener process is conducted. We prove existence of unique, strong Markovian, continuous solutions. The solutions live (a.s.) on bounded domains $D = [0, K]$ required by applications to biology, ecology and physics with nonrandom threshold parameter $K > 0$ (i.e. the maximum carrying constant). Moreover, we present and justify nonstandard numerical methods constructed by specified balanced implicit methods (BIMs). Their weak and $L^ p$-convergence follows from the fact that these methods with local Lipschitz-continuous coefficients of logistic SDEs “produce” positive numerical approximations on bounded domain $[0, K]$ (a.s.). As commonly known, standard numerical methods such as Taylor-type ones for SDEs fail to do that. Finally, asymptotic stability of nontrivial equilibria $x_∗ = K$ is proven for both continuous time logistic SDEs and discrete time approximations by BIMs. We exploit the technique of positive, sufficiently smooth and Lyapunov functionals governed by well-known Dynkin’s formula for SDEs.

Anderson acceleration is a kind of effective method for improving the convergence of the general fixed point iteration. In the linear case, Anderson acceleration can be used to improve the convergence rate of matrix splitting based iterative methods. In this paper, by using Anderson acceleration on general splitting iterative methods for linear systems, three classes of methods are given. The first one is obtained by directly applying Anderson acceleration on splitting iterative methods. For the second class of methods, Anderson acceleration is used periodically in the splitting iteration process. The third one is constructed by combining the Anderson acceleration and split iteration method in each iteration process. The key of this class of method is to determine a combination coefficient for Anderson acceleration and split iteration method. One optimal combination coefficient is given. Some theoretical results about the convergence of the considered three methods are established. Numerical experiments show that the proposed methods are effective.

In this paper, an effective oscillation-free discontinuous Galerkin (DG) scheme for a nonlinear stochastic convection-dominated problem is formulated and analyzed. The proposed oscillation-free scheme is capable to capture the steep fronts of solution automatically and distinguish the influence of the convection domination and noise perturbation. Under proper regularity assumptions, the optimal convergence rates in space and time are rigorously proved with the techniques of variational solution and conditional expectation. In the numerical simulation, the classical SIPG scheme and the proposed oscillation-free DG scheme are both performed and compared. The numerical convergence rates tests are first carried out to verify the theoretical results. The benchmark tests having the steep behaviors are further provided to illustrate the effectiveness and robustness of our proposed oscillation-free DG scheme.

In this paper, we consider the numerical solution of decoupled mean-field forward backward stochastic differential equations with jumps (MFBSDEJs). By using finite difference approximations and the Gaussian quadrature rule, and the weak order 2.0 Itô-Taylor scheme to solve the forward mean-field SDEs with jumps, we propose a new second order scheme for MFBSDEJs. The proposed scheme allows an easy implementation. Some numerical experiments are carried out to demonstrate the stability, the effectiveness and the second order accuracy of the scheme.

In this study, we explore two distinct rational approximations to the radiation condition for effectively solving time-harmonic wave propagation problems governed by the Helmholtz equation in $\mathbb{R}^d,$ $d = 2$ or $3.$ First, we focus on the well-known complete radiation boundary condition (CRBC), which was developed for a transparent boundary condition for two-dimensional problems. The extension of CRBC to three-dimensional problems is a primary concern. Applications of CRBC require removing a near-cutoff region for a frequency range of a process to minimize reflection errors. To address the limitation faced by the CRBC application we introduce another absorbing boundary condition that avoids this demanding truncation. It is a new rational approximation to the radiation condition, which we call a rational absorbing boundary condition, that is capable of accommodating all types of propagating wave modes, including the grazing modes. This paper presents a comparative performance assessment of two approaches in two and three-dimensional spaces, providing insights into their effectiveness for practical application in wave propagation problems.