In this paper, a pointwise goal-oriented residual-based a posteriori error estimator is proposed for linear elliptic equations with restricted source terms. The pointwise error is directly estimated by introducing the dual problem with a Dirac delta source term instead of using classical mollification technique. The goal-oriented error estimator is proved to be the upper bound of the pointwise error. Numerical experiments show the advantage of the adaptive finite element method (AFEM) based on this error estimator, which can preserve the monotonicity of the pointwise error, compared with the goal-oriented AFEM using the mollification technique.