In this work residual error estimates are constructed using Neural Networks for Finite Element Method. These can be used to do adaptive mesh refinement. Two neural networks are developed the Multilayer Perceptron and the Transformer model. The error estimates are made for 1d poisson equations but the idea will generalise to higher dimensions as well.