Prayer is one of the pillars of Islam. According to all the Muslims scholars, whoever gives it up while denying it is a Kafir (disbeliever). Whoever gives it up and neglects it or does not pray because of laziness; never praying it, is also, according to the chosen correct opinion of scholars, considered as a disbeliever.
The Prophet (SAW) said: “The difference between us and them is the prayer. Whoever gives it up becomes a disbeliever.” [Ahmad and Abu Daawood]