The child and his mum were allegedly approached while visiting a shrine to the Prophet Muhammad in Medina and asked if they were Shia Muslims.
The mum reportedly said yes and several minutes later a car pulled up beside them and the child was snatched away from her.
The boy was repeatedly stabbed in the neck with a piece of broken glass until he was beheaded, it is claimed.
His helpless mum could only watch and scream as the horror unfolded.
Saudi Arabia's predominant branch of Islam is Sunni, accounting for around three quarters of the population.
Human rights groups say practitioners of other forms of the faith - including Shia - are persecuted.
Shia Rights Watch, a Washington DC-based campaign group, said: "There was no intervention from anyone and no response from authority thus far.
"The Saudi Shia community came together in mourning and to show solidarity with the parents.