Leicester, and in particular their right-back Darren Kenton, will be kicking themselves for failing to take a point back to the East Midlands.
Thanks to goalkeeper Paul Henderson and the woodwork, Leicester had withstood everything that Albion had thrown at them before the unlucky Kenton headed into his own net in the 82nd minute.
Phillips then pounced from the spot 60 seconds later as Leicester's four match unbeaten run came to a shuddering halt.
It was a nightmare ending to the game for Leicester who had looked set to ride their luck and escape with a share of the spoils after the Albion midfielder Darren Carter had twice hit the woodwork.
With Henderson impressing after pulling off saves from Phillips and John Hartson, Leicester looked destined to secure their fourth successive clean sheet.
But their world caved in with Kenton's amazing error eight minutes from time.
He tried to head a cross from Nigel Quashie behind for a corner but only succeeded into steering the ball past the bemused Henderson into the back of the net.
It was a crushing blow for Leicester and the resilience they had shown all match deserted them.
Just 60 seconds later Phillips, who was making his home debut following his recent arrival from Aston Villa, surged into the area and was hauled down by Leicester skipper Patrick McCarthy.
The former England marksman picked himself up and calmly slotted the ball beyond Henderson to leave Albion relieved and Leicester stunned.
Albion had always looked the side most likely to win as they dominated the match for long spells with Leicester struggling to carry a threat in attack despite the best efforts of Matty Fryatt and Chris O'Grady.
Carter was frustrated for the first time in the 10th minute when his 25 yard free-kick rattled against the post.
Henderson then frustrated Phillips with a save low down to his right before pulling off a stunning stop to beat away a goalbound close range header from Hartson.
Albion continued to press after the break with Phillips being denied by a last ditch challenge from Kenton and another smart save from Henderson.
But it was the impressive Carter who carried Albion's greatest threat and he found it hard to believe when his dipping 20 yard volley in the 65th minute hit the bar.