They had been hanging in there when the right-back went up for a corner.
His vital touch at the far post came as defenders were out-jumped in the crowded goal area by Marvin Elliott.
West Brom had looked set for their first away win after Robert Koren had rounded off a fine attacking game with a 74th minute strike.
In the celebrations which followed Orr's lifeline, he was booked for dissent but hardly minded.
On Saturday these teams hit seven goals between them without reply but now took time to find their shooting range.
Just two shots were on target in the first half hour and each brought a fine save and a corner.
Michael McIndoe - soon to limp away with an ankle injury - got in the first for City with a long drive and Richard Chaplow put in a similar one for West Brom.
The visitors made three midfield changes after their 4-0 win over Ipswich and it was a pretty seamless reshuffle.
Paul Robinson was always there pushing up from left-back to support Chaplow and the Hungarian Zoltan Gera.
City soon missed the penetrating runs of McIndoe - who went off just short of the half-hour mark - as their £1million striker Lee Trundle often found himself shackled.
After pushing back enthusiastic home attacks, West Brom, who were beaten in their previous two away games, gradually imposed themselves.
It was Chaplow's fierce low drive which drew the first of numerous saves from Adriano Basso, who quickly became the busiest man on the field.
From a corner Robinson rammed a header against the bar before the flying Basso kept out a drive from Gera and then dropped rapidly onto a long effort from Carl Hoefkens.
City knew they had to stop the game slipping away from them in the second half and, aided by a couple of sloppy midfield balls, both Trundle and Ivan Sproule got in decent efforts.
But they were undone when Lee Johnson lost the ball on the edge of the area in West Brom's half.
A burst by substitute Ishmael Miller down the right and a cross knocked in at the far post by Koren looked like landing all three points for the Midlanders until Orr's late intervention.