Albion were trailing 3-2 heading into the final minute of the game when substitute James Morrison fired them back on to level terms with a 25-yard drive.
Bednar then completed the dramatic comeback when he headed home a Chris Brunt cross in the third minute of stoppage time to earn Albion only their third win in nine games.
It was rough on Colchester who had powered their way into a two-goal lead after just 16 minutes thanks to goals from Chris Coyne and a first-ever strike from teenager Medy Elito.
Albion were back on level terms before half-time thanks to Kevin Phillips' 22nd goal of the season and a classy finish from Brunt.
At that stage Albion looked set to roar to a comeback win as they enjoyed a purple patch at the start of the second-half.
Failing to take their chances looked set to be costly when Kevin Lisbie regained the lead for Colchester in the 75th minute.
But all that did was set up the frantic finale that saw Albion take a massive step closer to automatic promotion.
Colchester took full advantage of hesitancy by goalkeeper Dean Kiely to surge into a two-goal lead. Kiely was unable to deal with a left-wing free-kick from Dean Hammond and Coyne took full advantage to score his first goal of the season with an overhead kick from six yards.
Kiely's day got even worse two minutes later. He punched out a cross from Phil Ifil and the ball dropped to Elito, whose rising left-foot drive flew into the top of the net.
But holding on to the lead proved to be harder than taking it and Colchester were pegged back with Albion's own quickfire double.
Phillips, who had earlier hit the post, reduced the arrears in the 36th minute when he stroked home a centre from right-back Carl Hoefkens.
Brunt then fired Albion level with a stunning solo goal. The Northern Ireland international winger cut in from the right flank and drilled a low curling 20-yard drive past Dean Gerken.
Albion should then really have taken complete control in the opening ten minutes of the second half.
But Zoltan Gera hit the post, Paul Robinson headed wide and Roman Bednar blazed over the bar as well as being denied by a smart save from Gerken.
Those proved to be costly misses when, not for the first time, Albion were caught out by a set-piece.
Hammond lifted over a corner that Johnnie Jackson headed towards goal only to be denied by a smart save by Kiely.
The rebound flew straight to Lisbie, who had the simple task of stabbing home his shot from six yards.
It looked enough for Colchester to pull off a double against Albion before Morrison struck and then Bednar set the seal on a dramatic end to the game.