Hulse converted his side's only real chance of the game on 76 minutes to leave the Rams still looking for their first win.
It was Albion's fourth victory on the trot since an opening day hammering at Walsall and boss Gary Megson said the way his team had responded to that defeat was fantastic.
He said: "I still don't think we are playing particularly fluent football at the moment. After Walsall I put over a few things to the players, perhaps not in the most eloquent way, but the way they have come back to that day delights me.
"We never really got out of first gear, but football is about winning matches and that's exactly what we did."
Megson's men dictated the early play, but were lucky not the fall a goal and a man down on 11 minutes when Larus Sigudsson hacked down Lee Morris on the edge of the area after he was sent clear by Mathias Svensson.
However, although it appeared the Icelandic defender was the last man, Ref Mike Ryan caused uproar when he produced just the yellow card.
Seven minutes later Svensson was involved again when he spurned a glorious chance to put his side ahead.
The on-loan frontman volleyed wildly over from ten yards after Joost Volmer completely missed Simo Valakari's hopeful punt into the box.
Albion striker Scott Dobie then had a goal disallowed for offside before Svensson and Ian Taylor came close as Derby ended the first half well on top.
The Rams continued to press after the break and almost took the lead on the hour when Morris was denied by a smart save ex-Derby man Russell Hoult.
But just when it looked like the Baggies would settle for a point, Hulse popped up from six yards to acrobatically hook the ball past Andy Oakes after the Derby defence failed to clear Sean Gregan's flighted free-kick.
The home side pressed hard after Hulse's strike, but just couldn't find their way find a resolute West Brom defence and now find themselves rooted to the foot of the first division.
Afterwards Manager George Burley said: "We deserved at least a point. We showed we can compete with one of the top teams and a lot of positives came out of the match."