The 21-year-old Baggies midfielder had been on a season long loan deal at Kidderminster Harriers but has ended his stay at the Aggborough to sign a permanent deal with the Stags.
Albion will receive an initial £50,000 fee for Brown with up to another £50,000 based on appearances.