Rangers dominated affairs and an Alan Brazil header came back off the bar and Ian Little also went close.
Ten minutes from time, Damon Gray was tripped in the box and the substitute scored from the spot-kick.
Berwick: Peat, Notman, Smith, McLean, Ewart, Callaghan, Little (Gray 65), Currie, Brazil, McLaren, David Greenhill. Subs Not Used: Ellison, Russell, McMullan, Radzynski.
Booked: Callaghan, McLean, Gray.
Goals: Currie 70, Gray 80 pen.
Albion: Gaston, Reid (Stewart 46), McGowan, McKeown, Benton, O'Byrne, Boyle (McCusker 56), Tyrrell, Pollock, Walker, Barr (Donnelly 46). Subs Not Used: Ewings, O'Boyle.
Booked: McGowan, Benton, O'Byrne, McCusker.
Att: 471
Ref: C Charleston
Source: BBC Sport
Source: BBC Sport