Albion slumped to their fifth straight defeat following an excellent second-half comeback by Montrose.
The visitors had the best of the early opportunities with Kevin Webster seeing a shot deflect just over after six minutes.
Webster then turned provider 10 minutes later when he teed up Steven Kerrigan for a shot that brought a fine save from Chris Fahey in the Albion goal.
Montrose were well on top at this point and creating some excellent opportunities.
However, it was Albion who took the lead on 20 minutes when Jered Stirling curled an excellent 25-yard free-kick into the net.
Only seconds earlier Mark Yardley saw a header rebound off the post as Albion tried to get back into the match.
Rovers were hanging on for the rest of the half as Montrose looked for the equaliser.
On 25 minutes Fahey produced a superb double save to first deny Craig Smart and then Martin Wood with the follow up.
Wood was then denied by the woodwork and his neat turning shot rebounded back off post.
Rovers had changes to go further ahead but striker Paul McManus squandered some good opportunities as he looked unusually out of sorts in front of goal.
Montrose dominated much of the second half and deservedly got an equaliser on 62 minutes.
Paul Doyle was adjudged to have been bundled over inside the box and Smart hammered the ball home from the resulting spot kick.
Then the visitors claimed all three points on 71 minutes when Woods drove towards goal and fired an unstoppable shot into the top corner from 25 yards.