Both Joleon Lescott and Shaun Wright-Phillips had been guilty of wasting final passes and Jo of failing to outmuscle his marker while through on goal before Italian Balotelli opened the scoring.
Bellamy's persistence earned a corner on the left and Collins got across his marker to turn home Ramsey's near-post cross, the Aston Villa defender scoring his second international goal.