Arshavin had a shot deflected onto the crossbar before substitute midfielder Aaron Ramsey sealed victory with 11 minutes left, cutting into the box and sliding a shot past goalkeeper Thomas Sorensen.
But Jelavic was not to be denied with his next attempt, the striker brushing off the challenge of Mulgrew to latch on to Weiss's quickly taken free-kick and sliding a shot past Forster, the ball rebounding off the right-hand post but spinning back over the line.
However, Blues keeper Ben Foster smartly got a hand to the ball to crucially slow down the Nigerian's left-foot shot before a sliding Ridgewell cleared off the goal-line.