But Jelavic fired in off the post in the 98th minute to win it for Rangers.
Nikica Jelavic headed Rangers ahead but David Robertson levelled for United on the stroke of half-time.
Jelavic headed over late on as Rangers saw out their fifth straight league win of the campaign.
Pernis parried away a drive by McCulloch and then intervened to prevent a Jelavic cross reaching Miller.
Another Guzan parry fell invitingly and this time Jelavic slotted home into the bottom corner from 10 yards.
The game was beginning to open up and Steven Whittaker and Jelavic lifted shots over the visitors' crossbar.
But minutes after Nikica Jelavic had missed from six yards, the Croatian scored the decisive goal from 10 yards.
Jelavic got the ball stuck under his foot yards from goal and saw his two efforts blocked by Guzan.
It was Rangers who took the lead on 19 minutes when Jelavic bulleted home a header from El-Hadji Diouf's free-kick.
Nikica Jelavic then ran on to a woeful back-pass from Richie Towell to round Graeme Smith and roll the ball home.
The home side fell behind to Nikica Jelavic's third minute goal but recovered to level in the 20th minute through Scott McDonald.
"Contrary to all expectations, Nikica Jelavic said that he did not want to play for SK Rapid, " he told the club's website.
El-Hadji Diouf replaced Jelavic for the closing stages and set up Weiss for a shot that required a last-ditch save by Forster.
United continued to be opened up at the back and Jelavic laid on another simple finish for Miller eight minutes from the end.
Vienna continued to press and should have equalised through Jelavic after Guzan parried a spectacular overhead kick from Maierhofer into the Croatian's path.
The 20-year-old winger is expected to be joined at Ibrox by Rapid Vienna's Nikica Jelavic, who will undergo a medical with Rangers on Friday.
Rangers are without defenders Madjid Bougherra and Andy Webster, while midfielder Maurice Edu and strikers James Beattie and Nikica Jelavic are all still missing.
Jelavic was unmarked six yards from goal but missed the target.
Rapid president Rudolf Edlinger indicated that Jelavic, who joined Rapid from Zulte Waregem in 2008, had been suspended by the club for his refusal to train.
Rangers thought they had been awarded a penalty when Jelavic went to ground following Rogne's tackle and referee Thomson initially appeared to point to the spot.
Jelavic robbed Scott Severin in the opening seconds of the second half and blasted a shot towards goal that forced goalkeeper Pernis to beat the ball away.
Indeed, Jelavic's move was only re-ignited after the striker was left out of Rapid's squad for the 1-1 draw after refusing to play in Thursday's first leg in Vienna.
But Jelavic, who had scored Rapid's goal in the first leg, scored to put Rapid through on away goals, after keeper Brad Guzan had parried a Stefan Hoffmann shot.
The Slovakian was fouled on the edge of the box and Jelavic's effort from the free-kick was deflected off the wall but Forster reacted superbly to claw the ball over.
It was a cross that Everton striker Nikica Jelavic would have craved and it was probably that realization that prompted Slaven Bilic to introduce the striker midway through the second half.
But Rangers got their second goal in the 69th minute when Miller slotted home a left-foot shot from seven yards after the ball broke to the striker off Jelavic's attempted volley.
Nikica Jelavic scored the only goal of the game and it could have been worse for Villa with Christopher Drazan having a shot cleared off the line for the dominant hosts.
Jelavic - who scored 21 goals last season, including three against Celtic in the Europa League - is viewed as a replacement for Kris Boyd, who left Ibrox for Middlesbrough under freedom of contract.
The visitors could have doubled their advantage when Weiss robbed Rankin and whipped a tempting cross from the right into the path of Jelavic but the striker was just too slow to connect with the goal at his mercy.
Even so, goals from midfielder Leon Osman and substitute Nikica Jelavic kept David Moyes' team in sixth place, three points ahead of city rivals Liverpool with a game in hand as the club's manager celebrated 11 years in charge.
CNN: Mancini furious as City defeat gives United 15-point lead
应用推荐