Daniel Vettori, Tim Southee and Iain O'Brien all fell to Johnson before Clark bowled last man Chris Martin with a yorker, leaving Daniel Flynn high and dry on 39 not out.
Flynn was bowled by Mitchell Johnson for 29 after helping Taylor add 84 for the fifth wicket and Elliott went to Stuart Clark for nought, giving Clark figures of 3-23.