Blackcaps spinner Ajaz Patel carved through Sri Lanka's top order and took five wickets to even the odds for the visitors after the second day of the opening test in Galle.
New Zealand were bowled out for 249 in the first session, but Sri Lanka struggled to get going during Patel's spells and finished on 227/7, trailing by 22 runs.
Openers Dimuth Karunaratne and Lahiru Thirimanne patiently saw off the new ball attack of Trent Boult and Tim Southee before Patel was introduced as early as the 11th over in spin-friendly conditions.
The left-armer, who finished with 5/76, struck in his first over to have Thirimanne stumped. Karunaratne was next to go with a similar delivery that trapped him leg before with the score on 66/2.
Kusal Mendis and Angelo Mathews built a 77-run partnership for the third wicket before Mendis (53) was dismissed the very next ball after bringing up his 10th half-century just before tea.
After a middle-order collapse, where three batsmen added only six runs, wicketkeeper Niroshan Dickwella (39 not out) and Suranga Lakmal (28 not out) wrested back control with an unbeaten 66-run partnership to keep the test evenly poised.
Earlier, Ross Taylor, unbeaten on 86 at stumps on Wednesday, fell to the very first delivery he faced, missing out on his 19th century after edging fast bowler Lakmal to the wicketkeeper.
After Akila Dananjaya picked up five wickets on the first day, Lakmal (4/29) quickly cleaned up the tail.
The two-test series, with the second match set for Colombo, is the first for both teams in the newly launched World Test Championship.