Winston Peters, in one of his final moments as Acting Prime Minister, called on the Speaker to "throw fatty out" of Parliament.
It is not certain who the comment was directed at, though the preceding dialogue indicates National's Gerry Brownlee was likely the intended target.
The comment was made while tensions were high at Parliament - New Zealand First MP Tracey Martin had just announced the head of an inquiry was stepping down after scrutiny over her endorsements on LinkedIn.
The jeering from Opposition benches was so loud the Speaker said he couldn't hear Ms Martin, despite the microphone.
"I'm first of all going to have the Shadow Leader of the House withdraw and apologise," the Speaker was saying, when Mr Peters' voice was heard through the microphone.
The Shadow Leader of the House is Gerry Brownlee.
"Yeah, throw fatty out," Mr Peters can be heard saying in the clip.
It's far from the first time offensive comments have been made in Parliament.
Last month, Kelvin Davis called National MP Jacqui Dean "hysterical" during a select committee.
In May, the Speaker Trevor Mallard says he heard someone in National call the Prime Minister a "stupid little girl."
After a six-week stint as Acting Prime Minister, Winston Peters will hand responsibility back to Prime Minister Jacinda Ardern as his plane leaves New Zealand at midnight.
Mr Peters is travelling to Singapore to attend the ASEAN Regional Forum and East Asia Summit Ministerial Meetings.