Prime Minister John Key has apologised in Parliament for shouting "you back the rapists" at Opposition MPs during a rowdy debate last month.
At the time, Labour and Greens demanded an apology and several women MPs walked out after identifying themselves as victims of sexual violence.
Speaker David Carter couldn't order Mr Key to apologise because he didn't hear the comment at the time it was made.
A petition carrying 13,000 signatures was handed in to parliament asking the Prime Minister to apologise, but he didn't.
Today he said he had reflected on his comment.
"On this last sitting day, so close to Christmas, I would like to withdraw and apologise for that response," he said.
Mr Key made the comment after Opposition MPs had accused the Government of not looking after New Zealand criminals who were being deported from Australia.
He said the MPs were more concerned about the criminals than they were about protecting people from them.
After Mr Key had given his apology, Labour leader Andrew Little stood to say he regretted comments made by his MPs about Mr Carter.
Mr Little and two of his MPs have been referred to the privileges committee for comments that could be interpreted as meaning Mr Carter is biased, which is an offence.
"I would like to draw the attention of the House that correspondence has gone to the chair of the privileges committee accepting that those comments were unparliamentary and expressing regret for them," he said.
That means the case against Mr Little and his MPs won't go ahead.