John Key says he was so in the moment he didn't see Speaker David Carter before he kicked him out of Parliament for the first time as Prime Minister.
The debate in the House got fiery yesterday over the Panama Papers and New Zealand's foreign trust industry.
Controversy has surrounded New Zealand's foreign trust industry as a way for people to hide their money to avoid paying taxes in their home countries.
Earlier, Mr Key had linked three international charities -- Greenpeace, Red Cross and Amnesty International to foreign trusts which were made public in an online and searchable database.
There were calls for him to apologise for it because those charities had done nothing wrong.
Green Party co-leader James Shaw asked why he wouldn't say sorry. Mr Key said it proved his point that New Zealanders should take mentions from the database with a grain of salt because it did not necessarily mean they were doing anything illegal.
"I am sorry they're in the database, I don't know all the details. I'm sure there's rinky dinky stuff in terms of how they've been put in there. My point is that they're in the database," Mr Key told More FM this morning.
It was the first time Mr Key was kicked out of the House as Prime Minister, but not the first as an MP.
He joked about the incident today saying "I've done my lines. I've written 500 out."
Mr Key went back to his office following his ejection, and on his way out recalled some of his fellow National MPs saying things like "see ya" to him.
He claims he didn't see the Speaker when he was calling for order in the House.
"There are times where I probably deserved to be thrown out and I'm not saying that wasn't one of them.
"There's a lot of noise down there and I was just focusing on the next thing I was saying to James Shaw which was actually quite amusing but anyway, I got my marching orders."
He defended his record of not being the MP most kicked out of Parliament.
"I'm nowhere near Winston Peters who literally is the Usain Bolt of getting the boot out of Parliament."