It's understood the substance had been in the museum for a significant period of time, police said.
"This is a long-term issue that requires long-term solutions."
He has been charged with wounding with intent to cause grievous bodily harm.
There are no reports of injuries but officials say State Highway 5 is expected to stay closed for some time.
The public rushed to help the man and some also followed the alleged offender to help police find him.
The ground surrounding the kiwi was covered in dog and human prints.
"Our heart goes out to Graham and the Waipawa Four Square whānau."