Police are urging people to report the discovery of bombs rather than delivering them to the local cop shop after the Gisborne Police Station was forced to close when an Air Force practice bomb was brought in.
The station was closed on Sunday after the bomb - placed inside a plant pot - was brought in along with a number of other ballistic-looking items.
A section of the main street was also cordoned off.
A Defence Force bomb squad were to destroy the items on Monday morning.
The man had potentially put himself and others at risk, Eastern District Police said in a Facebook post.
"Police urge anyone finding suspicious items to leave the items where you find them and to call your local station.
"For your own safety and the safety of others, only experts should handle items of this nature."