The Government is putting pressure on manufacturers and retailers to improve safety around button batteries.
About 60 children are hospitalised each year in New Zealand, after getting the batteries stuck in their throats or noses, often with devastating consequences.
In just a matter of hours, a button battery can burn through the flesh of a child's throat.
"The most serious injuries are burns to or through the esophagus, so your gullet, as well as your breathing apparatus and your voice box," says Staship Hospital director of child health Dr Mike Shepherd. "These can be permanent, long-lasting injuries.
"The other main site of injuries is the nose, where it can burn through the structure of your nose and cause permanent deformity."
Button batteries have caused at least 60 deaths worldwide and while there haven't been any deaths in New Zealand, several children have been left seriously injured.
Eash year, Starship sees about 20 children who've swallowed batteries. Some require serious emergency operations, and it can lead to life-long problems with breathing and swallowing.
Symptoms can include, not wanting to eat, crying for an unknown reason or discharge from the nose, but it's often difficult for parents to tell, so the Government wants to make it harder for children to access button batteries in the first place.
Commerce and Consumer Affairs Minister Kris Faafoi wants to turn the focus onto retailers and manufacturers.
"For those who make the batteries, they might be able to make their packaging safer." says Mr Faafoi. "For those who are selling it, retailers might be able to put some messaging out to raise the awareness of the harm they can cause.
And for manufacturers of products that use batteries: "It might be something small, like adding a screw onto a lid, where the batteries are."
Mr Faafoi says he will work with the industry, but isn't ruling out future regulatory intervention to stop more children being injured by button batteries.