Double-Olympian Sir Peter Snell believed the singlet auctioned off earlier this month was the one he wore at the 1964 Tokyo games.
But the authenticity of the singlet is now in question, and auction winner Te Papa has decided not to go through with the purchase.
Te Papa bought the singlet from Cordy's auction house for $140,000 including auction fees.
It was purportedly worn by Sir Peter in the 1964 Toyko Olympics where he won gold in the 800m and 1500m .
But in a statement this morning, the museum said it couldn't be satisfied about the garment's authenticity.
Both parties have now agreed the sale won't go ahead.
Auctioneer Andrew Grigg says he's disappointed the sale won't go ahead.
"[Te Papa] have done their tests and they've convinced me it isn't the one Peter Snell wore when he won double gold back in '64 in Tokyo."
The material of the singlet did not match the ones of the time.
Cordy's believed the item was the real deal, and did what it could to find out whether it was authentic.
"When you look at it, all the elements were right, when you look at the numbers, the lettering, the name tag in the back - Snell P G - everything about it was right," Mr Grigg says.
Sir Peter told Newshub there was "something fishy" about what had happened.
"The vendor, the person who sold it, wouldn't know because he received it as a package of stuff I believe, so he would have no idea, he would have assumed it was the real thing.
"The whole thing still sounds pretty fishy. For someone to go to that sort of trouble...
"I was aware that [Te Papa] were checking on it. I thought, it seemed to me, to look authentic, but there must have been something else that they're not saying right now."
Sir Peter sold the singlet for charity, but did not know where it ended up.
He has none of his Olympic or Commonwealth Games uniforms or gear left.
"I have no blazers for both the Olympics and the Commonwealth Games, I don't know where those are, the track suits, I don't know where those are. There's always some good cause who are wanting that kind of stuff for charity."
He believes whoever has the real singlet will be "salivating".
Te Papa chief executive Rick Ellis said he was disappointed at the outcome.
"We are very disappointed and we know the public will be too," Mr Ellis said.
"We believe the item was offered for sale in good faith, but our inquiries have shown that the singlet is not the one worn by Peter Snell at the 1964 Tokyo Olympic Games."
Mr Ellis says Cordy's is a reputable business and will also be disappointed at the outcome.
The singlet will now be returned to the vendor.
The museum will ask an independent expert to review the process which led to the auction bid.
Immediately after the auction, Sir Peter expressed his surprise that the museum would spend that much on it, as he'd estimated it would only fetch between $30,000 and $50,000.
He says he's glad now Te Papa won't have to spend the money.
However, he says he'll need to reconsider a plan to have two of his medals sit beside his singlet at the museum.