Lab tests in Australia commissioned by Bannister Law found seven of the brand's sprays didn't live up to the SPF 50 plus advertised.
The best performing sunscreen tested had an average SPF rating of only 20.2 while the worst scored 10.7.
Banana Boat manufacturers Edgewell have questioned the results, saying they're inconsistent with their own and that all their products meet the advertised SPF claim.
"We disagree and question the anomalous SPF results provided," the company said in a statement.
"These results are entirely inconsistent with the testing Edgewell has commissioned at reputable, independent laboratories. All Banana Boat products meet the SPF claim as labeled on pack."