West Ham crashed out at Wigan while Tottenham Hotspur needed a late Harry Kane equaliser to avoid a shock loss at fourth-tier Newport County.
Will Grigg was the Wigan hero of the tie at the DW Stadium, scoring both goals as the League One side stunned the 10-man Hammers 2-0.
The Northern Ireland international opened the scoring after just seven minutes and then added the second from the penalty spot on 62 minutes.
To compound West Ham's misery, Arthur Masuaku was sent off early in the second half for spitting at Nick Powell in an off-the-ball incident.
"It was despicable," Hammers manager David Moyes said of Masuaku's conduct.
"He will deserve everything he gets and he will get something he deserves from us. It's unacceptable."
Liverpool have been dumped out of the FA Cup with a 3-2 defeat at home to West Brom in a match littered with video assistant referee (VAR) referrals which caused confusion on the pitch and in the stands.
Three major incidents in the first half required the intervention of technology - including the first time a match referee has used a pitchside television replay.
And although it eventually led to the correct decisions being made the time it took, with almost four minutes elapsing between Mohamed Salah being fouled and Craig Pawson awarding a penalty which Roberto Firmino missed, left supporters and players on all sides angry and frustrated.
"We know the system's in play and you've got to keep focused until the referee gives a goal," West Brom defender Craig Dawson said about the VAR confusion.
"It's just the time delay, obviously, and there was a lot of waiting around and uncertainty.
"You've just got to keep focused and keep ready for when the play starts again."
It detracted from an impressive Baggies performance who became the first team since Real Madrid in October 2014 to score three goals in the first half at Anfield.
Jay Rodriguez netted twice in the first 11 minutes after Firmino's fifth-minute opener, with Joel Matip's own goal on the stroke of halftime gave the visitors a two-goal cushion.
Salah's first FA Cup goal, and his 25th of the season, set up a thrilling finale but the Baggies held on to end Liverpool's 19-game unbeaten home run.
Spurs survived a scare at League Two Newport after falling behind to a Padraig Amond header in the first half.
Kane spared their blushes with a close-range tap-in eight minutes from time, for his 30th goal of the season.
"The key for the game was to match the motivation, desire, fight and challenge of Newport," Spurs manager Mauricio Pochettino said.
"But the team didn't show really that they want to go to the next stage of the FA Cup.
Leicester City had no such concerns as they eased past their League One opponents Peterborough 5-1.
New signing Fousseni Diabate and Kelechi Iheanacho both struck twice with Onyinye Ndidi also on the scoresheet, while Andrew Hughes replied for the well- beaten hosts.
Glenn Murray netted a 90th-minute winner as Brighton & Hove Albion snatched a 1-0 win at Championship side Middlesbrough.
Jack Stephens's fourth-minute goal proved the difference as Southampton beat Watford 1-0 in their all-Premier League clash at St Mary's, in Javi Gracia's first match in charge of the Hornets.
Premier League bottom side Swansea City were held to a 1-1 draw by League Two promotion challengers Notts County while Huddersfield's clash with Birmingham also ended 1-1.
Socceroos midfielder Jackson Irvine played 87 minutes as Hull City beat Nottingham Forest 2-1 in their all-Championship contest.
Ben Thompson scored a late equaliser as Championship side Millwall drew 2-2 with League One strugglers Rochdale.
League Two Coventry City beat higher division opponents as Maxime Biamou's 64th- minute strike earned the Sky Blues a 1-0 win at League One MK Dons.
An 80th-minute penalty from Billy Sharp gave Sheffield United a 1-0 win over Preston North End.
PA
