An own goal from Oliver Norwood and a stoppage-time effort from substitute Hakim Ziyech ensured that the Blues are into the last four of the competition for the fourth time in five years.
Thomas Tuchel's side were far from their vintage best, struggling to create many openings, but some impressive individual moments made the difference on the night.
Ben Chilwell's effort was turned into Sheffield's net by Norwood while making an awful clearance. Ziyech, who was also on target against Atletico Madrid in midweek, scored a brilliant goal to confirm the win.
With their win, Chelsea join Southampton and Manchester City in the FA Cup semi-finals. One of Leicester City and Manchester United will complete the last four later in the day.
On that note, let's have a look at the player ratings for Chelsea:
Kepa Arrizabalaga - 6/10
There were a few nervy moments in the second half from Kepa Arrizabalaga, but the Spaniard came out unscathed to register a rare clean sheet.
Andreas Christensen - 6/10
The Dane lived up to his tag of a ball-playing centre-back, as he produced a passing masterclass, completing 91% of his passes. Andreas Christensen also produced a dangerous but crucial block for denying Osborn in the second half.
Kurt Zouma - 7/10
Starting in the centre of a three-man Chelsea backline, Kurt Zouma was rock solid in defence. He read the game impressively, making six clearances, including a vital one in the 90th minute, to keep his team's citadel intact.
Emerson - 7/10
The Italian oozed quality on the night. Dominant in the air, strong with his tackles, and always anticipating danger, Emerson made four clearances, more than anyone in the match except Ben Chilwell.
Callum Hudson-Odoi - 7/10
He was a productive outlet for Chelsea as Callum Hudson-Odoi constantly probed Sheffield's defence with his energetic runs. However, his end product was lacking in the game.
Billy Gilmour - 6/10
The teenager showcased his excellent passing range in the first half, completing 92% of his passes. But Billy Gilmour struggled to cope with Sheffield's pace when they upped the ante for a while after the break.
Mateo Kovacic - 6/10
Mateo Kovacic excelled with his accurate passing and demonstrated his prolific dribbling skills, completing six of them in seven attempts.
Ben Chilwell - 7/10
It was an excellent performance at both ends from Ben Chilwell, who played his part in Sheffield conceding the own goal in the first half. Chilwell also assisted Ziyech for Chelsea's second of the night as he strengthened his case to start at the Euros for England.
Mason Mount - 6/10
Mason Mount, the Chelsea captain on the day, stepped up to the plate once again. He created his team's opening goal from a corner kick and then helped out defensively when Sheffield looked to score in the second half.
Christian Pulisic - 6/10
This was an opportunity for Christian Pulisic to find some form, but the American cut an anonymous figure and failed to truly test Sheffield United.
Oliver Giroud - 5/10
It was not the best game for the Frenchman, as he struggled to link up play and create any chances up front. Giroud also didn't attempt a single shot in 63 minutes of action.
Ratings of Chelsea Substitutes:
Reece James - 6/10
Reece James was crucial in diffusing critical situations for Chelsea late in the match.
Kai Havertz - 6/10
Kai Havertz lost possession a few times and couldn't really fire up Chelsea's attack.
Hakim Ziyech - 7/10
Hakim Ziyech toiled hard until he scored the game-clinching second goal for Chelsea late in the game.
Cesar Azpilicueta - 6/10
McGoldrick made Cesar Azpilicueta look like an average player by outmuscling him a few times late on.
N'Golo Kante - N/A
N'Golo Kante didn't get much game-time to make any impact on proceedings.