Brendan Rodgers’ side looked to be on course for a hard-earned point on his return to Anfield after James Maddison’s low finish 10 minutes from time equalised Sadio Mane’s 40th-minute strike.
And then, with the clock ticking into stoppage time, a communication breakdown between Leicester keeper Kasper Schmeichel and substitute Marc Albrighton ended with the latter bring down Mane.
The penalty was awarded after consultation with the video assistant referee and Liverpool manager Jurgen Klopp celebrated wildly as Rodgers turned away in despair.
Leicester will feel they had done enough to earn a point and their frustrations boiled over at the final whistle, with substitute Ayoze Perez involved in angry exchanges with several Liverpool players – who were celebrating increasing their lead over Manchester City at the top of the table to eight points.
City can reduce the gap to five once more when they host Wolves on Sunday (14:00 BST).
Liverpool forced to sweat again
Leicester looked to have seen out the worst to claim a point. However, it was no surprise Mane was involved in the decisive incident the Foxes players contested so hotly.
Mane was not just his usual attacking threat – scoring his 50th top flight goal for the Reds after racing on to Milner’s pass and expertly beating Schmeichel in the first half – he also worked tirelessly in his defensive duties to subdue the threat of Leicester full-back Ben Chilwell in the first half.
The Senegal forward now has world-class status and came to Liverpool’s aid once more when it looked like the visitors would repeat their feat of last season and leave Anfield with a draw.
They are also unbeaten in 44 league games at home – 34 wins and 10 draws – which is the longest current run in Europe’s top five leagues (England, France, Germany, Italy and Spain).
The hosts were not at their fluent best but, like at Sheffield United last time out, they got a break when needed and cashed in.