Peterhead 2 Hearts 1
Jason Brown had opened the scoring with a powerful header, and the winner came in the 90th minute as Rory McAllister stroked home after Brown's brother Jordon had been felled.
Kyle Lafferty took his strike rate for Hearts to four goals in three games as he bagged the leveller against Peterhead, but the Jam Tarts were ultimately shocked by a Peterhead team no strangers to a cup upset.
Jason Brown had opened the scoring with a powerful header, and the winner came in the 90th minute as Rory McAllister stroked home after Brown's brother Jordon had been felled.
The Blue Toon started brightly and even before they took the lead, Willie Gibson had curled a free-kick narrowly beyond Jack Hamilton's left hand upright.
The opener should have come a minute before it did as McAllister flicked the ball over Hamilton for strike partner Russell McLean, but with the teenager thinking he had scored against his former club, Jordan McGhee popped up to clear off the line.
But from the resulting corner, Gibson's delivery from the right was met powerfully by the head of Jason Brown, finding the top right corner.
Hearts began to adapt to the game, however, and striker Isma sent a low show wide just moments before Lafferty found the net from close range after a Michael Smith cross.
Hamilton fumbled a McAllister free-kick just before the half hour, with Krystian Nowiak doing well to clear.
However, after that the chances began to fall the way of Ian Cathro's men.
Jamie Walker in particular was impressive, and he tested Greg Fleming with a low strike from 20 yards before turning provider as Isma curled narrowly over.
The second half continued to ebb and flow, but Hearts' superior fitness began to tell and it took some plucky defending for Peterhead to remain in the game.
And they were certainly rewarded when Arnaud Djoum bundled Brown over in the final minute, with McAllister remaining cool to slot the ball low to Hamilton's right as the keeper went the other way