But a resurgent Heed stormed back into the game and earned a crucial 4-2 win thanks to a brace from top goalscorer Macaulay Langstaff and a goal apiece from Connor Malley and Olley himself.
The importance of the result was further enhanced when news broke that title rivals Brackley Town had fallen to a 1-0 home defeat against Boston United.
The results mean Mike Williamson’s side hold a four-point lead at the top of the table with just four games left in the season.
Championship transfer news: £13.6m ex-Sunderland flop returns after loan as Middlesbrough transfer list trio
Sunderland AFC transfer news: Former £10m-rated 'target' joins Joey Barton at Bristol Rovers in League One
Championship news: Ex-Sunderland, West Brom and Aston Villa man snubbed for Birmingham City job
Manchester United close in on deal for Newcastle United 'target'
Sunderland's strongest starting XI following Daniel Ballard’s arrival from Arsenal plus new contracts
Rather than focus on the results of their main title rivals, Olley just wants his side to producing the performances that has taken them to within sight of a return to non-league’s top tier.
He told The Echo: “We do our job first, that’s what matters.
“I didn’t hear a thing on the pitch when news from elsewhere was going around.
“I was focused entirely on our game and what we could do to affect it.
“When the game was finished, I was told about the other results but it’s about us doing the right things.
“Maybe the defeat against them (Brackley) was the best thing that happened because it made us switch on again, refocus and made us realise what we had to do over the coming weeks.
“We’ve shown that we can do it, we just need to keep going.”
A trip to play-off contenders Kettering Town is next on the agenda for Gateshead this weekend.
Olley has warned his team-mates they have achieved nothing yet as they prepare to visit a Poppis side that currently occupy the sixth and final play-off spot in the National League North
“It’s another away performance we need to put on,” he explained.
“They are fighting for a play-off place so we have to show the same attitude.
“No game is easy in this league but we are confident.
“We haven’t won anything yet and we still have a long way to go until we do.”