Former Leigh scrum-half Jay Duffy proved the difference between the two sides, kicking a crucial drop-goal eight minutes from the final hooter to break the Leythers’ hearts.
This was an end-to-end game, with Lee Greenwood opening the scoring for the Northern Rail Cup winners before Ashley Lindsay replied.
This pattern was repeated; tries from Dana Wilson, Rob Roberts and Danny Halliwell ruled out in turn by Craig Lingard, Jamie Stokes and Chris Langley.
This left the score poised at 22-22 before Duffy had the final say, putting the Bulldogs through to the next round.
Batley will now play Whitehaven at the Recreation Ground next week after the Cumbrian side secured a relatively comfortable 38-10 win over Rochdale Hornets.
Rob Jackson touched down for a hat-trick of tries for Haven with Aaron Smith, Spencer Miller, Craig Calvert, John Duffy and David Seeds getting in on the act.
Meanwhile, in National League Two Danny Kirkwood scored a hat-trick as Featherstone Rovers came from behind to beat Gateshead Thunder 46-18.
The north-east side had held a 12-4 lead early in the piece at Post Office Road but Rovers recovered to progress to the next round.
Featherstone will now take on Swinton Lions who scored four tries to beat Barrow Raiders 26-20 at Park Lane.