Arsenal remain top of the Premiership on goal difference from Chelsea, who drew level on points following their win at West Bromwich. The leaders are three points ahead of third-placed Everton.
Substitute Van Persie, who scored in the Carling Cup win over Manchester City, was buried under relieved colleagues after cracking a 15-yard left-footed curler past the impressive Saints keeper Antti Niemi.
He rescued Arsenal from a second successive Premiership defeat after their loss at Manchester United last Sunday.
Southampton have never beaten Arsenal at Highbury in 13 Premiership visits and this was their 10th league game without a win under head coach Steve Wigley.
They had only scored once away from St Mary's before today and stay 18th despite their precious point.
Arsenal began their 49-game unbeaten Premiership run with a 6-1 win over Southampton in May 2003.
A similar result was never on the cards today, especially after Premiership top scorer Thierry Henry missed a 29th minute penalty.
Southampton defended deep, frustrated the Gunners and exploited their vulnerability to crosses at set pieces.
With Sol Campbell absent through injury, replacement Pascal Cygan often looked the weak link in the Gunners' suspect rearguard.
The watching Panathinaikos scouts doubtless made notes ahead of their Champions' League visit to Highbury on Tuesday – for which Campbell is already a doubt because of a calf injury.
Injuries also cut a swathe through Southampton's squad. They were without strikers James Beattie, Kevin Phillips, Marians Pahars and Peter Crouch today.
Dexter Blackstock, the 18 year old who scored a Carling Cup hat-trick against Colchester, started alongside Brett Ormerod up front.
Jason Dodd, Michael Svensson and Matthew Oakley were also sidelined. Graeme Le Saux was ruled out with knee trouble and Fabrice Fernandes failed a test on an ankle injury.
Henry headed an early chance wide and Niemi grabbed the Frenchman's cross at the second attempt.
But it was Southampton who almost grabbed a 20th minute lead. Mikael Nilsson met Danny Higginbotham's cross with a shot that beat Jens Lehmann but cannoned back off an upright.
Southampton's luck changed nine minutes later when Henry should have broken the deadlock.
Darren Kenton brought down Dennis Bergkamp and referee Matt Messias awarded a penalty despite Saints' protests. Henry sent Niemi the wrong way then cracked the spot-kick against the foot of an upright before Edu volleyed the rebound wide.
Patrick Vieira drove Arsenal forward as they cranked up the pressure before the interval. Bergkamp finished a neat one-two with Vieira by shooting wide.
Higginbotham blocked a fierce Lauren volley then Niemi pulled off a brilliant save in the 38th minute flinging himself full length to tip away a Henry curler.
Henry had a shot deflected against the outside of a post as the second half opened before Edu sent a speculative 30-yarder over.
Robert Pires replaced Jose Antonio Reyes on the hour after the Spanish winger suffered a serious hand injury following a challenge with Nilsson.
Henry finally broke down Saints resistance in the 67th minute. Bergkamp played a through ball, Henry checked his run to beat the offside trap and buried his shot beyond Niemi for his ninth league goal of the season.
Niemi kept Saints in contention with a brilliant 78th minute save from Freddie Ljungberg after Henry's 50-yard run ripped the defence apart.
Niemi's save looked even better two minutes later when substitute Neil McCann swung in a corner and Delap nipped in to head past Lehmann for a shock equaliser.
It was the Saints' first Premiership goal in 477 minutes.
Four minutes later the same combination stunned Arsenal again. McCann whipped in another free-kick and Delap escaped Cygan to head Southampton in front.
Shades of the two goals that Bolton scored in their 2-2 draw at Highbury in September suddenly echoed around Highbury.
But then Van Persie rescued Arsenal.
MAN OF THE MATCH: Rory Delap (Southampton) Scored two fine goals and worked tirelessly in midfield as his side earned a valuable point.