Grant McCann put Barnsley ahead with two superbly-struck free-kicks from long distance before Southampton began the process of clawing back the deficit.
McCann struck first in the 27th minute when he bent a free-kick over the Southampton wall from 25 yards to give goalkeeper Kelvin Davis no chance of saving.
Southampton twice hit the post through Bradley Wright-Phillips and Adam Hammill before McCann scored again from a similar position in the 40th minute.
McCann's free-kick from the right side of the area was perfectly placed and again left Davis clutching thin air.
Southampton were thrown a lifeline in first-half stoppage time when Marek Saganowski scored from two yards to set up an exciting second half.
Saints manager George Burley replaced Wright-Phillips with Stern John for the start of the second half and Southampton took immediate control.
But Jhon Viafara and John were both guilty of bad misses and Heinz Muller made a great save to deny Andrew Surman.
Barnsley had Dominik Werling sent off for a second bookable offence in the 70th minute and had to defend stubbornly as Southampton committed men forward in a desperate search for an equaliser.
Six minutes were added on at the end of the match and almost immediately Southampton drew level with the goal they deserved on pressure alone.
John burst into the area and crossed from the left and substitute Inigo Idiakez steered the ball into an empty net.
But as Southampton celebrated they were caught with a sucker-punch when Devaney burst clear and drilled a shot against Davis.
The Southampton goalkeeper was unable to hold on and Devaney made absolutely sure from the rebound.
Devaney was booked for ripping his shirt off in celebration and Southampton failed to break down the obstinate Barnsley defence in the four minutes which remained.