Chances are you might have enabled "Direct Monitor" on your audio interface, thus you hear 1) the faster direct sound from the interface and 2) the slightly slower processed sound from the PC/Mac (since it goes thru AD and DA stages).
To eliminate the delayed sound, make sure you turn off "Direct Monitor" on your audio interface.