There are basically three factors when it comes to setting the lowest buffer size (latency) without crackling and popping noise, they are Audio interface, CPU and RAM on your computer.
Using an Audio interface with a good quality driver and a good AD/DA conversion is definitely important to the sound quality and the latency issue, but it is also important that your computer is powerful enough to handle the lowest buffer size that can be set,
Using a MacBook Pro (2.5 GHz Intel Core i5) with 16 GB RAM, and a RME 802 interface, the lowest buffer size one can use without noise at 44100 sample rate is 32 samples.
Please also refer to the links below:
3.Optimising your PC for Audio on Windows 10 – Focusrite Audio ...