The mom of mine used to employ a saying that she utilized all of the time.
She'd point out "Don't throw the baby away with the bathtub water". I will admit that for a long time the single reaction of mine was "Huh?