Does Mold Affect Your Brain? … When the toxins enter your body, via skin or air, the toxic gases cause a disturbance in many organs, including your brain. Often confused as just allergies, or “just being out of it,” mold infections can affect cognitive function, especially in the frontal cortical area.