I did "Get to the bottom of this"... kinda
I ended up taking the S5 harness out of my other car, and combinging the big power wires with the S4 small gauge connector wires.
The "splice" that nodohc is talking about was corroded to all hell. The S4 harness uses a single alternator wire, and crimps it near the fuse box. The S5 harness runs TWO wires from the alternator instead, making the entire harness a better design.
IT turns out my starter is dying on me as well, because right after I did this, it got stuck, and the car started just fine after I tapped the solenoid... IT seems that the corroded wiring might have worn out the solenoid somehow, so coupled with a bad connection, my starter is old as well.
Luckily, the car has cranked like a champ ever since that fix. Right now, I got an optima yellow battery, a NAPA alternator, and rebuilt wire harness, so if the starter goes bad, it will be the last item in the "starting-charging" system that usually goes bad.
It's not as bad, but the hesitation is still there, so I am going to run a few extra grounds and see what happens.