Harley has a video on installing the saddlebag speakers. The process is the same for the tour-pak speakers. You will need an "intermediate" harness to connect the harness that came with the speaker pods to the under-seat connector on the bike. From your post it sounds like you already have one. If so then just watch the Volunteer Audio video that I linked below to see how to get it installed. He goes step-by-step showing how to remove the 4 plastic block-off pins and properly connect the 4 wires from the intermediate harness.
I got my harness from Volunteer Audio for $50 USD. It's a small Y-harness that fits under your seat and has two quick connects so that you can connect tour-pak speakers, saddlebag speakers or both.
Here's the Volunteer Audio video showing how to install it (keyed to start where the speaker wiring bit starts):
Here's the harness from Volunteer Audio:
volunteeraudio.com
Here's the Harley video (keyed to start where the speaker wiring bit starts):
