No. You leave the amp completely disconnected. You don't need this anymore as your new head unit has a built in amplifier.
The larger connector you removed from the amp plugs into the bypass lead, the bypass lead then plugs into the ISO lead (or in your case will need joining with terminal blocks), the ISO lead then plugs into your new head unit.
Fitted the unit yesterday, I unplugged the bigger connection from the amp and plugged that into the bypass cable which i plugged into the head unit but the smaller connection is still connected to the amp as there was no where to connect that to. The stereo works and sound from all speakers. Where should the smaller connection from the amp plug into?