There are a few possibilities. One of the two solenoids could be sticking, or there could be an electrical short applying power to one of the solenoids.
IMHO, the fender mount solenoid is less likely to stick because it carries vastly less current than it is designed for (it is designed to carry full gasser starter current). The usual reason that a solenoid sticks is from excessive current which causes the contacts to literally spot weld themselves to each other. That is not to say that it cannot fail.
Quick check - check the continuity between the two large terminals on the fender mount solenoid, if you have continuity then it is stuck. Although often they will un-stick once current is no longer flowing and things cool down.
Next check, unhook the (heavier) starter wire from the fender mount solenoid. Now hook up one of the batteries. If the starter still engages, then the starter mounted solenoid is stuck.