You may have a "lazy" vacuum pump. The one on my 92 (previous truck) was this way. It would do something very similar to what your are describing. It would give 1 or 2 boosts and then get behind. When it really showed the issue was leaving the driveway. Fire up the truck, back out and maneuver to leave, sometimes the last time you would hit the brakes, the pedal would start to get hard. Also, if it was cold (air temperature) it would take a few seconds to build vacuum at startup. Connected straight to the booster, it would do this. I'd say the pump should help a lot.
When I've had boosters fail, when you would apply the brakes, you would get a loud swoosh sound from the booster. Not all fail this way, but the ones I've had fail, failed this way.
When I've had boosters fail, when you would apply the brakes, you would get a loud swoosh sound from the booster. Not all fail this way, but the ones I've had fail, failed this way.