Some years back, the Banks exhaust system on my '86 F-250HD fell off on a trip. It had rusted away after only 18 years or so (nothing seems to last any more ( : < ) and I had to replace it with an emergency system from, "The Llttle Mufflershop of Horrors" in some podunk Ohio farming town. It was a miserable crimp-bent system with a Chinese copy of a "performance" turbo muffler. Yeah, it was 3-inch but severely crimped at the bends and I noticed a slight uptick in EGT in normal driving but I didn't notice any power loss. Didn't tow or wail on it because I knew what the result would be. Couldn't wait to replace it!
After checking on the price to replace the Banks parts (which were still available), and having to be difibrillated back to life, I shopped and installed the back half of the Dynomax 4-inch kit (part #19429, you can look up and buy the individual parts too) for the IDI trucks, running the 4-inch as far forward as possible. That left about three feet of 3-inch from the (non-wasgated) turbo and the 4-inch tied in near the transmission.
The 4-inch swap delivered about 2-3 extra psi boost over that restricted situation, noticeably faster spooling up and EGT about 35 degrees lower at 65 mph. It gained 1 or 2 psi boost over what the Banks mandrill-bent system delivered with that gigantic "canoe" of a muffler Banks used to offer. Didn't notice all that much difference from the original banks on EGT, but it trended lower. I didn't gain any power over the Banks system. One disagreeable result was an exhaust drone at about 2000 rpm.
The choke point is the turbo outlet pipe, which is 3-inch. The outlet pipe size and the turbo map are all carefully calculated so that's an equation you don't want to change without knowing what you are doing (not to mention the turbine outlet is only 3-inch). On my truck, 3-inch is all that will fit between the trans and the firewall anyway.
My opinion is that, on most IDIs running 10-15 psi boost (which is about all they can safely take without compression reduction and an intercooler) a 4-inch system is a zero gain over a 3-inch mandrill bent system with a good free-flow muffler (or no muffler).
If you calculate exhaust flow, even at 10-12 psi boost, 3-inch flows enough anyway, at least when calculated to the Max EGT an IDI can handle anyway. Bigger than what you need is not beneficial. It may not hurt performance (at least on a turbo system, it might on an NA at low speeds) but it may hurt your wallet. I pretty much knew that going in, but the Dynomax system was about 40 percent the price of replacing the Banks parts.