Sorry if this is over-generalized, and I know I'm leaving a lot out, but...
Back in the 60s, master volumes were almost unheard of. Overdriven sounds were simply the byproduct of turning the amps up load (which was necessary because good PAs were almost unheard of, too). If you wanted more sustain, more drive, you had to use a boost. There wasn't a lot of choice there, either, but some true classics came from that era: Rangemaster, Tonebender, Fuzz Face, Big Muff, etc.
In the 70s, both Marshall and Fender (among others) introduced master volumes, most notably IMHO, the JMP 2203. At the time, the argument was between the "full" sound of a NMV amp vs. the "fizzy" tone of a MV amp. Once either was cranked to 10, there wasn't a lot of difference, but neither is what we now consider high-gain. Boost pedals evolved, too. The sound of a Rat or an MXR Distortion+ into a Plexi or a 2203 is still as relevant today as it was 40 years ago. But we still wanted more...
In the 80s, we finally got into true high-gain territory with the Boogie Mark Series and many flavors of modded Marshalls. These are also still relevant sounds, but we still wanted more. As the 80s progressed, many builders started refining these designs (think Soldano, Bogner, Fryette (VHT), etc.) in an effort to give the sound more focus, more definition, more tightness, more feel, more aggressiveness... This continues to this day with builders like Diezel, Friedman, Cameron, Fortin, Engator, etc., etc., etc.
FourT6and2 makes a great comparison with painting. These are all different colors. Maybe we started out with red, but now we have bright red, Ferrari red, fire engine red, rocket red, candy red... sometimes we need candy red, sometimes just plain old red is exactly what's called for. The additional colors don't make any of the rest any more or less relevant; they're all just different shades. It all comes down to what the particular painting calls for, on any given day, for any given painter.
There's no way that my old Ross Distortion (my first pedal) into my JCM 800 2204 or JMP 2203 is ever going to be as searingly aggressive as plugging into my Diezel Herbert, but it still brings a huge grin to my face and is still amazingly fun to play, if that's what the music I'm making calls for. On the other hand, sometimes the Herbert is exactly what I need and nothing else will do. The smile it brings to my face is just as big. Sometimes, I need the vocality of my Bogner XTC. Sometimes, I need the fluidity of my Boogie Studio Pre and Switchtrack 395. Sometimes, I need the raw aggression of my Fender 6G6B Bassman.
I guess my point is that they are all just different flavors. Comparing a boosted MV Marshall to a Friedman is like comparing a Soldano SLO to a BE-100. It's really apples and oranges. All tube amps share similar circuitry. There are only so many ways to wire up a tube to make it produce gain, but the devil is truly in the details, which is why a Friedman doesn't sound like a Cameron. And thank God for that!! There are so many colors available today, so much more left to be discovered.
Anyway, I hope this helps a little. I have a mix of older and newer amps, and boxes full of boost/drive pedals. I'd be more than happy to discuss more specific details of anything I have or why I've made the choices I have. At the end of the day, the only thing I think that truly matters is the tone in your head and how best to get there and make it come out of your speakers.
