The size/# of speakers affect the tone, which can make a big difference in perceived volume and make a 1x12 sound small compared to a 4x12, even if the actual volume isn't that much different.
That's not to say that a 1x12 won't actually be more quiet, but to answer your question, it's not like using a 1x12 will be nice and quiet for late-night recording, unless you're in a big detached house with understanding roomies/girlfriend.
I had a similar issue. I live in a tiny apartment in Paris where you can hear all the neighbors. I have a Tubemeister 36, so the attenuator helps, but even playing at 1 watt with a 1x12 will drive the neighbors (and my wife) nuts.
My solution was to get a ZT Lunchbox cab. It's got a 6.5" speaker, has the right impedance rating, can handle the wattage (at least at 1 watt and 5 watts which is what I use it at) and can be driven for decent tones without getting stupid loud.
It's not the ultimate cab by ANY means...and I realize that posting about such a pussy cab on Rig-Talk is sacrilegious, but for the problem that you and I have, it's been a great solution. And it's more than enough to get great tones out of my tubemeister.
But if you don't have an attenuator, then it's gonna be tough :/