Yes but take a look at 1More Triple driver, 1More Over ear, Denon AH-MM400, Panasonic RP-HD10. You can find headphones that go either way.
I'm not arguing here I know lower output impedance is preferable, but I am just saying that it doesn't always matter as much as you think.
I have a DX7, A30, D30, M8, Fiio K1, and JDS O2... my A30 passthrough to my O2 to use headphones that are affected by output impedance.
Also in the last example of...