I generally find, on London/Manchester/Birmingham away days - in fact, any large city - it is exactly as you say; much easier to ditch the dickheads. It is when we are in small towns with only a few watering holes that you end up seeing them.
If we reach the Championship again, I think we'll see less of these people, because of the ticket prices. I can't imagine them paying £34 plus £20 on the coach to go to Birmingham City, when they might be paying £20 to go to Walsall now.
Someone else mentioned earlier that these people have never been on an away day outside of Division Three. They'll have a shock when they go to somewhere like Derby County, with 30000 home fans, some of whom will be game for a ruck. Or a day out at Elland Rd with 30000 baying Leeds fans after we have lost 2-0. They're used to seeing United for the past 5 years, sometimes having 50% of the crowd at away days like Rochdale, Bury, Oldham, Port Vale etc. Wait until they try to go shouting their mouths off at Nottingham Forest or Middlesbrough.