There is beauty in brutalism - but for many their only choice is to live in a concretopia

It reads like the final manifesto of a manic loner about to carry out a murder spree at a local government building, but for all the pompous toss in the speech on beauty by Transport Minister John Hayes, its themes are worth thinking about.

"It may be that there are even those who still cling to what must have driven some of the post-war planners: that to strive for the beautiful after the horrors of the twentieth century would be to pursue mere triviality or sentimentality," Hayes said.

"Perhaps they think it better for our architecture, like all our arts, to revel in the suffering and brutality of the human experience. Well, yes, life is sometimes ugly. Which is precisely why we must create all of the sublimity of which we are capable – to enthral and inspire; to counter the disappointment and harm which are bound to be part of human frailty.

"Be warned! The descendants of the brutalists still each day design and build new horrors from huge concrete slabs to out of scale; rough-hewn buildings, and massive sculptural shaped structures which bear little or no relationship to their older neighbours. Consider swathes of the worst of our towns and cities; then say that I am wrong."

He's tedious, but not wrong. A lot of our architecture is, by any reasonable definition, hideous. Much of what was built in the post-war period to replace, renew, and expand was done on an austerity budget, quickly, and with a ruthless functionalism in mind.

But let's not dismiss entirely brutalism and its lesser offshoots (much of what is seen to be brutalism is actually just cheap mimicry). As the old cliché goes, beauty is in the eye of the beholder. Yes, a lot of it is ugly. But some of it is fun, powerful, intelligent. Like mock Tudor suburbia, gothic revivalism, and swinging, it's not for everyone.

Hayes' primary concern is transport infrastructure. But where the brutalist tendency has the biggest impact on people's lives, and is therefore at its most contentious, is housing; a sprawling grimness encases large areas of our towns and cities, and it's where many people live.

Though the original intentions of the planners and designers were idealistic – streets in the sky, and all that — these places have come to symbolise deprivation, neglect, crime, and decline. Some research suggests that these estates, their grey tunnels and walkways weaving through like mortar chains, actually facilitate crime and ill health, part of why many are being torn down and rebuilt in a vast programme of regeneration.

There is danger in idealism. Ideas can become more powerful than reality even as people suffer. A vein of this thinking runs through many proponents of brutalism — they cling to the noble intentions and perceived beauty of the products, rather than what life is actually like for people living in these places. We shouldn't get drunk on utopianism.

Choice is what's really important. You, as a middle class professional, may choose to live in a giant breeze block because it's to your architectural tastes. You might like the look of the ominous grey giants in London's landscape as you walk the dog in Greenwich Park or at Parliament Hill, but you have the wealth to choose not to live in these damp gargantuan slabs.

There are, however, plenty of poor people who have few or no housing options, especially in London, where the line councils take is: you get what you're given. These people are forced by circumstance to live in brutalist estates, or other modernist projects with a similar style.

They don't have pricey service charges to maintain the estates and their environs like those who can afford to live in some High-Rise idyll like the Barbican. They rely on hard-pressed local authorities to look after the estates, which is why so many are left to rot. Too many people have no choice but to live there.

What matters most of all is the opinion of people living in these buildings and on these estates. What do they want? How do they feel about life in the unrelenting greyness? Maybe they love it, maybe they hate it. Too often, decisions are made above, around, and for people by a high-minded paternalistic few who think they are doing what's best.

But the views of residents should be the first of any sought out. Neither Hayes nor the brutalists should ignore the experiences of those forced to live in the creations they hate and love respectively.

My family lived in 1960s council towers and were scratching at the walls to escape. When we did, it was sweet relief. I suspect there are many similar families to mine. Fashionable lovers of brutalism are welcome to take our places. It'd be their choice to live like that. But they should remember, as they plead with government not to demolish these urban concretopias, it was never ours.