Roger Scruton: A pessimist's guide to life

The taxi drivers of Swindon do not seem to agree with Roger Scruton that his farm is easy to find, and so it is that the philosopher has to drive out and rescue a townie standing helplessly on a grassy verge in Wiltshire. The mud-splattered blue pick-up in which he arrives is almost ostentatiously utilitarian, as is his attire: wellington boots, lived-in, slightly stained pullover, all topped off by a weathered face and wild red-grey hair.

Sunday Hill Farm, when it appears, also looks the part: low buildings of 250-year-old stone, set back from the road by an orchard in bloom. Horses graze nearby. Inside it's dark wood, framed prints all over the walls, haphazardly layered carpets, old knick-knacks and books everywhere – on shelves, in piles on the desk, on tables: The Teachings of Rumi, Poems of St John of the Cross, The Naked Public Square Reconsidered. Scruton's home, and his working farm, reaches for a particular kind of English bucolic, and enacts a particular brand of conservatism.

Scruton's new book, The Uses of Pessimism, is intended as a warning. It is a defence of a world in which freedom is not the "freedom to believe anything at all, provided you feel better for it", but "a precious achievement that human communities have arrived at through many sacrifices, and it requires institutions, laws – it requires a habit of obedience, as much as anything else."

At the same time he believes the state must take up as little of civic life as possible. "All of us have social instincts which prompt us. When we see somebody in trouble, we help. And the great question is, when the state steps in, do they still go on doing this? And actually, they don't – and you find when you look to eastern Europe" – Scruton taught in the underground university in the former Czechoslovakia in the 80s – "when the state took over everything, you find this great vacuum of charitable feeling, which is a huge loss of social capital. I think we still have social capital here because the state hasn't expropriated all these things … The question is how to release it and make it work."

So he believes in Cameron's big society? "It's words, isn't it," he says dismissively. He will add, later, that "it is scarcely a mark of intellectual profundity to have noticed that our society is big", but for the moment prefers to plead an ironic distance – "old geezers like me become steadily less relevant. That's our purpose." There was a time, however, when he did want to be closely involved – when, in 1978, for example, he applied for selection as a Conservative MP and his inquisitor, "Dame Something Something, who conformed exactly to the old image of the blue-rinse maiden aunt", as he put it in his memoir, Gentle Regrets: Thoughts from a Life, was distinctly unimpressed with his Burkean credentials. So instead he published The Meaning of Conservatism, "a defence of Tory values in the face of their betrayal by the free marketeers", became editor of the Salisbury Review, and continued his work as a founding member of the Conservative Philosophy Group, which met in Jonathan Aitken's house. None of it changed his eventual conclusion there was "no such career in England as that of an intellectual Conservative".

He soon discovered that there was probably no such career as conservative intellectual, either. As a professor at Birkbeck in London he was so vilified by his colleagues he concluded that a "socialist conscience was the one requirement for success in the only spheres where I could aspire to it": cultural pursuits (he has written novels, operas, and books on music and art), and academia.

The Meaning of Conservativism, he says, "blighted what remained of my academic career". He now insists that his decision to quit British academia, to concentrate on writing and the US, where he has been much better received, suited him, yet there is an unmistakable relief in his voice when he says that he has been "rehabilitated" – for the past two years he has been professor of philosophy at Oxford.

But while there may be more room in the academy, these days, for critics of what he would see as a leftwing status quo, Scruton doesn't think the situation in politics is markedly different, because, Philip Blond et al notwithstanding, there is "no career comparable to that of, say, Tony Giddens on the left, with open access to both academic and political influence and office". Part of the reason, as he concluded a good 20 years ago, is that there is something inherently paradoxical about being an intellectual Conservative: a good measure of the party's raison d'etre resides in the counter-intuitive belief that ideas "should be inherited and ignored, not acquired and defended. And they never take the form of convictions."

The puzzle about Scruton has always been the way in which humane, often elegant argument – recognisable as such whether one agrees with him or not – co-exists with outbreaks of cranky, ungrounded nonsense. On Pessimism does not, as previous works have done, dismiss his critics as "sanctimonious halfwits", but it does lump together, for example, Y2K, "the apocalyptic vision of global warming … the campaigns against lead in petrol and passive smoking". And it isn't just words: in 2002 he was dropped as a columnist by the Financial Times after leaked emails showed he had offered to place pro-smoking stories in the press for a fee from Japan Tobacco.

Among the ills of the post-1968 cultural revolution are, apparently, the "deconstructionist, feminist, counter-cultural" ideas that encourage the destruction of all hierarchy. Feminism is an ill? Radical feminism only, it turns out, "which tries to overthrow the whole system of thinking on which we have hitherto depended" as opposed to "the tradition of female emancipation, beginning with Wollstonecraft and people like that, which is I think a completely different thing, and part of the natural reform of our institutions and our way of seeing things." Wollstonecraft was, of course, no friend of Burke's; interesting how radicalism, after the passage of years, can safely be absorbed as tradition.

And gay rights activists? In the past Scruton has written that homophobia is understandable. "I took the view that feeling repelled by something might have a justification, even if it's not a justification that the person themselves can give. Like, we're all repelled by incest – well, not all, but most people are. And there's a perfectly good justification, if you look at it in terms of the long-term interest of society. And in that essay I experimented with the view that maybe something similar can be said about homosexuality. And I don't now agree with that, because I think that – it's such a complicated thing, homosexuality. It's not one thing, anyway. So I wouldn't stand by what I said then." Though yes, "people got very cross".

Although he seems to derive a certain satisfaction from seeing himself as a kind of prophet in the wilderness, confirmed in his pronouncements by the degree of disagreement they provoke, there is a kind of fatal attraction to them too, an almost juvenile way in which he can't stop himself from saying outrageous things, or is seduced by the (considerable) power of his own polemic into positions less nuanced than it later seems he intends them to be. And then he is somehow surprised when they cost him friends, or that people should in turn be reliably surprised, when they meet him, to find that he is not a bogeyman – that he is, in fact, good, thoughtful company, if somewhat awkward, and, that although he has belatedly found happiness with a second marriage and children, has a presence veined through with an unmistakable skein of sadness, a slightly flayed vulnerability unexpected in someone who has been attacked so much that one feels they must have developed a thick skin. (It is somehow not a surprise, however, that he once admitted he felt such guilt about the collapse of his first marriage that "he came to see public ignominy as a kind of atonement".)

He once said, in a typically arresting formulation – ie both revealing but too neatly aphoristic to quite convince – that "My life divides into three parts. In the first I was wretched; in the second ill at ease; in the third hunting." The first ended at 16, when he escaped a home dominated by his depressed father, who was the clever son of a drunk who had refused to let him go to school past 14 and for whom class war had become the only way to make sense of his own reality. In his memoir, Scruton does not need to underline the sadness of the moment when he tells his father he has got into Cambridge and his father stops speaking to him. The second part encompasses university, his ill-fated first marriage, his travels through the Middle East and communist eastern Europe, and the end of his academic career; and the third the moment when, accidentally – hacking about the countryside on a bored, borrowed pony that instinctively joined in – he discovered hunting and took to it with such enthusiasm that he was flying back from America every weekend to hunt.

On one of these weekends he met his wife, Sophie: he fell off his horse, and she stopped to pick him up – "You see, if someone stops to pick you up while hunting, she's lost her day, because they've all gone, and you won't find them again. So it was a sign … of something more than just courtesy." She was 22, a recent history graduate; he 50. Their two children are now nine and 11. What has he learned from marriage? "Well," – a short laugh – "it's more having children". At which they appear, right on cue, well-behaved and startlingly self-possessed, and are introduced. Sam, the oldest, is on a day off from boarding school, and his father on the brink of a six-week trip to America; when, later, they exchange awkward goodbyes, my presence feels incredibly intrusive.

Did fatherhood make him think differently about his relationship with his own father? "One of the things it does – and this is purely personal – it made me recognise that, first of all you have to forgive, because you yourself, in your time, must be forgiven. And this isn't something that you can do automatically. You've got to earn the right to be forgiven, and one of the things is learning how to feel grateful – even for difficult things that have happened to you. I'm sure everybody goes through something like that – that process of spiritual discovery, where you recognise that you have been taking for granted things which you shouldn't have taken for granted. And other people might have seemed to be nasty to you when in fact it was your fault for not recognising the difficulties and the sufferings that they had to put up with. [I realised that] only in a condition of mutual forgiveness is life worthwhile." Was he able to do that while his father was still alive? "It was difficult. And it's always too late, of course, when they're dead." I'm suddenly aware he's not sure he can continue, and change the subject.

How has he changed, over the years? "I've always been … soft. Too soft. Yeah – everybody mellows. People – young people tend to be unforgiving, obviously, and impatient, and that was true of me as much as anyone else. But I'm much more disposed to recognise, first of all, that I may be completely wrong about things that I think, and secondly, that even when I'm completely right, that it might be wrong to expect other people to agree with me, and that people should have room to make mistakes."

Too soft is a surprising assessment – "Well, I … I … uh, I've tended to overlook the actual underlying … um … precariousness of human life, so thinking we could all just arrange things by sticking to nice, agreeable procedures, being the decent stiff-upper-lip Englishmen that we've always been, and let the whole thing manage itself. I think that is a kind of softness, because the more I live, the more I see that humanity is always poised on the brink, and can fall into chaos and disaster at any time."

The world according to Scruton

On Thatcher 'I was identified in the intellectual world with this terrible woman – who I never knew, although I do now – and was blamed for everything she did, without me knowing what she would do.'

On politics 'Leftwing people find it very hard to get on with rightwing people, because they believe that they are evil. Whereas I have no problem getting on with leftwing people, because I simply believe that they are mistaken.'

On women 'The disappearance of female modesty and sexual restraint has made it hard for a man to believe, when a woman yields to his advances, that her doing so is a special tribute to his masculine powers, rather than a day-to-day transaction, in which he, like the last one, is dispensable.'

On the Qur'an 'A lot of the Qur'an is, frankly, cantankerous, vitriolic, man-hating stuff.'

On homosexuality 'Our acceptance of the homosexual lifestyle, of same-sex couples, and of the gay scene has not eliminated our sense that these are alternatives to something, and that it is the other thing that is normal.

'This other thing is not heterosexual desire, conceived as an "orientation". It is heterosexual union: the joining of man and woman, in an act which leads in the natural course of things not just to mutual commitment but to the bearing of children, the raising of a family and the self-sacrificing habits on which, when all is said and done, the future of society depends.'