In this heartfelt polemic, based on his family's experiences in the health service, our writer argues that it suffers from deep flaws – and we are wrong to ignore them

It was a simple thing. Another blood test, some more investigations into whatever flawed gene or missing protein might be the cause of my daughter's troubled life, with her terrible seizures, her blindness, her inability to walk or talk or eat unaided. Over the past 15 years, there have been many such attempts to identify her condition.

One year later, we asked the doctor, a top geneticist at one of the world's most famous hospitals, what had happened to the results. His office told us a rambling story about financial restrictions and the need to send such tests to a laboratory in Germany. They said there was little he could do but promised to pursue our case.

It was a bare-faced lie. The precious vial of blood had been dumped in storage and forgotten. The following day it was despatched to a laboratory in Wales and 40 days later the specialists came up trumps. They identified her condition, an obscure genetic mutation called CDKL5.

The breakthrough was rather mind-blowing, giving us some peace of mind and the chance to talk to families of the hundred or so other children worldwide identified with the condition. It was also life-changing, since it means our other child and close relatives are in no danger of passing on the condition. Indeed, had we known sooner we might have even tried for more children.

But the most shocking thing was not the lying. Nor even the incompetence. It was our total lack of surprise at the turn of events, since after 15 years suffering from the failings of the National Health Service we are prepared for almost any ineptitude.

Of course, everyone loves the NHS now. It is officially sacrosanct. Our doctors are deities, our health care the envy of the world. And anyone who says anything different is an unpatriotic schmuck who should go and join those losers in the United States. (Although American doctors terrified of litigation would have done all the tests possible on my daughter if I'd sufficient insurance, and would think twice about lying to patients.)

So forgive a harsh dose of reality. I used to share these delusional views, wrapped in a comforting blanket of national pride over Bevan's legacy. But that was before the birth of our daughter sent us hurtling into the hell of our health service. Since then, hours and days and months and years have been spent battling bureaucracy, fighting lethargy and observing inefficiency while all the time guarding against the latest outbreak of incompetence.

Despite my daughter being under palliative care, my wife currently spends two hours a day struggling against the system, to say nothing of the other endless appointments that go with being primary carer of a severely disabled child. Right now, following some dramatic hormonal and physical changes, we are waiting to talk to one of our daughter's doctors: the first call went in three weeks ago, followed by three more phone calls and one email. No reply yet.

Or take the request for a bigger size of nappies, urgently needed because of our daughter's sudden weight spurt. A simple thing to sort, you might think. Not in the parallel universe of the NHS. It has taken four weeks, three phone calls, two home visits from community nurses to assess our needs and fill in the requisite forms – and still looks like being one more week before there is any hope of delivery. It may seem comical, but the result is a distressed child and endless extra laundry.

The warning signs of what lay ahead came on our first visit to Great Ormond Street, when there was a young couple who had travelled down from the north-east of England in front of us, their tiny sick baby almost lost in its blankets. "Didn't anyone tell you – your appointment's been cancelled?" the receptionist told them breezily. They looked at each other despairingly.

Such insensitivity is all too typical. When my daughter was seven she underwent a major review at a specialised unit in Surrey, spending three days and nights with sensors connected to brain-scanning devices glued to her head, under constant video surveillance while my exhausted wife comforted her and stopped her ripping off the electronic pads. A huge strain, but worth it given the hope of a breakthrough. When we went to get the results a few weeks later, there was the usual wait. After eventually summoning us, the neurologist asked why we were there. Then she opened our daughter's notes and asked what was wrong with her. Then she couldn't find the results. We stormed out, me in fury, my wife in tears.

There are countless other examples. The celebrated neurologist who measured our heads before blithely asserting that our daughter – suffering up to 30 fits a day – would just have a slightly lower IQ than the average person. The GP who gave her an MMR injection against our wishes, despite warnings it might prove fatal. The nurse who, having been told our daughter was blind, asked if she would like to watch a video. And that is to say nothing of the endless minor irritations: the over-crowded waiting rooms, the blasé receptionists, the unanswered emails, the blinkered attitudes to people with disabilities.

It used to be said money was the problem, but that fails to explain why American health outcomes are not drastically better, given their profligacy. Or indeed, why Scottish death rates from heart disease, cancer and strokes were so much worse when spending levels were one-fifth higher than in England; it cannot be blamed entirely on haggis suppers. And it is striking that for all the money poured in recently, there is little evidence of further improvement in cancer survival rates, for example, or of solving the postcode lottery.

There is no doubt that nearly tripling the health budget in a decade has led to visible advances, especially in the infrastructure. Some of the new hospitals are vast improvements on the crumbling Victorian buildings they replaced, and seemingly small things such as spruced-up waiting rooms and toys for children make a big difference. Unfortunately, it is equally clear that billions have been wasted, poured into a centralised monopoly that focuses on the manipulation of a target culture rather than delivery and innovation. It was little surprise to learn that more managers than doctors were hired last year. And all too often these managers seem to reinforce rather than challenge the patronising attitudes that often predominate, while failing to tackle glaring waste.

One visit to the gastroenterology department of a major teaching hospital summed up many of the enduring problems. Like any hospital regulars, we booked the first appointment to ensure the wait would not be too long. The young consultant was courteous and empathetic, going out of his way to explain the pros and cons of the invasive surgery under discussion. At one point he needed to call a colleague, so picked up the receiver of an old phone on his desk rather than the high-tech device jutting out of his computer screen. He explained that the new system cost £3m but didn't work properly, so no one in the hospital bothered to use it.

After 10 minutes, we left his consulting room. The waiting area felt tense, with harassed parents, bored children, raised voices and too few seats. This unfortunate doctor had to see more than 50 patients during his two-and-a-half hour clinic – or one patient every three minutes, with no time for reading notes, let alone a break. And we had already ruined his schedule. No wonder people were getting exasperated.

These are, of course, just snapshots over more than a decade. We may have been desperately unlucky, and friends who have suffered heart problems, cycling accidents or had very premature babies will testify to flawless treatment. But then I know of other friends with equally terrible experiences of arrogant doctors, disinterested nurses, lost files and suchlike. I could tell you of the single mother in Scotland rung in the middle of the night and asked if she would like doctors to resuscitate her profoundly-disabled child – and then they did nothing until the mother reached the hospital and berated them. Or the parents of another child with a life-threatening tumour whose care was a litany of mistakes, but when they complained to the hospital's chief executive the notes went mysteriously missing. Or the elderly cancer patient constantly ignored by her doctors. And so on and so on.

For all the rhetoric, this is daily reality in our health service. This is not to denigrate the many fine workers, both on the frontline and behind the scenes. We have come across doctors, nurses, paramedics, therapists and many others who have been supportive, caring and inspirational. Some have gone way beyond the call of duty to help in times of distress or difficulty, such as our palliative care team and the community nurses. But equally, we have come across too many ground down by a sclerotic system that crushes out the idealism or caring nature that presumably made them join the health service.

Clearly there is systemic failure. And it is a question of management, not money. Some of the worst problems encountered have been at the hallowed Great Ormond Street Hospital for Sick Children, which uses the strength of its brand to suck up money and increase its reach. Many in the medical world are infuriated by its endless growth, but scared to take on the behemoth. But behind the soft-focus fund-raising and cuddly image lurks inefficiency and, all too often, needless insensitivity.

Indeed, should you feel moved to give money to help sick children, I would advise you to give to the children's hospice movement instead. As I write, my daughter is at Shooting Star in Hampton, Middlesex, a particularly deserving recipient. It is interesting to note that this sector, which derives a paltry five per cent of its income from statutory sources, does not seem bedevilled with the woes that afflicts so much of the public sector.

Anyone who has used health services in other Western nations knows that visiting the doctor or a hospital does not always have to be a frustrating experience. It is possible to run a health service around the needs of the patients, with appointments kept, notes read and consultations in a pleasant, friendly environment.

Given the swelling black hole in public finances, ageing population and rising costs of health care, Britain needs a serious debate about the future of the NHS. Sadly, the indications of the past fortnight are that we are too infantile to have such a discourse. A deranged Tory MEP became engulfed in the crossfire over Obama's reforms after some fatuous remarks in the US media, and back home – in a depressing foretaste of the election campaign – Labour uses it to smear the Conservatives, and panicked Tories rush to pay homage at the altar of Aneurin Bevan.

For all the supposed cost-effectiveness of the NHS, no other country has followed our model, despite what some Republicans might claim. Instead, we should be looking at what we can learn from abroad. No one in their right mind would want to import the American system here. But there are elements to admire: their popular community hospitals, the emphasis on effective diagnosis, even aspects of the much-derided compensation culture. And turning to Europe, there are systems that enshrine consumer choice, meld public and private systems, are cheaper than our own and have better health outcomes.

France is famous for its centralised approach to government. It also performs well on almost all health rankings, and has been top-ranked by the World Health Organisation. Its insurance-based scheme appears a chaotic blend of public and private partnership, but in reality is a sensible solution that blends the interests of patients with the need for some centralised direction, professional autonomy and safeguards for the poor. Like elsewhere in Europe, it has found a way that for all its faults harnesses the benefits of competition within a universal, patient-centred system. We are fumbling our way there, but it is one step forward and three back.

So what should be done here? I can only offer a few suggestions towards a wider debate. Firstly, it seems obvious that any organisation employing 1.5 million people is going to struggle with the concept of dynamism. I suspect the Chinese People's Liberation Army and Indian Railways – two other similarly-sized employers – suffer from similar deficiencies. Surely it makes sense to break up the monolith, thereby introducing genuinely competitive elements while retaining the principle of state-financed care that is free at the point of use. The more patient choice, the better the service will be. And trust me, patients can make highly-complex choices when it comes to their own health.

Secondly, the target culture should be made less proscriptive and the quality of managers raised. I don't mind managers, just bad managers. Thirdly, these managers and all the medical staff should be given greater freedom to experiment and innovate. This means some failures, but it is vital in any giant organisation. And the Government provides a safety net. Fourthly, there needs to be as much transparency as possible, covering everything from spending to surgical outcomes. This is the information age, after all – and it is our money and our health service.

Fifth, health workers must all realise they are meant to be serving the public. I wonder if medical schools should place greater emphasis on personal skills rather than just narrow academic criteria. And has the drive towards graduate nurses necessarily been a total boon for the care of patients? Finally, politicians should stop trying to micro-manage the NHS – and in return voters and, yes, the media should stop blaming them for everything that goes wrong.

Over the years, I have raised these issues with many politicians. I suggested to William Hague when he was Tory leader that he just tell the truth to the electorate and admit the NHS was a disaster zone. He laughed, and replied that he couldn't possibly say such a thing: "You're far too right-wing on health for us."

Later, I wrote an article for a weekly journal that ended with a challenge to the then Chancellor, Gordon Brown, at the time that Tony Blair and Alan Milburn were coming to terms with the need for root-and-branch reform. Unfortunately it was delayed a couple of weeks, coming out on the day of a group breakfast at Number 11. As I entered the dining room, Mr Brown gave me a wolfish smile and ushered me to sit down between him and Ed Balls, before the pair took me to task for the next half hour. Both seemed unabashed statists when it came to health, who saw more money as the answer to all problems and had little sympathy for the idea of introducing competitive or patient-led elements.

Likewise, David Cameron's experiences have turned him into a cheerleader for the NHS. He is angered by the failures of specialist education and shortfalls in respite provision, but was genuinely moved by the healthcare offered to his late son, as I know from many discussions with him. Days after becoming leader of his party we met for dinner. "I am not going to do what you want on the NHS," he said. "I will reform it if I get the chance, but I won't rip it apart."

Then there was the senior Labour Cabinet minister who told me about the nightmare he was enduring with his elderly relative. "I used to think you had been driven a bit nuts on the health service," he concluded. "Now I think you don't go far enough. It's awful. Absolutely bloody awful. We've got to do something about it."

I won't hold my breath.

Like the health secretary, I am an Everton fan. And like Andy Burnham, the national health service and Everton are among the most cherished institutions in my life. My daughter is still alive, for which I give thanks to the support, dedication and friendship of many in the health service. But it is precisely because I am such a fervent admirer that I believe it is so shameful that the NHS is allowed to limp on in its current state. For too many people, especially many of those most in need of its help, it is something of a disaster zone. The NHS is a sick institution, and cheap political point-scoring will do nothing to solve the problems. We need to find a cure.