Lies, damned lies, and statistics

If you missed the media storm that descended upon us on the 5th of June, where were you hiding? Anaesthesia published what is surely going to be one of the landmark papers not just in our beloved specialty, but across all academic medicine. Our very own John Carlisle, one of the editors of this journal, has spent years designing a statistical technique to analyse the baseline patient characteristics of randomised controlled trials, and applied this to more than 5000 published studies over a 15-year period across six of the largest anaesthetic journals, as well as the New England Journal of Medicine and the Journal of the American Medical Association. By comparing the reported to the expected distribution of variables such as age, gender, height or weight, the Carlisle Method gives the probability of these characteristics coming from a truly randomised population sample. He showed that 1.6% of published RCTs use data that is either erroneous or fabricated. The Carlisle Method demonstrated a higher frequency of non-random sampling in retracted studies, but found no differences between the anaesthetic and general medical literature. There are numerous reasons for the data to be erroneous, including simple mistakes by both authors and journals, as well as data reporting and analysis, and the publication and method used will now be subject to further scrutiny.

Loadsman and McCulloch contextualise Carlisle’s paper, both applauding and expressing reservations. Perhaps one of the more interesting takes on the issue is that fraudsters may adapt to find newer ways of overcoming the Carlisle Method. They take the example of software that is currently being used to overcome the issue of plagiarism in literature, and state that there are now accessible methods to circumvent the software. Whilst a massive undertaking is called for to clean up the potential mess that John Carlisle has picked up, the authors are not clear how and when this methodology will become the industry standard.

Of course, followers of this blog and of Anaesthesia will be familiar with the Carlisle Method, as it has been used to pick up non-random sampling in 31 trials published by Yujhi Saitoh after previous investigative successes with Yoshitaka Fujii. Anaesthesia has since decided to apply the Carlisle Method do all submitted RCTs to the journal with the aim of increasing the quality of published studies, and combat scientific misconduct.

In the June edition of the Journal, another high-impact paper was published looking at the impact of  implementing the Lifebox pulse oximeter in Malawi. These simple yet game-changing devices are not broadly available in low- and middle-income countries. Introducing them initially requires staff training before the monitors could be demonstrated to improve safety. That is precisely what Albert et al assessed, and found that understanding and knowledge retention of pulse oximetry increased. Perhaps more importantly, they also found a 36% reduction in oxygen desaturation episodes after training staff with the device.


Scott and McDougall’s accompanying editorial suggests that the data is a testament to dealing with a real-life practical problem. Looking beyond pulse oximeters, the authors remind us that 77,000 operating theatres did not have one, but the simple introduction of the WHO Surgical Safety Checklist led to tangible outcome improvements. The key, argue Scott and McDougall, is an effective education programme that aids in attainment and retention of knowledge, which is what was beautifully demonstrated in Alberts study. It is not enough to merely donate equipment, but training in its use is just as important.

Moving back from a global stage to the UK, exciting trainee research networks are beginning to produce high-quality and practice-changing data. The PAINT Study is one such paper, which looked at how often physicians document (if it isn’t documented, it didn’t happen!) assessing pain in critical care patients. In a 24-h snapshot study, they assessed documentation from all adult critical care patients across 45 centres in London and the South-East of England. They found that 21.2% of the 750 patients had no documented pain assessment by anyone, 28.6% had no documented pain assessment by a nurse, and 64.5% had no documentation of pain being assessed by a physician. This included many patients that were receiving opioid infusions, and even patients where changes in analgesic regimen were concurrently implemented. This is certainly an area that all clinicians, not just critical care physicians, must work hard to improve. I know I’ve changed my practice since this paper!

At an institutional level, El-Boghdadly et al prospectively assessed awake fibreoptic intubation (AFOI) practice at their tertiary centre. They reported that the most common indication for AFOI was limited mouth opening, and less than 1 in 100 were truly ‘awake’ intubations (I know I would rather have some sedation if I needed AFOI!). Interestingly, three-quarters of AFOIs were done by trainees, and the success rate was independent of training grade, but dependant on practice. Only 1% of AFOIs were not successful, but there were no episodes of severe complications, CICV or hypoxia. They have progressively taken up high-flow, heated, humidified nasal oxygenation (remember the big hitter of 2015: THRIVE!) during AFOI.

Murphy and Howes critically appraised this study in an interesting accompanying editorial. They question the generalisability of data from a single-institution study, particularly as the training opportunities afforded in that institution seem higher than most, alluding to a previous editorial suggesting an increasing role for videolaryngoscopy rather than AFOI. Additionally, the editorial points to the increased use of THRIVE did not reduce the rate of complications, and if anything increased the incidence of over-sedation. Are we now over-relying on THRIVE without the evidence to support it? Lots of interesting questions asked here, and it is worth considering where AFOI sits in current practice.


The June edition of the Journal was exciting and varied. We published interesting ultrasound data demonstrating delayed gastric emptying in patients with renal failure, a brilliant bench study revealing that different spinal needles have variable flow characteristics, a game-changing Cochrane review demonstrating the superiority of suxamethonium over rocuronium for RSI, and a terrifying case report of airway ignition with THRIVE. If you pick it up this month, you won’t be able to put it down.


Kariem El-Boghdadly

Trainee Fellow, Anaesthesia

Andrew Klein


A new year, another investigation

The January edition of the journal is out today, which feels strange considering Christmas hasn’t even happened yet (but is definitely coming). In an era of evidence-based medicine, scientific misconduct remains a real threat to medical research. John Carlisle, one of the editors of this journal, has developed an analytical method in order to determine whether baseline data is truly random in what is now known as the Carlisle Method. The new year brings another potential case of data fabrication. This was triggered by the submission of a suspect article to another journal, and when the data from this and other manuscripts by the same author were examined closely, there was evidence of non-random sampling. In other words, that the data was not random in its distribution in 31 trials published by Yujhi Saitoh. The majority of these papers were about neuromuscular monitoring, and they were broadly spread around the anaesthetic journals worldwide. Seven of these were published in the Canadian Journal of Anesthesia, six in the British Journal of Anaesthesia, four in the European Journal of Anaesthesiology, three in Anesthesia and Analgesia, three in Anaesthesia, three in the Journal of Anesthesia, two in Acta Anaesthesiologica Scandinavica, two in the Journal of Clinical Anaesthesia, and one in the Fukushima Journal of Medical Sciences.

This is clearly very concerning, and we await the findings from the investigation that is ongoing by the Japanese Society of Anaesthesia. However, we can’t only look backwards at studies that are already published – we also need to look very closely at what is submitted to journals in all different specialties in the future. To that end, at Anaesthesia, we have decided to screen all randomised controlled trials submitted to the journal from 2016 using the Carlisle Method. We believe we are the first journal to do this. Any that fall foul due to suspicious data that are not consistent with random sampling will be rejected and the authors informed of the reason for rejection. We hope to persuade all the other anaesthetic journals to follow suit soon, and will look to involve other specialties and organisations over the coming years.

We are seeing more and more ‘disposable’ single-use devices in our practice. These include laryngoscopes, bougies, and now even fibreoptic scopes. While there are clear advantages in terms of infection control, there remains concern about comparable efficacy, design, cost and the ‘green’ effect of throwing away so much plastic and other materials. With this in mind, it is very tempting to re-use these single-use devices in the same patient repeatedly, both on the same day and perhaps even on subsequent days, especially in the case of fibreoptic scopes. Surely if they are going back into the same patient then that can’t do any harm? Wrong – a study published in this month’s edition of the journal showed that 16 out of 20 bronchoscopes cleaned then kept after use were contaminated after 48 hours. There is a very clear clinical lesson here – single-use means exactly that, and you can’t re-use them later even in the same patient. This will have significant implications for many hospitals I suspect.

This is a welcome update on consent for anaesthesia in this month’s journal, and this a ‘must read’ for every anaesthetist. The twelve key tenets include intuitive as well extremely thoughtful recommendations. Full consent should be obtained as early in the patient pathway as possible (not in the anaesthetic room), and the information provided should be tailored to each patient, with adequate time allowed for patient questions. Documentation should be made of the consent obtained although specific consent forms are not required. The fluid nature of consent means it is an ongoing process and should be confirmed at each interventional stage. If a patient lacks capacity, the reasons should be documented, efforts should be made to reverse or reduce temporary incapacity, and if this is unachievable we should always act in the patient’s best interest. Seeking a lasting power of attorney (LPA), valid advanced decisions, a validly appointed health and welfare LPA or a court-appointed deputy are legally binding. A knowledge of the existing frameworks regarding consent in patients aged 17 or younger is recommended. Finally, when training in practical procedures is undertaken, maximising benefit whilst minimising risk to the patient is important and alternative means of training, such as virtual models or manikins should be considered. These guidelines are clear and thorough and will be the mainstay of clinical practice for years to come.

Finally for now, we have published a comparison of the adjustable pressure-limiting valves in two well-known anaesthetic machines. The manufacturer of the APL that was shown to be ‘unusual’ in its performance has also commented on the study, and an accompanying editorial has put this into perspective. The clinical lesson here is know the machine you are using and read the instructions for use. Admittedly, so many of us don’t, and if you read this article you will see how important it is to know the difference between different designs of APLs and how they function in practice, especially for paediatric use. My final comment is, why are different APLs produced and why aren’t clinicians telling manufacturers what they want and being involved in the design of new equipment? It seems nonsensical to me that there should be such a difference, with such important implications, in APLs on different common anaesthetic machines. Should we accept this from a safety perspective?

Andrew Klein


Understanding Uncertainty

How good are you at understanding chance, risk, uncertainty and probability? The UK referendum on whether to leave or remain in the European Union has brought statistics, risk and uncertainty back into our everyday language. We have (re) learnt that statistics without context can be misleading, tolerance of an acceptable risk is opinion–based, and that both financial markets and individuals struggle to deal with uncertainty. This is emphasised in an excellent article, which makes the point that 50% of anaesthetists are actually worse than average at understanding risk. Anaesthesia has made a point of providing easy-to-understand, concise, educational articles in the last year, our statistically speaking series, and this will continue into next year. We plan to publish a series called ‘methodological madness’, in which we invite readers to write in and ask our statistical guru (Dr Choi from Hong Kong email: about what authors have got up to when designing their methods for studies. The main message is, we all need to understand more about statistics, probabilities and risks.

Airway management is the prime professional skill of the anaesthetist; research into this topic is widespread, and Anaesthesia receives many such submissions. In a study from Switzerland, Kleine-Brueggeney and colleagues compared the performance of the Bonfils™ and SensaScope™ rigid fibreoptic scopes in 200 patients with a simulated difficult airway. They note in their introduction that rigid scopes such as the two studied are relatively underused in anaesthesia despite being favoured in otolaryngology and respiratory medicine.  The authors simulated a difficult airway by applying a cervical collar to each patient such that mouth opening was limited to a mean of 23 mm. The patients were randomly allocated into two groups; the primary outcome of the study was overall success of intubation. In this, the overall success rates were high for both devices (88% for the Bonfils and 89% for the SensaScope (p = 0.83), although median intubation times were a little shorter with the SensaScope (34 vs. 45  seconds).

In an accompanying editorial, Ward and Irwin explore the ethical implications of airway research where the normal airways of routine patients are rendered ‘difficult’ for the purpose of evaluating the performance of new devices (or those using them). Notwithstanding the fact that there are many reasons for an airway being ‘difficult’, and the difficulty created by the methods commonly used in the research context may not reproduce all of them, there are also important questions to consider about the nature of risk and benefit in such studies.

As Ward and Irwin note, patients taking part in such studies do not themselves benefit from such participation; instead, the data may contribute to the common good of future patients in general. In this context, the possible harms implied by the intervention are thrown into sharper focus. The members of research ethics committees may lack the specialist knowledge of anaesthesia devices to allow them to make a fully informed judgement about the balance of benefits and harms. Here, the anaesthetist’s first duty is the responsibility of a physician to a patient, not a researcher to data. An innovative Consensus on Airway Research Ethics is proposed, and I have also added a note advising anyone conducting airway device evaluation studies that manuscripts will need to comply with the recommendations in the Consensus if they wish to be considered for publication in Anaesthesia.

Also on the theme of airway management, this report from the Netherlands details the development of an audit tool to identify prospectively all peri-operative adverse events during airway management over an 8 week period. Data were collected daily by  questionnaires from, and interviews with, anaesthesia trainees and anaesthetic department staff members. A total of 168 airway-related events were reported out of 2803 patients undergoing general anaesthesia. The incidence of severe airway management-related events was 24/2803 (0.86%). There were 12 (0.42%) unanticipated ICU admissions and two patients (0.07%) required a surgical airway. There was one (0.04%) death, one ‘cannot intubate cannot oxygenate’ (0.04%), one pulmonary aspiration of gastric contents (0.04%) and eight (0.29%) severe desaturations (defined as an oxygen saturation less than 50%). Whilst this survey is restricted to one hospital, the authors suggest that the methodology they used could easily be followed by others within their own departments of anaesthesia.

Finally, this being December and Christmas being just around the corner, we have published our first-ever Christmas special in the journal, CRAC-ON, as in why don’t you just CRAC ON and give the anaesthetic! CRAC ON stands for complete relinquishing of anaesthetic conscientiousness, optimisation and nuance. This special article is meant to be light-hearted and satirical, and I really enjoyed reading it. It is included as an extra article, and the rest of the journal contains as many serious articles as normal. I hope you enjoy it too, and would be interested in receiving your feedback. CRAC ON and have a good Christmas!

Andrew Klein


Just breathe!

Breathing seems to be a major theme in the literature (and at meetings) at the moment, and there are a number of articles in this month’s edition of the journal that are relevant.

Apnoeic oxygenation and nasal oxygen administration are two concepts that are hardly new in anaesthesia, but are rapidly taking centre-stage for management in a wide variety of situations. Dr’s Patel and Nouraei coined the term THRIVE – Transnasal Humidified Rapid-Insufflation Ventilatory Exchange – and described a case series in Anaesthesia in 2015. This paper has just received the award for ‘Best paper in Anaesthesia of 2015’ at the AAGBI Annual Scientific Meeting in Birmingham.

The situation of rapid sequence induction of general anaesthesia is one in which we are poor at predicting airway management difficulty (see e.g. Norskov et al. Anaesthesia 2015; 70: 272 – number 2 ranked of the 2015 Anaesthesia articles), yet we produce an unstable situation of complete muscle paralysis before the definitive tracheal intubation procedure. In this edition of the journal, Pillai et al. using the Nottingham Physiology Simulator have  shown that, under ideal conditions, oxygen delivery during apnoea might increase the time to desaturation of a pregnant subject from 4.5 min to 58 min. This is incredible if true, and will be of great interest to all obstetric anaesthetists, but needs to be further investigated in pregnant patients – I believe such trials are ongoing.

There are likely to be more papers on this subject in Anaesthesia in the near future – watch this space for progress that may dramatically change how we do things in one of the high-risk areas of our practice.

This edition of the journal also includes a paper describing the current state of airway training in UK anaesthetic departments. The Fourth National Audit Project (NAP4) recommended routine and regular airway training for trainee and trained anaesthetists. However, in this survey from 206 hospitals (62%) covering all regions of the UK, 16% of hospitals did not provide airway workshops for staff at all, and 51% only for trainees. Of those providing workshops, more than half were being run less than annually. The authors concluded that workshop-based airway training is variable in provision, frequency and content, and is often not prioritised by departments or individual trainers. I agree that the provision of appropriate training identified in NAP4 is sadly lacking in many hospitals, and the reasons for this are many, but surely include resources as well as motivation. Getting Consultants out of the operating theatre into an hour or two-long airway workshop is what is needed, and regularly, but this isn’t easy, especially when getting any time out of theatre (or ICU) is getting harder and harder in the current climate. Should this be made part of mandatory training? And are workshops really the answer?

Finally, what about trainees challenging consultants? There is a perception that trainees should challenge their ‘seniors’ more frequently, especially when they are obviously wrong. This is borne out by this simulation study which explores the concept of ‘barriers to challenging seniors’ for anaesthetic trainees. The authors concluded that more senior trainees challenged their consultant supervisor quicker, allowed fewer intubation attempts, established quicker adequate rescue oxygenation and ventilation and less simulated patient desaturation was observed. This is not really surprising as experience and maturity should improve performance, especially in this sort of scenario, but the authors make some interesting observations about improving training to give trainees the confidence to challenge more effectively and with less hesitation. Take a deep breath and go for it!


Andrew Klein


Occam’s razor

Occam’s razor (or the law of parsimony) is a problem-solving principle attributed to William of Ockham (c. 1287–1347), who was an English Franciscan friar, scholastic philosopher and theologian. The principle states: ‘other things being equal, simpler explanations are generally better than more complex ones’. This month’s blog looks at simple interventions and principles, and whether Occam’s razor hold true in our practice.

Paracetamol – great drug, right? I swear by it, and dispense it liberally, both at home and in the operating theatre and the intensive care unit. A simple study with a very simple hypothesis is published in October’s edition of the journal, comparing oral (enteral) with intravenous (parenteral) paracetamol in ICU patients. Patients who received the intravenous formulation were much more likely to suffer hypotension and require vasoconstrictors to ameliorate this. Wow – so paracetamol is not as harmless as we all think? If one off doses can cause hypotension like this should we still be giving it at all in our ICU patients, or should we only be giving it enterally (presumably via the NG tube – hypotension still occurred but less commonly)? Maybe this simple, effective drug is not the panacea? A very interesting study I am sure you will agree.

What about teaching tracheal intubation to novices? We have traditionally taught the use of the Macintosh laryngoscope first, but will they learn it and remember it better if they are taught with a videolaryngoscope? Actually, perhaps not – this excellent study studied a group of medical students and looked at skill retention. This study showed that the students learnt how to use the Macintosh, A.P. Advance™, C-MAC® and Airtraq® laryngoscopes equally well at first, but one month later, they seemed to have retained the skill-set required for laryngoscopy significantly better with the Macintosh and Airtraq laryngoscopes. In this instance, simpler seems to be easier to pick up again and get to grips with more quickly, a salutary lesson.

This month, we also publish an interesting systematic review of the effect of propofol compared with inhalational anaesthesia on postoperative outcomes including pain. This well-conducted rigorous review found that patients who received total intravenous anaesthesia with propofol did indeed have reduced pain scores 24 hours after surgery, although the effect size was quite small. But, it also confirmed that postoperative nausea and vomiting was markedly less common. Is this enough to make you switch your technique to TIVA? Maybe not, after all these are not really important outcomes like mortality, but will we ever recruit enough patients to see a difference in mortality if there was one – I doubt it. So what are we waiting for, or do we just not believe there is actually a difference? Perhaps it is simpler to believe the opposite, that inhalational and propofol are much of a muchness for maintenance of anaesthesia and we are not convinced either way yet.

Finally, our statistics article explains why odds and risks (and other numbers) often confuse things, both for researchers and for readers (consumers). Why do we like to use complicated statistics to describe things, when simpler explanations are often possible if not preferable? If you, like me, don’t know the difference between the odds of something and the risk of the same thing, then read it and learn – I did, and I am off to put a bet on the 2-30 at Newmarket…


Andrew Klein


The Olympics for anaesthetists

Well, that’s it for another four years. Months and years of preparations, and in the end it came down to five days of non-stop action and one night of celebration, then everyone goes home. This is the week that was the World Congress of Anaesthesiologists in Hong Kong, and what a good one it has been. I am going to give you a bit of a flavour of the event and how this journal fared at the pinnacle of our professional calendar of conferences.

The whole event was organised by Mike Irwin, Professor at Hong Kong University and one of the editors of Anaesthesia. Siu Wai Choi, the statistical advisor to this journal was also ever-present, coaching us to rise above any mathematical missed passes. However, John Carlisle, another editor of the journal, was the real star turn when he shared the podium with Steve Shafer (ex Editor-in-Chief of Anesthesia and Analgesia @stevenlshafer) and Nathan Pace (from the University of Utah and Senior Statistical Advisor at Cochrane). John presented his initial analysis of Fujii that led to the retraction of 183 publications and his undisputed Number 1 position on the Retraction Watch Hall of Shame (@RetractionWatch). He then presented the rationale for Monte Carlo simulation and his updated analysis – the Carlisle Method – which he announced he has now applied to all randomised controlled trials published over the last 15 years in this and seven other journals. Interestingly, another author has come to light after the application of the Carlisle Method following the submission of a suspicious manuscript to another journal, and this further analysis previewed at the World Congress will be published shortly in Anaesthesia.


One of my own personal highlights was the sight of the delegates queuing patiently to get hold of souvenir USB sticks containing the China Special Edition and World Special Edition of Anaesthesia produced especially for the Congress, and you can see these special editions yourself on our website by clicking on the links above. I also particularly enjoyed crossing the city to Hong Kong University to present a workshop to upcoming biomedical researchers on what and how to publish. The researchers packed the room and posed many questions about publication, research and the world of journal intrigue and peer review.



There was great interest at the Congress in several recently published articles. A Korean group presented their findings about complications during subclavian central venous catheterisation, and their conclusion that a wire-through-needle technique is safer than a catheter-over-needle technique should finally put to bed the longstanding debate about the two techniques. There were several fiery debates and discussions about routine or otherwise use of dexamethasone, with the presentation of data about the inescapable rise in glucose concentration associated with its use in diabetics and non-diabetics. Finally, novel regional blocks were much in evidence, and particularly the serratus anterior plane block, which was dissected in detail.

To finish up, a bit about the final night of celebration. A crowd of us were shown the sights of the harbor from the top deck of a boat, followed by a seafood dinner and even one or two local beverages. I understand that a number then went on to do a crawl around the top 10 rooftop bars of the city, but myself, I was tucked up in bed ready for the Closing Ceremony. I am sure everyone is looking forward to the next Olympics in Prague in 2020 – I certainly am.


Andrew Klein


Stop, Look, Listen, Think

The theme of this month’s blog is Safety (the capital S is deliberate). We teach road safety to our children by telling them to “Stop, look, listen and think” before crossing the road, and maybe as medical providers we need to try this too?

This month, researchers from Manchester publish an analysis of patient safety incidents reported to the National Patients Safety Agency (NPSA), now the National Reporting and Learning Service (NRLS), between 2004 and 2014 from critical care units in England and Wales. Of 1743 incidents, 389 (22%) may have contributed to the patient’s death, and 1555 (89%) may have been avoidable. Over the 10 years, the number of reported incidents per year went up, as did the number of patients cared for in ICU (but not enough to explain the increase in safety incidents). Why are patient safety incidents on the increase, despite our best efforts? Are we just reporting them more frequently? Or are they actually more common, despite the introduction of numerous guidelines, safety procedures and other central and local initiatives? Can we as medical practitioners and the ICU team as a whole do more? The study shows that there was a decrease in the number of incidents related to infection, but an increase in the number of medication incidents. There is certainly much to reflect on in this excellent analysis.

What about cricoid pressure? We were/are all taught to use it for rapid sequence induction, but, although a number of studies have cast doubt on its efficacy and safety, our practice hasn’t really changed. This month, a new study using ultrasound in awake volunteers shows that standard cricoid pressure does not actually narrow the oesophagus at all, let alone occlude it. This would be an interesting study to repeat in anaesthetised patients, but if it held true, does cricoid pressure do what we think it does? The authors also studied a newly described technique which they name paralaryngeal pressure. This was found to be much more effective at occluding the oesophagus, and certainly merits further study. the future of cricoid pressure is eloquently debated in the accompanying editorial, including its safety and when to release the pressure to prevent patient harm, and I urge you to read both the article and the editorial and join in the debate in our correspondence pages or on Twitter.

The final article related to safety describes an investigation modelling oxygen supplementation during tracheal intubation in pregnant women. Because pregnant women desaturate much more quickly during apnoea, rapid sequence induction can be more fraught and dangerous. Recent studies (THRIVE) have shown that high-flow nasal oxygen during intubation can prolong the time to desaturation during apnoea in the non-pregnant population. This most recent study demonstrated that increasing FiO2 at the open glottis increased the time to desaturation, extending the time taken for SaO2 to reach 40% from 4.5 min to 58 min in the average parturient model (not in labour). The greatest increases in time to desaturation were seen at FiO2 1.0, which could be delivered by high-flow nasal cannulae under ideal conditions. Obviously, clinical studies are needed in pregnant women, but my conclusion from this modelling study is – should we now be administering oxygen routinely via nasal specs during rapid sequence intubation, and certainly in pregnant women? Weighing up the risk-benefit model it would certainly seem to, and this may be a very significant change which will increase safety in this and other patient populations. I look forward to more research in this rapidly evolving safety field.

Finally, I am about to decamp to China (for the Chinese Society of Anesthesiology meeting) and Hong Kong (for the World Congress). I will be blogging from there with some special editions of the journal to coincide with these meetings and more updates on breaking research.


Andrew Klein