Differential diagnosis project update and presentation

Had a chance yesterday to share the last developments in the quest to improve differential diagnosis in physical therapy through the use of causal models and Bayesian networks.

The presentation is online here (I plan to add voice over and break it up into smaller online webinar based sessions soon):

http://www.knowledgebasedpractice.com/differential_diagnosis/

All the files for it – including the example networks built with samiam and full conditional probability tables (estimates, but at least these assumptions are explicit and shared) are on GitHub:

https://github.com/scollinspt/differential_diagnosis

The most amazing part of the presentation to me was that with 41 therapists in the room; no one challenged my repeated statements that all of their decision making is founded on probabilistic causal inferences within complex causal networks. In practice they simply do not consider the underlying assumptions and probabilities, but when they make a decision they are making a statement about what they believe, and therefore about a probability. There seemed to be general ascent on this claim.

Thank you to all of you that attended – I look forward to more work and interaction with collaborators as the project continues to unfold; and to offering additional continuing education and even training opportunities for anyone interested.

The general premises of the project are:

– Clinical practice situations are complex systems (see this post)
– Complex systems include high dimensional, multi – variable causal networks.
– Reasoning with such systems is challenging and often includes several hidden assumptions.
– Both for teaching and for research aimed to improve the decision making from such reasoning, it is valuable to make as many of the previously hidden assumptions explicit.
– Existing methods to attempt to improve differential diagnosis (use of diagnostic accuracy such as sensitivity, specificity, likelihood ratios) don’t go far enough. They are far too simple compared to actual complexity of actual clinical practice situations.
– Bayesian networks offer – for clinical educational and clinical research purposes – a bridge between the highly complex system of actual clinical practice and existing methods that attempt to improve the process.
– Clinical education and clinical research support clinical practice and are necessary components to the system attempting to improve practice, which is something that providers, clients and payors are all seeking.

Screen shot of a sample samiam model for back pain with several clinical signs observed and instantiated leading to adjusted posterior probabilities:

Screen Shot 2017-04-02 at 7.33.00 AM.png

The structure of structures: anatomical networks in PT anatomy education

A few months ago I started thinking about what anatomy, as foundational knowledge for physical therapy practice, would look like and how it would be taught in a program considering “knowledge based practice: cause, models and inference” as a clinical epistemology (here). Now I am taking that a step further, in a KBP – concept based curriculum – with core movement concepts including causation, adaptation and systems.

It is easy to see how “cause” is important in the study of anatomy. Cause is, after all, how we use anatomical knowledge in practice. Another way to say this (flipped), is that anatomical knowledge is one set of knowledge from which we reason causally. When using abduction to generate a list of possible “causes” of a set of signs and/or symptoms we generally consider the anatomy: “moving the leg like that could be due to a tight X, or a weak Y, or a stiff Z, …, etc.”

It is also easy to see how adaptation is important in the study of anatomy. After all, all of our current adult anatomical musculoskeletal forms are, within a set of boundary constraints, the result of the sum of adaptations of our life long journey to where we are right now. So how we live is predictive of our future anatomy (causal induction and causal deduction), and our current anatomy testifies to the journey we have taken (causal abduction).

Now how about a system? It is certainly easy to talk about the “systems” of our body anatomically. After all, we learn our anatomy in A&P I and II as a set of systems. This is how A&P books are divided. It is a fine way to divide them. Then we get into gross anatomy and we divide up the body regionally – again a fine way to divide up the body to learn it. But what about learning about the system of the system? For example, the system of the skeletal system.Or the system of the muscular system. Or the system of the musculoskeletal system. There is a modularity balanced with integration in all of these (and other) systems.

Well, this is what we can learn from a rather new application of an emerging analytic approach. The application of network analysis (studying systems by their network structure using graph theory (logical, mathematical), and by the way, for those of you that have been reading along with this blog – a “graphical causal model” is simply a network, a particular type of network, a DAG (directed acyclic graph)). This new application to anatomy has been termed: Anatomical Network Analysis (AnNA).

AnNA is exactly what these authors (Diogo R, Esteve-Altava B, Smith C, Boughner JC, Rasskin-Gutman D) have been doing for the past few years:

http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0140030

AnNA is original, insightful and very useful.  Making this particular paper even more useful, the authors have subscribed fully to reproducible research, so their supplements, data, details regarding the analysis including R code – and a vast set of networks – is available on FigShare (here).

The paper citation is:

Article Source: Anatomical Network Comparison of Human Upper and Lower, Newborn and Adult, and Normal and Abnormal Limbs, with Notes on Development, Pathology and Limb Serial Homology vs. Homoplasy
Diogo R, Esteve-Altava B, Smith C, Boughner JC, Rasskin-Gutman D (2015) Anatomical Network Comparison of Human Upper and Lower, Newborn and Adult, and Normal and Abnormal Limbs, with Notes on Development, Pathology and Limb Serial Homology vs. Homoplasy. PLOS ONE 10(10): e0140030. doi: 10.1371/journal.pone.0140030

And I hope it is ok – but here are the first two figures in an attempt to encourage readers to get the paper:

Figure 1:

journal.pone.0140030.g001.PNG
Figure 2:

journal.pone.0140030.g002.PNG

Needless to say, I strongly believe that Anatomical Network Analysis (AnNA) will provide many benefits to the education of anatomy (in addition to the benefits discussed by the authors in this paper (and their other papers)), and potentially to the physical therapy profession as a whole as we work together on the human movement system. I am looking forward to fully exploring the possibilities and implementing an anatomy course and research that utilizes the concepts that emerge (pun intended), when we consider the network structure of the structures we use in practice. What it tells about about anatomical causal associations, what it tells us about anatomical adaption, and what it tells about about the system of anatomical systems with implications for movement.

As a final word – one of the authors on this paper also has a new anatomy book out – from what I have read so far, it is highly recommended (and will be required for PSU – DPT students).

51FtN42PRkL._SX348_BO1,204,203,200_.jpgDIOGO, R., D. NODEN, C. M. SMITH, J. A. MOLNAR, J. BOUGHNER, C. BARROCAS & J. BRUNO (2016). Learning and understanding human anatomy and pathology: an evolutionary and developmental guide for medical students. Taylor & Francis (Oxford, UK). 348 pages.

 

 

 

 

 

 

 

It’s great to be back blogging – it has been a long 18 months getting the PSU-DPT program into the “candidacy” pre-accreditation phase. But now as we look to accept our first class an implement this new program I look forward to blogging more as things unfold.

Randomized controlled trials – a KBP perspective

Re blogging this today in light of a discussion regarding this paper:

Morris PE, Berry MJ, Files D, et al. Standardized Rehabilitation and Hospital Length of Stay Among Patients With Acute Respiratory Failure: A Randomized Clinical Trial. JAMA. 2016;315(24):2694-2702. doi:10.1001/jama.2016.7201.

see: http://jama.jamanetwork.com/article.aspx?articleid=2530536&linkId=26010051

Cause, Models & Inference in Physical Therapy

In evidence based practice randomized controlled trials (RCTs) have a very high standing. In fact, by the GRADE approach to weighing evidence for a clinical practice guideline (CPG) a single, large, well conducted RCT can result in an evidence rating as high as a systematic review of RCTs. This post is not an attempt to argue against the use of RCTs to develop knowledge for practice. The purpose is to simply share some thoughts about the limitations of RCTs and what KBP suggests for RCT method0logy planning.

RCTs are highly regarded because when they demonstrate an effect with a large sample then the cause tested is the most likely explanation for the observed effect (low risk of bias in well designed, large sample RCTs). There will be variability in the effect and the amount of variability is important to consider for the clinician as the patients you treat are not the…

View original post 919 more words

As a professional no one tells you what is important to know for the exam….

These past few weeks I have been thinking a lot about the question – “What is important to know for the exam?” Or, “What should I focus on for the exam?” Usually followed up with propositional statements such as: “There is so much information.” Or “There is so much reading to do.”

Of course, “so much reading” is highly relative. I was recently told that a 3 page document I had provided was “too long” to read completely, but was rather skimmed for the gist of it.I was also asked whether I could provide a summary statement of a one page outline I had developed. If you believe 3 pages is too long , or that a one page outline needs a summary statement, then I am sorry, we may never agree on what too much reading is for a developing professional that would like to practice physical therapy someday.

One of the reasons I struggle with the question about what is important to know for the exam is that I am sympathetic to the students’ plight and I want to help them. So I have now come up with my response.

“What do you think is important to know for the exam?” Followed with questions about whether the student understands the problem they are attempting to solve.

It seems a simple fact but underlying my classes is the fact that I believe it is important that students learn to identify what is important. That is part of what they are learning in their education. They need to learn to assess a problem in such a way that they can identify the relevant from the irrelevant. This means that being able to figure out what is important to know for the exam is part of what you are being evaluated on when given an exam.

Of course this is not exclusive to DPT education, nor to graduate education, or even a college education. In elementary school when you start to work on word problems in mathematics you are essentially learning how to figure out what is important information to solve the problem at hand.

My role as an instructor is to help students develop the ability to figure out what is important. That does not mean (and should not mean) that I outright tell them what is on the exam. It is on me to generate realistic problems for them to work through, experiences to wrestle with, with materials and guidance that they need to read through and develop enough of an understanding  that they develop the ability to identify what is important. And once it is learned that something is important – learn it, make sure it is understood; that is, know it.

Better stop now – or this post may become too much reading 😉

Back to writing, no more wrangling

My previous post announced a move to a new platform. Since that announcement I have wrangled with the new system enough to realize that it detracts too much from the purpose of the blog – writing.

Therefore, with this post I welcome back my WordPress platform and the ease with which it allows me to simply write and post.

New site for Knowledge Based Practice

Just a quick announcement that the WordPress hosted “Cause, models and inference: developing a knowledge based practice” blog is now in its “legacy” stage. Over the past few weeks I have migrated the blog to a new URL, hosted by GitHub pages and developed it using Jekyll. More information is provided at the new site. All of the old posts are there – some as posts, others collected into collections for easier access and reading.

www.knowledgebasedpractice.com

Thanks for reading! I am hoping the new platform enables greater reach and development.

 

The challenge of proving a negative

A quick comment on the challenge of proving a negative as related to the effectiveness of interventions (if it is even possible is debatable). But lets look past proving, and shoot for knowledge as justifiable true belief (still difficult).

These thoughts have emerged after months of reading through and incorporating the “Minimum Required Skills” for a PT graduate (from 2004, revised in 2009) and wondering when those with justifiable evidence that they do not work should be removed?

I understand completely the need to keep those without justifiable evidence that they work if there is no justifiable evidence that they do not work. You have likely heard: a lack of evidence for effectiveness is not the same as evidence of a lack of effectiveness.

Then I realized – what is “justifiable evidence” or “knowledge” (justified true belief) that an intervention does not work. This is trying to prove a negative. Very difficult indeed.

Are we destined to continue to teach such skills? If so, what are the implications for the profession? Health care? Society?

Do we need something beyond the hierarchy of evidence for adjudicating such decisions?

As a quick example for those wondering why it is difficult to prove a negative. If I said that a “black swan” exists, I can put that hypothesis forward and continue searching. All I need to find is a single example that a black swan exists and I have proven the affirmative – a black swan exists. But how much searching do I need to do to prove that a black swan does not exist? Are we ever satisfied with the search for evidence, or might there be one around the corner that we have not looked? Or perhaps – like standing cows in the classic Far Side cartoon  – they disguise themselves when observed.

As a specific example, let’s take Continuous Passive Motion. There is now a large review of years of evidence that support the claim that it’s use is not justified (see here). But, are we comfortable yet in throwing it aside and not recommending it’s use – or do we find reasons why a particular person might benefit from it beyond the mean of several large samples and relying on the central limit theorem for a homogenized estimate of effect?

In other words – does this review on CPM warrant justified true belief (knowledge) of the negative – that CPM does not work? Or does the “There is weak evidence that continuous passive motion reduces the subsequent need for manipulation under anaesthesia.” continue to raise questions about better approaches to patient examination and evaluation for identifying those that will benefit from it’s use?

When we break this down to  the use of logical (material) implication it may seem possible to prove the negative “If – Then” implication. After all, with material implication, the implication is “False” whenever the antecedent (“If” condition) is true and the consequent (“Then” condition) is false. And for material implication that is the only time the If – Then statement is proven false (true antecedent and false consequent). However, things get muddy when attempting to justify a universal based on this approach. It is easy to justify a particular situation. If the cause occurred and the effect did not occur, you are justified to say that the cause did not cause the effect (the effect did not even occur) in a particular situation. But the challenge is whether this is a universal truth – and here we go from logical inference to statistical inference.

When considering this from the perspective of statistical inference, and attempting to justify a universal causal association we must consider the complexity of the system under study. In reality, none of the connections between the cause and effect we consider this simple:

dagitty-model.png

And they likely are not even this simple – but with this structure there are questions raised about what we need to know in order to understand the causal structure enough to know when the Cause will result in the Effect, such as whether the other variables in the path are possible (X1, X2, Y for example).

dagitty-model (1).png

With this causal structure the variability in the effect being measured for the use of CPM (Cause) is impacted by  X1 and X2 and Y,and a minimum their existence, perhaps even their nature or own causal structure. In which case we do not want to close the door on Cause -> Effect until we understand that structure and have studied the system effectively.

Ok, that is enough laps around the track for this morning. I realize I have come full circle -and still have not decided, after months of pontificating, whether anything on this list can be removed due to justifiable evidence of the negative.