Prediction without causation is wishful thinking

After months of slumber the KBP blog has been awoken. The slumber was not a true slumber, the author has been otherwise engaged in a variety of activities keeping him (me) from writing. A series of events have been pushing me back toward posting again.

  1. APTA Education Leadership Conference presentation with my colleague Kelly Legacy called “Making Causal Inferences Explicit” – at some point soon I will voice over that talk and post (or just post the slides)
  2. An advance release of a paper about causal mechanisms was released to be published in PTJ – that will certainly be worthy of a post
  3. Engagement with the wonderful folks as APHPT and was just recently interviewed by their founder / president Mike Eisenhart. It was a great time and made me realize that some of this stuff I have been blogging about could be useful, so I may not want to throw in the towel just yet.

But, what has inspired me just this moment, is a PT In Motion article about an advance release of a paper coming out in the journal “Arthritis Care and Research” called: “Minimum performance on clinical tests of physical function to predict walking 6000 steps/day in knee osteoarthritis: An observational study“.

The take home message here will be that prediction without causation is wishful thinking. It is possible. Yes, it is true, data the establishes an association can then be used to predict. However, without causation the direction of the events are not known and our attempt to use prediction for intervention is left as wishful thinking. As an extreme example, if we demonstrated an association between a herniated disk and radicular pain then do we believe that removing the radicular pain through cryotherapy or medications or hypnosis makes the herniated disk go away? No. We believe that there is a causal association between the herniated disk and radicular pain, and while all of those mechanisms to intervene on radicular pain may have their place (I am neither recommending nor advocating), we are more comfortable that if the herniated disk was intervened on and went away that the radicular pain would go away (I do realize it is more complex than this, that there are other causes of radicular pain, etc).

To get back to the study at hand.

The purpose of this study was to identify minimum performance thresholds on clinical tests of physical function predictive to walk ≥6000 steps/day.

Methods: Using data from the Osteoarthritis Initiative, we quantified daily walking as average steps/day from an accelerometer (Actigraph GTM1) worn for >10 hours/day over one week. Physical function was quantified using three performance-based clinical tests: five times sit to stand test, walking speed (tested over 20 meters) and 400-meter walk test. To identify minimum performance thresholds for daily walking, we calculated physical function values corresponding to high specificity (80 to 95%) to predict walking ≥6000 steps/day.

Results: Among 1925 participants (age [mean±sd] 65.1±9.1 years, BMI 28.4±4.8 kg/m2, 55% female) with valid accelerometer data, 54.9% walked ≥6000 steps/day. High specificity thresholds of physical function for walking ≥6000 steps/day ranged from 11.4 to 14.0 sec on the five times sit to stand test, 1.13 to 1.26 meters/sec for walking speed, or 315 to 349 sec on the 400-meter walk test.

Conclusion: Not meeting these minimum performance thresholds on clinical tests of physical function may indicate inadequate physical ability to walk ≥6000 steps/day for people with knee OA. Rehabilitation may be indicated to address underlying impairments limiting physical function.

Emphasis has been added. It is an observational study. Data collected was cross-sectional. In the discussion about limitations the authors casually admit:

Third, the cross-sectional design allowed us to identify a relationship between physical function and physical activity, but not to draw conclusions about causation.

This is the third of four limitations which are mentioned. Of course, like a lot of studies, the limitations are mentioned but the implications of the limitations are not discussed.

In this study data was collected on performance (3 tests that are reasonable to expect to be associated with walking) and walking (as number of steps per day). Despite the data being collected at the same time (cross-sectionally) and the admitted limitations in drawing causal conclusions the study still moves ahead with the following causal assumption:

dagitty-model.png

This seems perfectly reasonable. But so isn’t this:

dagitty-model (1).png

The problem here is that this study did not explicitly test either of these assumptions, yet its conclusion is completely committed to the first assumption.

Not meeting these minimum performance thresholds on clinical tests of physical function may indicate inadequate physical ability to walk ≥6000 steps/day for people with knee OA. Rehabilitation may be indicated to address underlying impairments limiting physical function.

It is equally valid to have concluded:

Not walking ≥6000 steps/day may indicate these minimum performance thresholds on clinical tests of physical function for people with knee OA. Walking ≥6000 steps/day may be indicated to address minimum performance thresholds on clinical tests of physical function.

In fact, there is another hidden assumption. They have assumed that if someone does not walk ≥6000 steps/day that they cannot walk ≥6000 steps/day. This is a major assumption that has a huge impact on the conclusions drawn from this study.

In the end, this is a valuable contribution. It contributes to an understanding that can be built on with further work. But it does not allow prediction of the capability of walking ≥6000 steps/day. It maybe allows prediction of actually walking ≥6000 steps/day; BUT if we start with walking ≥6000 steps/day, it allows prediction of physical test performance. But these examples of prediction are based on associations. And predictions based on associations are less useful for clinical interventions.

But please keep in mind, that predication without causation is simply wishful thinking.

 

Advertisements

Differential diagnosis project update and presentation

Had a chance yesterday to share the last developments in the quest to improve differential diagnosis in physical therapy through the use of causal models and Bayesian networks.

The presentation is online here (I plan to add voice over and break it up into smaller online webinar based sessions soon):

http://www.knowledgebasedpractice.com/differential_diagnosis/

All the files for it – including the example networks built with samiam and full conditional probability tables (estimates, but at least these assumptions are explicit and shared) are on GitHub:

https://github.com/scollinspt/differential_diagnosis

The most amazing part of the presentation to me was that with 41 therapists in the room; no one challenged my repeated statements that all of their decision making is founded on probabilistic causal inferences within complex causal networks. In practice they simply do not consider the underlying assumptions and probabilities, but when they make a decision they are making a statement about what they believe, and therefore about a probability. There seemed to be general ascent on this claim.

Thank you to all of you that attended – I look forward to more work and interaction with collaborators as the project continues to unfold; and to offering additional continuing education and even training opportunities for anyone interested.

The general premises of the project are:

– Clinical practice situations are complex systems (see this post)
– Complex systems include high dimensional, multi – variable causal networks.
– Reasoning with such systems is challenging and often includes several hidden assumptions.
– Both for teaching and for research aimed to improve the decision making from such reasoning, it is valuable to make as many of the previously hidden assumptions explicit.
– Existing methods to attempt to improve differential diagnosis (use of diagnostic accuracy such as sensitivity, specificity, likelihood ratios) don’t go far enough. They are far too simple compared to actual complexity of actual clinical practice situations.
– Bayesian networks offer – for clinical educational and clinical research purposes – a bridge between the highly complex system of actual clinical practice and existing methods that attempt to improve the process.
– Clinical education and clinical research support clinical practice and are necessary components to the system attempting to improve practice, which is something that providers, clients and payors are all seeking.

Screen shot of a sample samiam model for back pain with several clinical signs observed and instantiated leading to adjusted posterior probabilities:

Screen Shot 2017-04-02 at 7.33.00 AM.png

The structure of structures: anatomical networks in PT anatomy education

A few months ago I started thinking about what anatomy, as foundational knowledge for physical therapy practice, would look like and how it would be taught in a program considering “knowledge based practice: cause, models and inference” as a clinical epistemology (here). Now I am taking that a step further, in a KBP – concept based curriculum – with core movement concepts including causation, adaptation and systems.

It is easy to see how “cause” is important in the study of anatomy. Cause is, after all, how we use anatomical knowledge in practice. Another way to say this (flipped), is that anatomical knowledge is one set of knowledge from which we reason causally. When using abduction to generate a list of possible “causes” of a set of signs and/or symptoms we generally consider the anatomy: “moving the leg like that could be due to a tight X, or a weak Y, or a stiff Z, …, etc.”

It is also easy to see how adaptation is important in the study of anatomy. After all, all of our current adult anatomical musculoskeletal forms are, within a set of boundary constraints, the result of the sum of adaptations of our life long journey to where we are right now. So how we live is predictive of our future anatomy (causal induction and causal deduction), and our current anatomy testifies to the journey we have taken (causal abduction).

Now how about a system? It is certainly easy to talk about the “systems” of our body anatomically. After all, we learn our anatomy in A&P I and II as a set of systems. This is how A&P books are divided. It is a fine way to divide them. Then we get into gross anatomy and we divide up the body regionally – again a fine way to divide up the body to learn it. But what about learning about the system of the system? For example, the system of the skeletal system.Or the system of the muscular system. Or the system of the musculoskeletal system. There is a modularity balanced with integration in all of these (and other) systems.

Well, this is what we can learn from a rather new application of an emerging analytic approach. The application of network analysis (studying systems by their network structure using graph theory (logical, mathematical), and by the way, for those of you that have been reading along with this blog – a “graphical causal model” is simply a network, a particular type of network, a DAG (directed acyclic graph)). This new application to anatomy has been termed: Anatomical Network Analysis (AnNA).

AnNA is exactly what these authors (Diogo R, Esteve-Altava B, Smith C, Boughner JC, Rasskin-Gutman D) have been doing for the past few years:

http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0140030

AnNA is original, insightful and very useful.  Making this particular paper even more useful, the authors have subscribed fully to reproducible research, so their supplements, data, details regarding the analysis including R code – and a vast set of networks – is available on FigShare (here).

The paper citation is:

Article Source: Anatomical Network Comparison of Human Upper and Lower, Newborn and Adult, and Normal and Abnormal Limbs, with Notes on Development, Pathology and Limb Serial Homology vs. Homoplasy
Diogo R, Esteve-Altava B, Smith C, Boughner JC, Rasskin-Gutman D (2015) Anatomical Network Comparison of Human Upper and Lower, Newborn and Adult, and Normal and Abnormal Limbs, with Notes on Development, Pathology and Limb Serial Homology vs. Homoplasy. PLOS ONE 10(10): e0140030. doi: 10.1371/journal.pone.0140030

And I hope it is ok – but here are the first two figures in an attempt to encourage readers to get the paper:

Figure 1:

journal.pone.0140030.g001.PNG
Figure 2:

journal.pone.0140030.g002.PNG

Needless to say, I strongly believe that Anatomical Network Analysis (AnNA) will provide many benefits to the education of anatomy (in addition to the benefits discussed by the authors in this paper (and their other papers)), and potentially to the physical therapy profession as a whole as we work together on the human movement system. I am looking forward to fully exploring the possibilities and implementing an anatomy course and research that utilizes the concepts that emerge (pun intended), when we consider the network structure of the structures we use in practice. What it tells about about anatomical causal associations, what it tells us about anatomical adaption, and what it tells about about the system of anatomical systems with implications for movement.

As a final word – one of the authors on this paper also has a new anatomy book out – from what I have read so far, it is highly recommended (and will be required for PSU – DPT students).

51FtN42PRkL._SX348_BO1,204,203,200_.jpgDIOGO, R., D. NODEN, C. M. SMITH, J. A. MOLNAR, J. BOUGHNER, C. BARROCAS & J. BRUNO (2016). Learning and understanding human anatomy and pathology: an evolutionary and developmental guide for medical students. Taylor & Francis (Oxford, UK). 348 pages.

 

 

 

 

 

 

 

It’s great to be back blogging – it has been a long 18 months getting the PSU-DPT program into the “candidacy” pre-accreditation phase. But now as we look to accept our first class an implement this new program I look forward to blogging more as things unfold.

Randomized controlled trials – a KBP perspective

Re blogging this today in light of a discussion regarding this paper:

Morris PE, Berry MJ, Files D, et al. Standardized Rehabilitation and Hospital Length of Stay Among Patients With Acute Respiratory Failure: A Randomized Clinical Trial. JAMA. 2016;315(24):2694-2702. doi:10.1001/jama.2016.7201.

see: http://jama.jamanetwork.com/article.aspx?articleid=2530536&linkId=26010051

Cause, Models & Inference in Physical Therapy

In evidence based practice randomized controlled trials (RCTs) have a very high standing. In fact, by the GRADE approach to weighing evidence for a clinical practice guideline (CPG) a single, large, well conducted RCT can result in an evidence rating as high as a systematic review of RCTs. This post is not an attempt to argue against the use of RCTs to develop knowledge for practice. The purpose is to simply share some thoughts about the limitations of RCTs and what KBP suggests for RCT method0logy planning.

RCTs are highly regarded because when they demonstrate an effect with a large sample then the cause tested is the most likely explanation for the observed effect (low risk of bias in well designed, large sample RCTs). There will be variability in the effect and the amount of variability is important to consider for the clinician as the patients you treat are not the…

View original post 919 more words

As a professional no one tells you what is important to know for the exam….

These past few weeks I have been thinking a lot about the question – “What is important to know for the exam?” Or, “What should I focus on for the exam?” Usually followed up with propositional statements such as: “There is so much information.” Or “There is so much reading to do.”

Of course, “so much reading” is highly relative. I was recently told that a 3 page document I had provided was “too long” to read completely, but was rather skimmed for the gist of it.I was also asked whether I could provide a summary statement of a one page outline I had developed. If you believe 3 pages is too long , or that a one page outline needs a summary statement, then I am sorry, we may never agree on what too much reading is for a developing professional that would like to practice physical therapy someday.

One of the reasons I struggle with the question about what is important to know for the exam is that I am sympathetic to the students’ plight and I want to help them. So I have now come up with my response.

“What do you think is important to know for the exam?” Followed with questions about whether the student understands the problem they are attempting to solve.

It seems a simple fact but underlying my classes is the fact that I believe it is important that students learn to identify what is important. That is part of what they are learning in their education. They need to learn to assess a problem in such a way that they can identify the relevant from the irrelevant. This means that being able to figure out what is important to know for the exam is part of what you are being evaluated on when given an exam.

Of course this is not exclusive to DPT education, nor to graduate education, or even a college education. In elementary school when you start to work on word problems in mathematics you are essentially learning how to figure out what is important information to solve the problem at hand.

My role as an instructor is to help students develop the ability to figure out what is important. That does not mean (and should not mean) that I outright tell them what is on the exam. It is on me to generate realistic problems for them to work through, experiences to wrestle with, with materials and guidance that they need to read through and develop enough of an understanding  that they develop the ability to identify what is important. And once it is learned that something is important – learn it, make sure it is understood; that is, know it.

Better stop now – or this post may become too much reading 😉

Back to writing, no more wrangling

My previous post announced a move to a new platform. Since that announcement I have wrangled with the new system enough to realize that it detracts too much from the purpose of the blog – writing.

Therefore, with this post I welcome back my WordPress platform and the ease with which it allows me to simply write and post.

New site for Knowledge Based Practice

Just a quick announcement that the WordPress hosted “Cause, models and inference: developing a knowledge based practice” blog is now in its “legacy” stage. Over the past few weeks I have migrated the blog to a new URL, hosted by GitHub pages and developed it using Jekyll. More information is provided at the new site. All of the old posts are there – some as posts, others collected into collections for easier access and reading.

www.knowledgebasedpractice.com

Thanks for reading! I am hoping the new platform enables greater reach and development.