Using SSD for R in the Classroom: Another final project

We hope you enjoyed Shmuel's final project in last week's post.  We thought that this week, we would share Arthur Zaczkiewicz's final project.  You may remember Arthur - he was our creative student that, in great British tradition, motivated his fellow students with his "Keep Calm and SSDforR On" meme.

Today, you will learn a little more about Arthur's work with one of his clients....

**********

At his agency, Arthur frequently works with clients who are in financial distress.  To help these clients, he has adapted a group financial literacy intervention, Making Ends Meet, for use with individuals.  One of these clients was JD.

Like many folks in recent years, JD has had a difficult time making ends meet.  JD was recently divorced.  He also reported sleep disruptions and anxiety, which he attributed to his financial troubles.  JD was $30,000 in debt, which was left over from his divorce.  JD had contemplated bankruptcy, but decided to attempt to address his financial difficulties first by reaching out to Arthur's agency.

Arthur decided to empirically measure his client's progress over time with two different indicators:  JD's sleep disruption and his level of anxiety.  Both of these were measured on a five-point self-anchoring scale with higher numbers indicating more dysfunction.

Below, you will see simple line graphs depicting JD's progress on both indicators prior to and after the introduction of the intervention:

Using visual analysis alone, it appears as if the intervention is improving both JD's sleep and level of anxiety, but we need to be careful in drawing conclusions!  For both these indicators, Arthur determined that autocorrelation was problematic and trending was a problem in some phases.  Therefore, strictly visual analysis could be misleading!

So.... WHAT IS AN INTERVENTION ANALYST TO DO?

Our recommendation is to look for both statistical significance AND clinical significance.

Because of data issues with trending and autocorrelation, Arthur chose to analyze his data statistically using the conservative dual criteria for both indicators.  Arthur's presentation of his findings for JD's sleep disruptions are shown below.  You will see both the graph and statistical output in his slide.

Well, while it looked like the intervention was making a difference, Arthur found NO statistical significance for JD's sleep disruption.  His findings were similar for JD's level of anxiety; however, Arthur examined clinical significance by looking at effect sizes for each indicator.  In both cases, the intervention produced moderate changes.

As a clinician, this analysis should help Arthur inform his work with JD.  Here are his conclusions:  

Arthur has made some very good points:

1)  Perhaps the intervention needs to continue for a longer period as the intervention seems to be moving the indicators in the right direction. We simply may not have enough data yet.

2)  In the future, it may be useful to examine indicators that are more closely associated with financial difficulties.

3)  Research support can help guide the use of this intervention for others in the future.

Our suggestions:

1)  Keep going, Arthur!  Now that you have learned how to analyze your client's progress, you can do this again and again!

2)  Share what you have learned with others.  You can share your findings with colleagues and others interested in financial literacy programs.  Present at conferences.  Try writing this up and submitting it to a journal.  We need to share what we learn with others.

3)  Continue collaborating with researchers and/or conduct your own research.  The more we look at what is happening in close interactions between clients and practitioners, the more effective we can be in our work.

For more information about SSD for R, check out our website.  And, as always, feel free to contact us!