Using SSD for R in the classroom: FINAL projects

Well, there's nothing like summer in Washington Heights, but all good things must come to an end, and our summer semester is no exception.  We will not be teaching this class in the fall, but will be resuming in the Spring.  Before we move on to other topics, however, we thought we would share with you some highlights of presentations from student who shared their final projects with the class.

This week, we'd like to give a nod to an outstanding student who told me after his final presentation that he could not have imagined, at the beginning of the summer, understanding terms such as SSD for R, autocorrelation, and conservative dual criteria.  You will see below, however, that Rabbi Shmuel Maybruch did, in fact, not only learn those terms, but was able to apply these concepts to evaluating his own practice with great clarity....

The client that this student wanted to evaluate was Quentin, a young man who was looking to date in order to get married, but Quentin never got very far and he sought help.  The indicator that our student used to assess Quentin's progress over time was a self-anchoring scale measuring his despair with dating.  A score of 1 indicated the least amount of despair while a 10 indicated the most.

Five weeks of baseline data were collected before the intervention was initiated.  The intervention consisted of helping Quentin to be more appreciative of the women he dated by identifying and complimenting them on less-than-superficial traits and helping Quentin to identify the role that physical attraction played in his quest for a life partner.

The graph below shows a basic line graph comparing the baseline to the intervention.

Notice how Shmuel made a line graph labeling not only the phases, but added a mean line for each phase, clearly depicting a drop in means between phases.

When deciding how to evaluate his data further, Shmuel noted a problem with both autocorrelation AND trending in the intervention phase, so he decided to use the CDC (conservative dual criteria) to test statistically for a difference between the phases.

From the output in the Console of RStudio, Shmuel learned that he needed nine data points below both the mean and regression lines to achieve statistical significance, and he had twelve!  Looking good so far....

The students learned, however, that statistical significance is difficult to achieve with small samples, although Shmuel was able to detect this level of change in his project.  Effect sizes, however, are very important in intervention research because they can be indicative of clinical, or practical significance.  Shmuel noted a d-index of 1.822, which indicated nearly a 47% change between phases - a moderate change!

While it looks like the intervention worked well for Quentin, Shmuel shared with us what happened post-intervention....

Quentin and his bride are living happily ever after!

For more information about SSD for R, check out our website.  If you have any questions or comments about using SSD for R in your own practice or teaching using this software, feel free to contact us.

Using SSD for R in the classroom: Using the manuscript

About a month and half ago, I told you that we would be road testing a version of our  manuscript for our book with our Master's students this summer.  We thought that, since the summer semester is nearly over (graduation is on Thursday), we'd share with you what we have learned so far this summer.


The manuscript for our book is not at all complete at this point, but we do have a decent draft of six core chapters done.  This is what we have shared with our students.  The first chapter discusses how to quickly and easily import data into R.  The next chapter is an overview of SSD for R and its functionality.  The third chapter discusses how to analyze baseline data with the goal of understanding characteristics of the data.  Chapters 4 and 5 talk about visual and statistical comparisons between phases, and Chapter 6 is about analyzing group data.


But first, a few words about how we have been using our manuscript in class this summer.  We figure this will give you some context about its usage.  Since the manuscript is not complete and in draft-mode, we used the book as a supplement to our main text, Bloom, Fischer, and Orme's EXCELLENT text, Evaluating Practice:  Guidelines for the Accountable Professional.  As we have proceeded through the course, we suggested, but didn't require, students to read various chapters after the material was taught in class in order to provide clarification.

While we have not asked for feedback, one student contacted us earlier this week.  With her permission, we decided to share with you a portion of  Shira Levitt's e-mail:

Hi Professor,

I wanted to work on the final paper at home and was reading the manuscript and it is so well written.  It is clear and concise and truly helpful....I wanted you and Professor Auerbach to know that, as a student who is not good with computers or numbers, this manuscript is phenomenal.

Thank you so much.
Sincerely,
Shira Levitt


Thank you, Shira!

To sum up our feelings about this in a few words - we think we are on to something because this is what we are aiming for!

Come check out our website and, as always, feel free to email us!  We would love to hear from you!

Using SSD for R: Comparing the baseline to the intervention

With the summer semester beginning to wind down, we have reached what our students consider to the the most interesting part of the course - learning how to figure out whether the interventions they were doing with their clients are making a difference.

While we continued with some visual analysis, the students quickly learned why autocorrelated or trending data could not be analyzed as simply as one might think!  Therefore, we taught the students to use the ABbinomial() or ABttest() functions if neither phase has a trend or an issue with autocorrelation.  We also taught the students to use one of the chi-square functions if there was a trend in any phase and to use the critical dual criteria (CDC) if either has an issue of autocorrelation.

One thing that the students really liked was interpreting the statistical output with support from visual output.  

For example, in this example, we are comparing a client's level of enjoyment in the baseline to the level of enjoyment in the intervention.  The regabove() function informed us that there was a statistically significant improvement between phases with 40% of the baseline data being successful and 100% of the intervention data being successful.  This is really easy to visualize when you actually SEE that 2 out of the 5 baseline points are above the regression line in the baseline, while all the points are above it in the intervention phase.

Check out THIS cool output!

This really helps the student understand the notion of continually comparing the intervention to the baseline as they can visualize the baseline regression line being extended into the intervention.  

We only have three more classes to go, so we will be sure to keep you posted on how these final projects are shaping up!

For more information on SSD for R, please visit our website. As always, feel free to email us!

Using SSD for R in the Classroom: Funny anecdotes

So, two amusing things happened this week that we thought we'd share with you before going into the details of how the class went this week.

The first includes a shout-out to one of our most excellent (and creative) students, Arthur Zaczkiewicz, who, besides getting up at 4 am to attend classes in NYC, has decided to entertain and inspire his classmates as they were working on their midterm papers with this:

Brilliant, Arthur, brilliant!

Second amusing story:  As you know, our students have been busily working on their mid-term papers so they have obviously been talking about their projects in the hallways.  One of our students from LAST semester came up to me and said, "I heard your students talking about SSD for R in the hall.  I liked this research class so much that I actually felt nostalgic!"

Well, we don't really hear the words "research" and "nostalgic" in the same sentence, so we really enjoyed our student relating her experience back to us!

And now for our class progress.....

This was a short week for us due to the July 4th holiday.  This week we began talking about visual comparisons between baseline and intervention phases and introduced the students to the binomial function in SSD for R so they can compare what would be considered success in the baseline  and compare it to what would be considered success in the intervention.  To do this, we showed them standard deviation band graphs that extend the standard deviation bands from the baseline through the intervention.  Then we discussed the desired direction of change and how we could consider that data points outside of these bands would be considered "successful" if they were in the desired direction.  Then, using the ABbinomial function, the students could enter the number of successes they observed in both the baseline and intervention to figure out if there were statistically significant differences between the phases.

We thought this was a great way to start our analysis since a) it is visually based, and b) most people readily understand the notion of increasing or decreasing proportions of success across phases.

Next up.... SPC charts.  We offered the students some GREAT beach reading - Orme and Cox's excellent 2001 article entitled, "Analyzing single-subject design data using statistical process control charts."  We wonder how many enthusiastic students will be reading THAT on the beach this holiday weekend!