Next week, we are on tour! Well, not the wait-on-line all night for tix kind of tour or, in the modern era, pummeling Ticketmaster.com repeatedly until you get the seats you want kind-of-tour. This is more like the software developer/statistical-folks type of tour.
Actually, this is the first in a series of three workshops we are giving at conferences this year. Next Saturday, October 19, we will be at The American Evaluation Association's Annual Conference in the Morgan Room at the Washington Hilton from 10:45 - 11:30.
Two weeks later, we will be at the Council on Social Work Education's APM at the Hilton Anatole in Dallas, Texas. Our presentation is bright and early: 7:30 - 8:30 on Sunday, November 3 in the Edelweiss Room.
Then, we won't be presenting again until January, where we will be doing another workshop at Society for Social Work and Research's Annual Conference in San Antonio, Texas.
Three conferences tailored to three different audiences! Join us and let us know what you think. In the mean time, we will be posting our presentation slides on our website after each conference.
Saturday, October 12, 2013
Tuesday, September 3, 2013
Lots and lots of activity
Well, the summer session ended and you would think that we had a little bit of a break, but we have been busy, busy, busy!
First, we received acceptances to present SSD for R at two different conferences and we are waiting to hear about a third. In mid-October, we will be taking what seems to be our annual trek to Washington, DC, to present at the American Evaluation Association meeting. At the end of October, we will be flying down to Dallas to present at the Council on Social Work Education's Annual Program Meeting. At both of these conferences, we will be doing long-ish skills presentations.
The goal of both of these is to show people how to use SSD for R in their work as evaluators, professors, and researchers. Once the presentations are completed, we are planning to upload the PowerPoints to our website. As each of these meetings get closer, we will let you know when and where we are doing our talks. We would love for you to join us!
The second VERY EXCITING thing that we did was complete the first draft of our book, which we have finally given a working title: SSD for R: An R Package For Analyzing Single-Subject Data. We sent the entire manuscript off to the wonderful folks at Oxford today. The reviewers we had for our proposal were really helpful at giving suggestions that, we think, really strengthened the book overall, so we are looking forward to more helpful feedback in the coming weeks.
As usual, we look forward to hearing from you with any comments you may have. Feel free to e-mail us!
First, we received acceptances to present SSD for R at two different conferences and we are waiting to hear about a third. In mid-October, we will be taking what seems to be our annual trek to Washington, DC, to present at the American Evaluation Association meeting. At the end of October, we will be flying down to Dallas to present at the Council on Social Work Education's Annual Program Meeting. At both of these conferences, we will be doing long-ish skills presentations.
The goal of both of these is to show people how to use SSD for R in their work as evaluators, professors, and researchers. Once the presentations are completed, we are planning to upload the PowerPoints to our website. As each of these meetings get closer, we will let you know when and where we are doing our talks. We would love for you to join us!
The second VERY EXCITING thing that we did was complete the first draft of our book, which we have finally given a working title: SSD for R: An R Package For Analyzing Single-Subject Data. We sent the entire manuscript off to the wonderful folks at Oxford today. The reviewers we had for our proposal were really helpful at giving suggestions that, we think, really strengthened the book overall, so we are looking forward to more helpful feedback in the coming weeks.
As usual, we look forward to hearing from you with any comments you may have. Feel free to e-mail us!
Sunday, August 4, 2013
Using SSD for R in the Classroom: Another final project
We hope you enjoyed Shmuel's final project in last week's post. We thought that this week, we would share Arthur Zaczkiewicz's final project. You may remember Arthur - he was our creative student that, in great British tradition, motivated his fellow students with his "Keep Calm and SSDforR On" meme.
Today, you will learn a little more about Arthur's work with one of his clients....
At his agency, Arthur frequently works with clients who are in financial distress. To help these clients, he has adapted a group financial literacy intervention, Making Ends Meet, for use with individuals. One of these clients was JD.
Like many folks in recent years, JD has had a difficult time making ends meet. JD was recently divorced. He also reported sleep disruptions and anxiety, which he attributed to his financial troubles. JD was $30,000 in debt, which was left over from his divorce. JD had contemplated bankruptcy, but decided to attempt to address his financial difficulties first by reaching out to Arthur's agency.
Arthur decided to empirically measure his client's progress over time with two different indicators: JD's sleep disruption and his level of anxiety. Both of these were measured on a five-point self-anchoring scale with higher numbers indicating more dysfunction.
Below, you will see simple line graphs depicting JD's progress on both indicators prior to and after the introduction of the intervention:
Using visual analysis alone, it appears as if the intervention is improving both JD's sleep and level of anxiety, but we need to be careful in drawing conclusions! For both these indicators, Arthur determined that autocorrelation was problematic and trending was a problem in some phases. Therefore, strictly visual analysis could be misleading!
So.... WHAT IS AN INTERVENTION ANALYST TO DO?
Our recommendation is to look for both statistical significance AND clinical significance.
Because of data issues with trending and autocorrelation, Arthur chose to analyze his data statistically using the conservative dual criteria for both indicators. Arthur's presentation of his findings for JD's sleep disruptions are shown below. You will see both the graph and statistical output in his slide.
Today, you will learn a little more about Arthur's work with one of his clients....
**********
At his agency, Arthur frequently works with clients who are in financial distress. To help these clients, he has adapted a group financial literacy intervention, Making Ends Meet, for use with individuals. One of these clients was JD.
Like many folks in recent years, JD has had a difficult time making ends meet. JD was recently divorced. He also reported sleep disruptions and anxiety, which he attributed to his financial troubles. JD was $30,000 in debt, which was left over from his divorce. JD had contemplated bankruptcy, but decided to attempt to address his financial difficulties first by reaching out to Arthur's agency.
Arthur decided to empirically measure his client's progress over time with two different indicators: JD's sleep disruption and his level of anxiety. Both of these were measured on a five-point self-anchoring scale with higher numbers indicating more dysfunction.
Below, you will see simple line graphs depicting JD's progress on both indicators prior to and after the introduction of the intervention:
So.... WHAT IS AN INTERVENTION ANALYST TO DO?
Our recommendation is to look for both statistical significance AND clinical significance.
Because of data issues with trending and autocorrelation, Arthur chose to analyze his data statistically using the conservative dual criteria for both indicators. Arthur's presentation of his findings for JD's sleep disruptions are shown below. You will see both the graph and statistical output in his slide.
Well, while it looked like the intervention was making a difference, Arthur found NO statistical significance for JD's sleep disruption. His findings were similar for JD's level of anxiety; however, Arthur examined clinical significance by looking at effect sizes for each indicator. In both cases, the intervention produced moderate changes.
As a clinician, this analysis should help Arthur inform his work with JD. Here are his conclusions:
Arthur has made some very good points:
1) Perhaps the intervention needs to continue for a longer period as the intervention seems to be moving the indicators in the right direction. We simply may not have enough data yet.
2) In the future, it may be useful to examine indicators that are more closely associated with financial difficulties.
3) Research support can help guide the use of this intervention for others in the future.
Our suggestions:
1) Keep going, Arthur! Now that you have learned how to analyze your client's progress, you can do this again and again!
2) Share what you have learned with others. You can share your findings with colleagues and others interested in financial literacy programs. Present at conferences. Try writing this up and submitting it to a journal. We need to share what we learn with others.
3) Continue collaborating with researchers and/or conduct your own research. The more we look at what is happening in close interactions between clients and practitioners, the more effective we can be in our work.
For more information about SSD for R, check out our website. And, as always, feel free to contact us!
Sunday, July 28, 2013
Using SSD for R in the classroom: FINAL projects
Well, there's nothing like summer in Washington Heights, but all good things must come to an end, and our summer semester is no exception. We will not be teaching this class in the fall, but will be resuming in the Spring. Before we move on to other topics, however, we thought we would share with you some highlights of presentations from student who shared their final projects with the class.
This week, we'd like to give a nod to an outstanding student who told me after his final presentation that he could not have imagined, at the beginning of the summer, understanding terms such as SSD for R, autocorrelation, and conservative dual criteria. You will see below, however, that Rabbi Shmuel Maybruch did, in fact, not only learn those terms, but was able to apply these concepts to evaluating his own practice with great clarity....
The client that this student wanted to evaluate was Quentin, a young man who was looking to date in order to get married, but Quentin never got very far and he sought help. The indicator that our student used to assess Quentin's progress over time was a self-anchoring scale measuring his despair with dating. A score of 1 indicated the least amount of despair while a 10 indicated the most.
Five weeks of baseline data were collected before the intervention was initiated. The intervention consisted of helping Quentin to be more appreciative of the women he dated by identifying and complimenting them on less-than-superficial traits and helping Quentin to identify the role that physical attraction played in his quest for a life partner.
The graph below shows a basic line graph comparing the baseline to the intervention.
Quentin and his bride are living happily ever after!
For more information about SSD for R, check out our website. If you have any questions or comments about using SSD for R in your own practice or teaching using this software, feel free to contact us.
This week, we'd like to give a nod to an outstanding student who told me after his final presentation that he could not have imagined, at the beginning of the summer, understanding terms such as SSD for R, autocorrelation, and conservative dual criteria. You will see below, however, that Rabbi Shmuel Maybruch did, in fact, not only learn those terms, but was able to apply these concepts to evaluating his own practice with great clarity....
The client that this student wanted to evaluate was Quentin, a young man who was looking to date in order to get married, but Quentin never got very far and he sought help. The indicator that our student used to assess Quentin's progress over time was a self-anchoring scale measuring his despair with dating. A score of 1 indicated the least amount of despair while a 10 indicated the most.
Five weeks of baseline data were collected before the intervention was initiated. The intervention consisted of helping Quentin to be more appreciative of the women he dated by identifying and complimenting them on less-than-superficial traits and helping Quentin to identify the role that physical attraction played in his quest for a life partner.
The graph below shows a basic line graph comparing the baseline to the intervention.
Notice how Shmuel made a line graph labeling not only the phases, but added a mean line for each phase, clearly depicting a drop in means between phases.
When deciding how to evaluate his data further, Shmuel noted a problem with both autocorrelation AND trending in the intervention phase, so he decided to use the CDC (conservative dual criteria) to test statistically for a difference between the phases.
From the output in the Console of RStudio, Shmuel learned that he needed nine data points below both the mean and regression lines to achieve statistical significance, and he had twelve! Looking good so far....
The students learned, however, that statistical significance is difficult to achieve with small samples, although Shmuel was able to detect this level of change in his project. Effect sizes, however, are very important in intervention research because they can be indicative of clinical, or practical significance. Shmuel noted a d-index of 1.822, which indicated nearly a 47% change between phases - a moderate change!
While it looks like the intervention worked well for Quentin, Shmuel shared with us what happened post-intervention....
Quentin and his bride are living happily ever after!
For more information about SSD for R, check out our website. If you have any questions or comments about using SSD for R in your own practice or teaching using this software, feel free to contact us.
Saturday, July 20, 2013
Using SSD for R in the classroom: Using the manuscript
About a month and half ago, I told you that we would be road testing a version of our manuscript for our book with our Master's students this summer. We thought that, since the summer semester is nearly over (graduation is on Thursday), we'd share with you what we have learned so far this summer.
The manuscript for our book is not at all complete at this point, but we do have a decent draft of six core chapters done. This is what we have shared with our students. The first chapter discusses how to quickly and easily import data into R. The next chapter is an overview of SSD for R and its functionality. The third chapter discusses how to analyze baseline data with the goal of understanding characteristics of the data. Chapters 4 and 5 talk about visual and statistical comparisons between phases, and Chapter 6 is about analyzing group data.
But first, a few words about how we have been using our manuscript in class this summer. We figure this will give you some context about its usage. Since the manuscript is not complete and in draft-mode, we used the book as a supplement to our main text, Bloom, Fischer, and Orme's EXCELLENT text, Evaluating Practice: Guidelines for the Accountable Professional. As we have proceeded through the course, we suggested, but didn't require, students to read various chapters after the material was taught in class in order to provide clarification.
While we have not asked for feedback, one student contacted us earlier this week. With her permission, we decided to share with you a portion of Shira Levitt's e-mail:
Hi Professor,
I wanted to work on the final paper at home and was reading the manuscript and it is so well written. It is clear and concise and truly helpful....I wanted you and Professor Auerbach to know that, as a student who is not good with computers or numbers, this manuscript is phenomenal.
Thank you so much.
Sincerely,
Shira Levitt
Thank you, Shira!
To sum up our feelings about this in a few words - we think we are on to something because this is what we are aiming for!
Come check out our website and, as always, feel free to email us! We would love to hear from you!
The manuscript for our book is not at all complete at this point, but we do have a decent draft of six core chapters done. This is what we have shared with our students. The first chapter discusses how to quickly and easily import data into R. The next chapter is an overview of SSD for R and its functionality. The third chapter discusses how to analyze baseline data with the goal of understanding characteristics of the data. Chapters 4 and 5 talk about visual and statistical comparisons between phases, and Chapter 6 is about analyzing group data.
But first, a few words about how we have been using our manuscript in class this summer. We figure this will give you some context about its usage. Since the manuscript is not complete and in draft-mode, we used the book as a supplement to our main text, Bloom, Fischer, and Orme's EXCELLENT text, Evaluating Practice: Guidelines for the Accountable Professional. As we have proceeded through the course, we suggested, but didn't require, students to read various chapters after the material was taught in class in order to provide clarification.
While we have not asked for feedback, one student contacted us earlier this week. With her permission, we decided to share with you a portion of Shira Levitt's e-mail:
Hi Professor,
I wanted to work on the final paper at home and was reading the manuscript and it is so well written. It is clear and concise and truly helpful....I wanted you and Professor Auerbach to know that, as a student who is not good with computers or numbers, this manuscript is phenomenal.
Thank you so much.
Sincerely,
Shira Levitt
Thank you, Shira!
To sum up our feelings about this in a few words - we think we are on to something because this is what we are aiming for!
Come check out our website and, as always, feel free to email us! We would love to hear from you!
Saturday, July 13, 2013
Using SSD for R: Comparing the baseline to the intervention
With the summer semester beginning to wind down, we have reached what our students consider to the the most interesting part of the course - learning how to figure out whether the interventions they were doing with their clients are making a difference.
While we continued with some visual analysis, the students quickly learned why autocorrelated or trending data could not be analyzed as simply as one might think! Therefore, we taught the students to use the ABbinomial() or ABttest() functions if neither phase has a trend or an issue with autocorrelation. We also taught the students to use one of the chi-square functions if there was a trend in any phase and to use the critical dual criteria (CDC) if either has an issue of autocorrelation.
One thing that the students really liked was interpreting the statistical output with support from visual output.
For example, in this example, we are comparing a client's level of enjoyment in the baseline to the level of enjoyment in the intervention. The regabove() function informed us that there was a statistically significant improvement between phases with 40% of the baseline data being successful and 100% of the intervention data being successful. This is really easy to visualize when you actually SEE that 2 out of the 5 baseline points are above the regression line in the baseline, while all the points are above it in the intervention phase.
Check out THIS cool output!
This really helps the student understand the notion of continually comparing the intervention to the baseline as they can visualize the baseline regression line being extended into the intervention.
We only have three more classes to go, so we will be sure to keep you posted on how these final projects are shaping up!
Saturday, July 6, 2013
Using SSD for R in the Classroom: Funny anecdotes
So, two amusing things happened this week that we thought we'd share with you before going into the details of how the class went this week.
The first includes a shout-out to one of our most excellent (and creative) students, Arthur Zaczkiewicz, who, besides getting up at 4 am to attend classes in NYC, has decided to entertain and inspire his classmates as they were working on their midterm papers with this:
Brilliant, Arthur, brilliant!
Second amusing story: As you know, our students have been busily working on their mid-term papers so they have obviously been talking about their projects in the hallways. One of our students from LAST semester came up to me and said, "I heard your students talking about SSD for R in the hall. I liked this research class so much that I actually felt nostalgic!"
Well, we don't really hear the words "research" and "nostalgic" in the same sentence, so we really enjoyed our student relating her experience back to us!
And now for our class progress.....
This was a short week for us due to the July 4th holiday. This week we began talking about visual comparisons between baseline and intervention phases and introduced the students to the binomial function in SSD for R so they can compare what would be considered success in the baseline and compare it to what would be considered success in the intervention. To do this, we showed them standard deviation band graphs that extend the standard deviation bands from the baseline through the intervention. Then we discussed the desired direction of change and how we could consider that data points outside of these bands would be considered "successful" if they were in the desired direction. Then, using the ABbinomial function, the students could enter the number of successes they observed in both the baseline and intervention to figure out if there were statistically significant differences between the phases.
We thought this was a great way to start our analysis since a) it is visually based, and b) most people readily understand the notion of increasing or decreasing proportions of success across phases.
Next up.... SPC charts. We offered the students some GREAT beach reading - Orme and Cox's excellent 2001 article entitled, "Analyzing single-subject design data using statistical process control charts." We wonder how many enthusiastic students will be reading THAT on the beach this holiday weekend!
The first includes a shout-out to one of our most excellent (and creative) students, Arthur Zaczkiewicz, who, besides getting up at 4 am to attend classes in NYC, has decided to entertain and inspire his classmates as they were working on their midterm papers with this:
Brilliant, Arthur, brilliant!
Second amusing story: As you know, our students have been busily working on their mid-term papers so they have obviously been talking about their projects in the hallways. One of our students from LAST semester came up to me and said, "I heard your students talking about SSD for R in the hall. I liked this research class so much that I actually felt nostalgic!"
Well, we don't really hear the words "research" and "nostalgic" in the same sentence, so we really enjoyed our student relating her experience back to us!
And now for our class progress.....
This was a short week for us due to the July 4th holiday. This week we began talking about visual comparisons between baseline and intervention phases and introduced the students to the binomial function in SSD for R so they can compare what would be considered success in the baseline and compare it to what would be considered success in the intervention. To do this, we showed them standard deviation band graphs that extend the standard deviation bands from the baseline through the intervention. Then we discussed the desired direction of change and how we could consider that data points outside of these bands would be considered "successful" if they were in the desired direction. Then, using the ABbinomial function, the students could enter the number of successes they observed in both the baseline and intervention to figure out if there were statistically significant differences between the phases.
We thought this was a great way to start our analysis since a) it is visually based, and b) most people readily understand the notion of increasing or decreasing proportions of success across phases.
Next up.... SPC charts. We offered the students some GREAT beach reading - Orme and Cox's excellent 2001 article entitled, "Analyzing single-subject design data using statistical process control charts." We wonder how many enthusiastic students will be reading THAT on the beach this holiday weekend!
Subscribe to:
Posts (Atom)