Social Validity

The following are the three concerns of social validity (Wolf, 1978):

First, the social validity of the goals of a study must be determined. If the goals are not socially relevant, the purpose of our research is suspect. For example, suppose we had the goal of determining if students in a reading curriculum acquired and maintained information better if the teacher used fluency building exercises once students mastered a certain skill, as compared to  moving on to a higher level reading skill once students met mastery only. This research question seems desirable since the acquisition of reading skills is critical for future success. On the other hand, attempting to increase the reading skills of high school students with severe mental retardation through phonics instruction may not be justified, since the progress expected over a year of instruction will likely be slower than needed to learn how to read at a proficient level. It would be more important perhaps to provide vocational skills instruction at this point in the students' academic careers.

Second, it must be determined if the procedures used in the study were worth the findings that resulted. That is, do the ends justify the means? Suppose we wanted to improve the reading skills of high school students who were poor readers. The cost of the program may be very expensive and stretch the budget of a district. However, since the long-term outlook of these individuals is likely dim without such services, the long-term costs may outweigh the short-term costs. Society may value the rights of individuals to receive the most appropriate services available no matter the cost; therefore, the high cost of rehabilitation may be justified. On the other hand, attempting to get students to increase their efforts in a reading program though coercive techniques (i.e., threats and warnings) would not be considered justified.

Third, it must be determined whether the effects of the investigation were important. For example, we again had the goal of improving the reading skills of high school students. Upon completion of 1 year of instruction, the students' level of reading improved by three grade levels. We probably would consider this to be a large improvement. On the other hand, if we determined that the reading skills of students improved by half a year, we may not deem this as an important result. [Note: Grade equivalents are rank-ordered data and are not ideal scores to examine; the reader is referred to the use of standard scores].