Introduction

An Introduction to Scientifically-Based Research, the first in a three-course series on effective reading instruction, was designed to give background on Reading First as it applies to the No Child Left Behind federal legislation. The course discusses the research that supports scientifically-based research as it applies to phonetically-based instruction, assessment, and evaluation. The course explores myths and misconceptions concerning reading instruction and remediation. It also presents an evaluation checklist designed to assess the effectiveness of your current reading program. The goal of the course is to present you with research, trustworthy evidence, and background information that support the need for a reading program that is based on scientific research and proven methods.

This computer-based instruction course is a self-supporting program that provides instruction, structured practice, and evaluation, all on your home or school computer. Information on installation and technical support can be found, and will be covered in detail, in the User Guide section of your computer software.

Course Materials

Reading Fundamentals #1: An Introduction to Scientifically-based Research (software package)

Authors: Ronald Martella, Ph.D.

Instructor: Mick Jackson MS/ED

Please keep the CD. There is a $25 replacement fee for CD-roms if you need to replace yours due to theft, damage, misplacement, etc. Call 1-800-313-6744, with your credit card information, if you need a replacement.

Academic Integrity Statement

The structure and format of most distance-learning courses presume a high level of personal and academic integrity in completion and submission of coursework. Individuals enrolled in a CBI distance-learning course are expected to adhere to the following standards of academic conduct.

Academic Work

Academic work submitted by the individual (such as papers, assignments, reports, tests) shall be the student’s own work or appropriately attributed in part or in whole to its correct source. Submission of commercially prepared (or group prepared) materials as if they are one’s own work is unacceptable.

Aiding Honesty in Others

The individual will encourage honesty in others by refraining from providing materials or information to another person with knowledge these materials or information will be used improperly.

Violations of these academic standards may result in the assignment of a failing grade and subsequent loss of credit for the course.

Level of Application

This course is designed to be an informational course with application to educational settings. The curriculum suggestions and teaching strategies explained here were designed to be used for the teaching and remediation of students in kindergarten through sixth grade and an age range from approximately five years to twelve years of age. Some alterations may be needed if working with specific populations such as gifted, ESL, or special education.

Course Objectives:

1. Describe what is meant by critical thinking.

2. Explain what science is and illustrate the six scientific principles.

3. Explain the myths and misconceptions of science, and describe the ways in which we gain information.

4. Describe the impact science has had on medicine, clinical psychology, and education.

5. Illustrate the constraint levels in educational research.

6. Describe the concepts of reliability and validity.

7. Explain what is meant by variability, including the sources of variability.

8. Describe the terms internal and external validity, and explain the threats to each.

9. Illustrate the different research designs/methods (i.e., experimental, single-case, causal-comparative, correlational, and qualitative).

10. Describe the importance of replications and illustrate the types of replications.

11. Describe what is meant by the term research syntheses, and illustrate the National Reading Panel synthesis.

12. Describe the evaluation instrument for Stage I review of reading programs.

Course Description

States that receive funds from the No Child Left Behind, Reading First Act need to ensure that teachers are qualified to teach reading. They must have a working knowledge of scientifically validated instructional programs and practices. According to Kilpatrick (2003), the most critical part of the Act is that there must be an increase in teachers’ knowledge of the scientific process under which instructional programs are evaluated. (Note: A summary of this legislation regarding the use of scientifically-validated instructional materials appears in Course 2.)

According to Moats (1999), research should guide the teaching profession. Unfortunately, teachers are not adequately trained in research methodology in their pre-service programs. An interesting phenomenon occurs in teacher preparation programs. Undergraduate students are rarely required to take research methods or statistics courses. Contrast this with the situation of undergraduates in psychology. Psychology undergraduates are typically required to take research and statistics courses. The interesting aspect of this difference is that students in teacher preparation programs are highly likely to be accountable for the academic progress of students in their classrooms once they become teachers. In comparison, psychology students will likely be much less accountable for the progress of individuals in their charge (e.g., direct care services such as group homes, residential facilities). In other words, if we compare the responsibilities of education college students to psychology college students, the students who would be most in need of training in the scientific process (e.g., data-based decision making) would be those preparing to be teachers.

According to Kilpatrick (2003), approximately 80% of teachers have little to no background in the use and method of science. What is needed, then, is a training program that allows in-service teachers to learn about science. In other words, we need teachers to become consumers of science and to learn how to think critically about the vast amount of data emanating from real science and from what Park (2000) describes as “voodoo science.”

Student Expectations

As a student you will be expected to:

· Complete all 5 information chapters covering Scientifically-based Research, showing a competent understanding of the material presented.

· Complete all 5 chapter examinations, showing a competent understanding of the material presented.

· Complete a review of any chapter on which your examination score was below 70%.

· Retake any chapter examination, after completing an information review, to increase that final examination score to a minimum of 70% (maximum of 3 attempts).

· Complete all course journal article and essay writing assignments with the minimum word count shown for each writing assignment.

· Complete a course evaluation form at the end of the course.

Course Overview

Chapter 1: Introduction to Scientifically-based Research

This first chapter contains information on what scientifically-based research means and discusses the myths and misconceptions of science. This chapter will lay out the basic foundation of scientifically-based research that will be used as the basis for understanding the remaining sections. There will be discussion on the Reading Excellence Act and the impact of scientifically-based research on other professions.

Chapter 2: Constraint Levels, Validity, and Variability in Research

This chapter will discuss the various types of research and the constraint levels in educational research. There will be information on the issues of reliability and validity in research and the variability that has been seen in educational research.

Chapter 3: Internal and External Validity

The third chapter will deal exclusively with internal and external validity of educational research. This chapter focuses solely on these two issues due to their importance and a need for the issue or research validity to be clearly understood.

Chapter 4: Experimental Design

This chapter will discuss quasi-experimental design, pre-experimental design, true experimental design, and single case design. It will discuss causal-comparatives and correlational research as well as qualitative research. The chapter will also discuss objectives and methodology.

Chapter 5: Putting It All Together

Chapter 5 wraps up the course by presenting information on replication and research synthesis. It will discuss evaluation instruments for Stage 1 of a Reading First program. The chapter will end with a general review and prepare the user for information to be presented in the second course of this series.

Examinations

At the end of each course chapter, you will be expected to complete an examination designed to assess your knowledge. You may take these exams a total of three times. The software will save the last score, not the highest score. After your third attempt, each examination will lock and not allow further access. Your final grade for this course will be determined by calculating an average score of all exams. This score will be printed on your final certificate. As this is a self-paced computerized instruction program, you may review course information as often as necessary. You will not be able to exit any examinations until you have answered all questions. If you try to exit the exam before you complete all questions, your information will be lost. You are expected to complete the entire exam in one sitting.

Writing Assignments

This course has two required writing components.

To save your essays:

Macintosh: When you select the question or article you wish to write on, simple text or text edit will automatically be launched. When you are finished, simply click on FILE>SAVE. Do not select SAVE AS. You do not need to give the document a name before saving. When you are done, select FILE>QUIT. You must quit before you write another essay.

Windows: When you select the question or article, Notepad will automatically launch. When you are finished, click on FILE>SAVE. Do not select SAVE AS. You do not need to give the document a name before saving.

1) Essay Requirement: Critical Thinking Questions

There is a critical thinking question for each chapter or section. You will do research on the question and write a brief essay relating it to the course content (and your personal experiences when possible). To view the questions, click on ESSAY REQ, and then on CRITICAL THINKING QUESTIONS. You will see the questions, one for each chapter or section. Click on the question you would like to work on; this will bring up a screen where you can enter your essay. You must write a minimum of 500 words per essay.

2) Essay Requirement: Journal Articles

This task requires you to write a review of three journal articles of your choice on a topic related to this course. You may choose your topic by entering the Key Words (click on the Key Words button) into a search engine of your choice (Google, Dogpile, Yahoo, etc.). Choose three relevant articles and write a 200-word review of each. You may also access the ERIC system and choose a related topic from a journal listed in that system. Or you can access www.scholar.google.com or www.findarticles.com .Write a critical summary of the information given in each article, explaining how the information relates to, supports, or refutes information given in this course. Conclude your paper with your thoughts and impressions. (200 words per journal article minimum.) Be sure to provide the journal name, volume, date, and any other critical information to allow the instructor to access and review that article.

To write your essays, click on ESSAY REQ. In the dialog box below that, click on JOURNAL ARTICLE ASSIGNMENT; this will expand the folder which contains links to the screens where you can write your reviews. When you are ready to stop, click on FILE>SAVE. You may go back at any point to edit your essays. For more information on the features of this assignment, please consult the HELP menu.

_________________________________________________________________________________________________

Instructor Description

Reading Fundamentals #1: An Introduction to Scientifically-based Research has been developed by a team of professionals with educational backgrounds in the areas of clinical psychology, direct reading, and phonetic instructional practices. Mick Jackson, the instructor of record, is a Behavioral Intervention Specialist with a Master's Degree in Special Education and Behavioral Theory and a minor in Reading Remediation. He has 15 years’ combined experience in self-contained special education classrooms, resource rooms, and a hospital day treatment setting. He has conducted oral seminars, presenting to school districts, teacher groups, and at educational conferences.

Contacting the Instructor

You may contact the instructor by emailing Mick at mick@virtualeduc.com or calling him at 800-313-6744 Monday through Friday, 9:00 a.m. - 5:00 p.m. PST. Phone messages will be answered within 24 hours. Phone conferences will be limited to ten minutes per student, per day, given that this is a self-paced instructional program. Please do not contact the instructor about technical problems, course glitches, or other issues that involve the operation of the course.

Technical Questions

If you have questions or problems related to the operation of this course CD, please try everything twice. If the problem persists, please check our support pages for FAQs and known issues at www.virtualeduc.com and also the Help section on your course disk.

If you need personal assistance email support@virtualeduc.com or call (509) 891-7219. When contacting technical support, please know your course version number (it is printed on the CD label) and your operating system and be seated in front of the computer at the time of your call.

Minimum Requirements

Macintosh Operating Systems

Mac OS 9.x or OS 10.x, 256MB of RAM and 5MB of free hard disk space, 15" or larger color monitor with a minimum resolution of 800x600, CD driver 4x minimum speed and a printer connected to your computer.

Windows Operating Systems

Windows 2000, XP Home, Professional or newer, 256MB of RAM and 5MB of free hard disk space; 15" or larger color monitor with a minimum resolution of 800x600, CD driver 4x minimum speed and a printer connected to your computer.

Please contact VESi if you have any questions about the compatibility of these systems.

Refer to the addendum included with your software package regarding Grading Criteria, Course Completion Information, Items to be Submitted, and where to send your completed

information.

Bibliography

Adams, M. J. (1990). Beginning to read: Thinking and learning about print. Cambridge, MA: The MIT Press.

Atkinson, P., & Hammersley, M. (1994). Ethnography and participant observation. In N.K. Denzin & Y.S. Lincoln (Eds.), Handbook of qualitative research (pp. 248-261). Thousand Oaks, CA: Sage.

Barlow, D. H., & Hersen, M. (1984). Single case experimental designs: Strategies for studying behavior change (2nd ed.). New York: Pergamon.

Bornstein, R. F. (1990). Publication politics, experimenter bias and the replication process in social science research. Journal of Social Behavior and Personality, 5(4), 71-81.

Bogdan, R. C., & Biklen, S. K. (1992) Qualitative research for education: An introduction to theory and methods (2nd ed.). Boston: Allyn and Bacon.

Bracht, G. H., & Glass, G. V. (1968). The external validity of experiments. American Educational Journal, 5, 437-474.

Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin.

Chambless, D. L., & Ollendick, T. H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685-716.

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally.

Feldt, L. S., & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed.) (pp. 235-271). Washington, DC: The American Council on Education and the National Council on Measurement in Education.

Fetterman, D. M. (1989). Applied social research methods series: Vol. 17. Ethnography step by step. Newbury Park, CA: Sage.

Fleishman, S., Kohlmoos, J. W., & Rotherham, A. J. (2003, March). From research to practice. Education Week. http://www.edweek.org/ew/ewstory.cfm?slug=26fleischman.h22.

Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Educational research: An introduction (6th ed.). White Plains, NY: Longman.

Gilgun, J. F. (1994). A case for case studies in social work research. Social Work, 39, 371-380.

Gold, R. L. (1969). Roles in sociological field observation. In G.J. McCall & J.L. Simmons (Eds.), Issues in participant observation (pp. 30-39). Reading, MA: Addison-Wesley.

Gould, S. J. (1981). The mismeasure of man. New York: Norton.

Graziano, A. M., & Raulin, M. L. (1993). Research methods: A process of inquiry (2nd ed.). New York: Harper Collins.

Hartman, D. P., & Hall, R. V. (1976). The changing-criterion design. Journal of Applied Behavior Analysis, 9, 527-532.

Hempenstall, K. (2004). The importance of effective instruction. In N. Marchand-Martella, T. Slocum, & R. Martella (Eds.). Introduction to Direct Instruction. Boston, MA: Allyn and Bacon.

Hendrick, C. (1990). Replications, strict replications, and conceptual replications: Are they important? Journal of Social Behavior and Personality, 5(4), 41-49.

Howe, K., & Eisenhart, M. (1990). Standards for qualitative (and quantitative) research: A prolegomenon. Educational Researcher, 19(5), 2-9.

Kazdin, A. E. (1977). Artifact, bias, and complexity of assessment: The ABCs of reliability. Journal of Applied Behavior Analysis, 10, 141-150.

Kilpatrick, J. (2003). Leave no teacher behind. Education News.org. http://www.educationnews.org/writers/jimmy/Leave-No-Teacher-Behind.htm.

Lamal, P. A. (1990). On the importance of replication. Journal of Social Behavior and Personality, 5(4), 31-35.

Martella, R. C., Nelson, R., & Marchand-Martella, N. E. (1999). Research methods: Learning to become a critical research consumer. Boston, MA: Allyn and Bacon.

Meier, K. (1997, February 7). The value of replicating social-science research. The Chronicle of Higher Education, p. B7.

Moats, L. C. (1999). Teaching reading Is rocket science: What expert teachers of reading should know and be able to do. Washington, DC: American Federation of Teachers.

National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Washington, DC: U.S. Department of Health and Human Services.

National Research Council (2002). Scientific research in education. Washington, DC: National Academy Press.

Neuliep, J. W., & Crandall, R. (1993a). Everyone was wrong: There are lots of replications out there. Journal of Social Behavior and Personality, 8(6), 1-8.

Neuliep, J. W., & Crandall, R. (1993b). Reviewer bias against replication research. Journal of Social Behavior and Personality, 8(6), 21-29.

Patton, M. Q. (1990). Qualitative evaluation and research methods. Newbury Park, CA: Sage.

Popper, K. R. (1957/1996). Philosophy of science: A personal report. In S. Sarkar (Ed.), Science and philosophy in the twentieth century: Decline and obsolescence of logical empiricism (pp. 237-273). New York: Garland. (Reprinted from British philosophy in the mid-century: A Cambridge symposium, pp. 155-191, by C.A. Mace, Ed., 1957, New York: Macmillan Norwood Russe).

Potter, W. J. (1996). An analysis of thinking and research about qualitative methods. Mahwah, NJ: Lawrence Erlbaum Associates.

Risley, T. R., & Wolf, M. M. (1972). Strategies for analyzing behavioral change over time. In J. Nesselroade & H. Reese (Eds.), Life-span developmental psychology: Methodological issues (pp. 175-183). New York: Academic Press.

Park, R. (2000). Voodoo science: The road from foolishness to fraud. New York: Oxford.

Rosenthal, R. (1990). Replication in behavioral research. Journal of Social Behavior and Personality, 5(4), 1-30.

Rosnow, R. L., & Rosenthal, R. (1976). The volunteer subject revisited. Australian Journal of Psychology, 28, 97-108.

Sagan, C. (1996). The demon-haunted world: Science as a candle in the dark. New York: Ballantine Books.

Shaver, J. P. (1983). The verification of independent variables in teaching methods research. Educational Research, 12, 3-9.

Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology. New York: Basic Books.

Silverman, D. (1993). Interpreting qualitative data: Methods for analysing talk, text, and interaction. London, Sage.

Simmons, D. C., & Kame’enui, E. J. (2003). A consumer’s guide to evaluating a core reading program grades K-3: A critical elements analysis. Eugene, OR: Institute for the Development of Educational Achievement.

Slavin, R. E. (2003, February). A reader’s guide to scientifically based research: Learning how to assess the validity of education research is vital for creating effective, sustained reform. Educational Leadership, 12-16.

Snow, C. E., Burns, M. S., & Griffin, P. (Eds.). National Research Council. (1998). Preventing reading difficulties in young children. Washington, DC: National Academy Press.

Stanovich, K. E. (1993/1994). Romance and reality. The Reading Teacher, 47, 280-291.

Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage.

Tawney, J. W., & Gast, D. L. (1984). Single subject research in special education. Columbus, OH: Merrill.

The 1999 Omnibus Appropriations Bill. (1998). The Reading Excellence Act. http://www.nrrf.org/essay_ReadingExcel.html.

U.S. Department of Education. (2002). No Child Left Behind Act, 2001. http://www.ed.gov/nclb/landing.jhtml http://www.ed.gov/nclb/landing.jhtml

Wolf, M. M. (1978). Social validity: The case of subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203-214.