Goals of this unit:

1)  Students will understand the differences among scholarly research, program evaluation, and action research.
2)  Students will identify sources of research problems.
3)  Students will determine which type of research is most appropriate to study identified research problems.
4)  Students will know the ethical guidelines that must be followed in educational research.

Research is a valuable part of education, although it is not always used in meaningful or useful ways.

Here is some evidence:  Are you familiar with these educational programs or "theories"?

•     Open Concept Schools
•     Block Scheduling
•     Whole Language Instruction

Read more about educational fads at these sites:

SCHOOL REFORM: Resisting education's fads du jour

Educational Fads Hobble California Students

Multiple Intelligences Theory Makes Educators Feel Good

One of the principle reasons the ideas listed above failed is that they were implemented prior to any empirical research being conducted to verify or refute whether the methods were educationally sound.  This is a typical cycle in education:  an educational fad surfaces that administrators become excited about, usually because of the promise of increased test scores, more time on task, or improvements in attitude or motivation.  The fad is implemented in the schools.  The gains that are expected do not occur and the program or method is removed.

And what are the ramifications of this cycle:  wasted instructional time, low student achievement, teacher frustration, and bad PR (public relations) just to name a few.

So, what's the solution?? RESEARCH!

Here's how the process SHOULD work:

1)  A problem is identified
2)  Research questions result from the identified problem
3)  A review of relevant literature is conducted
4)  If appropriate, testable hypotheses are written
5)  A sample is chosen
6)  Methods for testing the hypotheses or answering the research questions are developed and carried out
7)  Gathered data are analyzed
8)  Conclusions are drawn
9)  Replications are conducted
 

Before ANY instructional method or program is implemented in the schools, it should undergo rigorous research over a long period of time (a few years at a minimum).  If several studies, conducted in different areas with different types of students, verify that the program is effective, then it can be implemented at the school level.  Once implemented, however, it should still be evaluated.  We must ensure the program is being implemented in the ways it was intended (block scheduling only works if teachers are teaching for 90 minutes...it doesn't work if they continue teaching using the 50 minute model and let the kids sit for the remaining 40 minutes!).  We must evaluate the progress of students as well to ensure the method is working in OUR setting.

II.  Research Problems

Research problems are abundant!  In fact, if you consider the notes from the previous section, several problems were identified (notice how research questions come naturally from the topic/problem):
 

• The open concept school:  Is this concept educationally sound?  What are its benefits?  What are its shortcomings?  Is the open concept school better for some types of students (gifted) than for others?  How do teachers feel about this concept?  Can we expect gains or losses in the areas of achievement, motivation, interest, etc. if this concept is adopted?

• Block scheduling:  What are the benefits of block scheduling when it is compared to traditional scheduling?  Is there an increase in student achievement when block scheduling is used?  How do teachers manage time differently in block and traditional schedule environments?  Are there non-achievement benefits of block scheduling (improved motivation to learn, fewer absences, etc.)?  How do teachers, students, and parents feel about block scheduling?

• Whole language:  Are there achievement differences when we compare students who are taught to read in whole language classes and those taught in phonics based classes?  Are these differences consistent when we compare students of differing abilities and backgrounds?  How are the activities of a whole language classroom different from those in a phonics based classroom?  What instructional elements are necessary to ensure success in reading within the whole language classroom?

• My classes are a great source of research problems.  Currently I am interested in determining 1) benefits and pitfalls to using distance learning and online learning to teach educational research; 2) how my students' learning styles affect performance when different types of instructional methods are used (cooperative learning, traditional lecture, independent study, online learning); 3) whether inservice teachers believe good teaching is a skill that can be learned or a natural ability people are born with.

• Educational theories are another source of research problems.  I have conducted research on apprenticeship learning models to determine if apprenticeship models are better than traditional instructional models (lecture/practice/test) in increasing transfer of learning in 7th graders.  My current area of interest is in writing.  Recently I conducted a study at Paulding County High School in order to determine if personality type, fluid reasoning ability, reading for pleasure, and reading/writing self-efficacy (one's beliefs in his/her own abilities) affect writing achievement.  The purpose of this study is to add to the knowledge base on the factors that predict writing achievement.

• Educational programs.  Although educational programs, such as block scheduling, Reading First (a program used in Georgia), and Georgia's Remedial Education Program, don't interest me much, sometimes I am hired as a consultant to evaluate the effectiveness of these types of programs.

    If you are a teacher, your classroom is a rich source of research problems.  You may be interested in determining the best way to improve time-on-task for a few students who have difficulty completing their work.  You may be interested in improving your students' reading comprehension.  Perhaps you have a special needs child in your class whose needs are not being met, and you wish to identify ways to increase the child's successes while meeting the needs of your other students as well.

    Guidance counselors, school administrators, athletics coaches all have rich sources of research problems in their unique environments.  Identifying these problems and then creating plans to deal with them will help you develop personally and professionally.  Your students (or clients, teachers, athletes) will benefit tremendously.  But the greatest benefit will be to you.  Here are some things teacher-researchers (West Georgia graduate students) have to say about conducting research in their own settings:

--"This project has left me with a lot to do in the future. I am currently trying to get all of my information organized. This was a great opportunity for me (being in the position of starting a new program). What I have learned will assist me with giving feedback on what "we" as a county must start doing for students that are not achieving."

--"The study has changed my classroom in the fact that students that were apathetic about their grades at least seem to care now whether they pass or fail.  This is based on the four students I researched on in my class.  It also has allowed me to plan my lesson plans better because I know how my students like to learn."

--"I found this project to be an answer to a problem that I was having with a small group of 5th grade long-term ESL students.  I was concerned about them going to middle school next year, and not having adequate writing skills…When my students were finally able to write something that was enjoyed and appreciated by three classes of kindergarten students, their view of themselves as writers improved dramatically.  As a process, it was very important for me to value their culture and use it as part of my teaching strategy.  My constant modeling of correct English usage as well as the other mechanics of writing helped to improve their writing skills."
 

There are three broad categories of research in education:  1) Scholarly Research, 2) Program Evaluation, and 3) Action Research.  The method we choose is determined by the goal of our research.  The table below outlines the three categories and how they differ.
 
 

Research Category/Method	Who conducts this type of research?	Who is the intended audience?	What is the purpose of this type of research?	Who are the participants in this type of research?	What types of data are collected in this type of research?
Scholarly Research	Typically, professors and instructors in different educational fields conduct scholarly research.	The intended audience is other professionals in education.  Although this can mean teachers and principals, these types of studies are usually written using statistical terms that few lay persons understand.  The audience is usually other professors/instructors in education or other behavioral sciences.	The purpose is to test hypotheses about educational theories.  Purposes can include comparing instructional methods, predicting achievement based on other variables (aptitude, motivation, self-esteem, etc.), evaluating tests and other types of assessment, and explaining differences among groups (male/female, ethnic groups, ability groups).	In scholarly research, it is most desirable to have a random sample of participants chosen from a large population (though this rarely occurs).  Scholarly researchers wish to have a random sample so generalizations or inferences can be made from the sample to a larger population (in other words, I'd like to be able to say that the results of my Paulding County High study are true for all high school students in Georgia).	Quantitative data (test scores) are highly desirable.  Other types of data that can be collected are survey or questionnaire data, data from scales (behavioral, school climate, social, etc.).
Program Evaluation	Administrators at the district or state level usually conduct program evaluations.  Consultants may be hired to assist with certain parts of the evaluation (data collection and analysis).	The intended audience is the stakeholders, which could mean administrators at the district or state department level.	There are many purposes of program evaluation: 1) to determine if programs are being implemented correctly; 2)  to determine short-term and long-term benefits and/or shortcomings of the program; 3)  to determine if the program is cost effective.	If the program evaluation is taking place at the district or school level, typically all individuals involved in the program are used.  If the evaluation occurs at the state level, a random sample is usually chosen because of its cost effectiveness.	Achievement test scores, interview information  from focus groups conducted with teachers or other individuals responsible for program implementation, surveys and questionnaires, anecdotal information.
Action Research	Teachers or other school personnel conduct action research within their setting.	Many times there is no intended audience.  The teacher-researcher simply conducts a study for his or her own purposes.  Action research can be published, however.	The purpose is to identify and study a problem identified by the teacher-researcher.  This may be to increase student learning, to deal with behavior problems, to help the teacher better understand the learning processes of students, etc.	The teacher-researcher is a primary participant.  Students are generally participants in action research, though a single student may be the focus of the project.	Data can include interviews, observations, journal entries by the teacher-researcher, student artifacts (tests, papers, journals), surveys, or questionnaires.

Consider the example of the open concept school and the research problems identified:

• The open concept school:  Is this concept educationally sound?  What are its benefits?  What are its shortcomings?  Is the open concept school better for some types of students (gifted) than for others?  How do teachers feel about this concept?  Can we expect gains or losses in the areas of achievement, motivation, interest, etc. if this concept is adopted?

    FIRST:  Test the open concept theory.  This can occur using scholarly research methods at university lab schools.  Several studies at various locations should be conducted.  We begin by observing teachers and students in the open environment.  We describe a theory of what a open concept school is.  We then begin testing the educational soundness of the open concept school.  We compare student achievement at traditional and open concept schools.  We control as much as we possibly can in our study (make sure students in both types of schools are similar in terms of ability and background, make sure teachers in both settings are similar--we don't want good teachers in the open concept school and poor teachers at the traditional school!).  After studies at several sites are conducted, we analyze the data to determine whether there are positive effects of the open concept school.  We publish our results.

Once our results are published, schools or school districts may wish to implement open concept schools (providing they've been found to be effective).  At this time, the school or district should develop its own program evaluation plan.  The school needs to evaluate the program to make sure it is implemented properly and to evaluate the ways in which students and teachers are adapting to the new concept.  If there are difficulties, they are identified at this stage and can be dealt with accordingly.  Data must be collected so the evaluators can determine the effectiveness of the program.  If the program has been implemented correctly but there is no educational benefit, a determination must be made about whether to keep the program in place.

Teachers may wish to engage in action research in this setting, if an educational problem arises that they wish to study.  For example, a teacher notices that three of her students can not stay on task because of distractions from another class.  The teacher may wish to observe the students to determine what is causing the distraction, then create a plan to deal with the distraction.  The teacher can then implement and evaluate the effectiveness of the plan.

Extra practice  Consider these research problems discussed previously:

• Block scheduling:  What are the benefits of block scheduling when it is compared to traditional scheduling?  Is there an increase in student achievement when block scheduling is used?  How do teachers manage time differently in block and traditional schedule environments?  Are there non-achievement benefits of block scheduling (improved motivation to learn, fewer absences, etc.)?  How do teachers, students, and parents feel about block scheduling?

Whole language:  Are there achievement differences when we compare students who are taught to read in whole language classes and those taught in phonics based classes?  Are these differences consistent when we compare students of differing abilities and backgrounds?  How are the activities of a whole language classroom different from those in a phonics based classroom?  What instructional elements are necessary to ensure success in reading within the whole language classroom?

Based on the information provided about the different methods of research, which method(s) would be best for answering these questions?

---

IV.  Research Ethics
 

When we conduct research in education, we must be aware of ethical considerations.  The four basic guidelines for ethical research are
 

• protect from harm

• get informed consent from research participants

• maintain confidentiality of collected data

• use deception only when necessary

Studies such as the Tuskegee experiments, in which research subjects were not informed of the true nature of their participation and were harmed as a result of their participation, brought to light the ethical problems with conducting certain types of studies.

Read more about the Tuskegee Experiments at:

AMERICA'S DIRTY LITTLE SECRET.  BAD BLOOD: THE TUSKEGEE SYPHILIS EXPERIMENT

High standards have been developed for medical, psychological, and educational research.  You may think that such rigorous guidelines are not necessary in educational research, but they are.  One thing a researcher must consider is how to protect his/her research participants from harm.  As an educational researcher, I do not conduct studies in which I give students experimental drugs, but I do use experimental treatments in some studies.  Consider this example:  As a researcher interested in increasing writing achievement for 7th grade students, I conduct a study at a middle school for the entire school year comparing two methods for teaching students how to write essays.  At the beginning of the study, students in both groups are writing just below grade level.  At the end of the study, I measure writing achievement and find that one group is writing slightly above grade level, but the second group is now writing at about a 5th grade level.  The way I taught that second group has resulted in decreased writing achievement.  I have harmed these students!  Don’t think that educational research is exempt from ethical scrutiny.  We must ensure that we follow the highest ethical standards.

Here's what should happen when a university professor or graduate student is preparing to conduct an educational research study:

1) The researcher must do no harm.  The researcher must consider the risks and the benefits to participants.  If there are risks, the researcher must decide if the study is important enough to warrant these risks.  Think about medical research.  When an experimental treatment for cancer is being studied, one group gets the treatment and the other group gets a placebo (meaning no treatment).  There are serious risks to those who are not being treated, but generally, we believe that the benefits warrant the risk.  We may end up not treating a group of individuals, but if the result is a cure or treatment for a certain type of cancer that will help thousands of people, the harm is outweighed by the benefits.

In educational research, we typically have NO REASON to harm students in any way that is justifiable.  If harm is possible, then we must determine a better way to conduct our study.  For example, in the previous writing example I should be concerned about potential harm.  If I had monitored student progress throughout the school year, I would have known that students in the second group were having a decrease in writing achievement.  I could have stopped the study early on (after all, I already have my answer; I know that the first method is better than the second for increasing writing achievement), and then I could have used the first instructional method for all students so all could reap the benefits.

2) The researcher must get informed consent from the research participants and/or parents if the participants are minors.  In the informed consent forms, the nature and purpose of the study are described.  Risks and benefits of participation are outlined.  The researcher explains what an individual will have to do as a participant in the study.  The researcher gives contact information so participants and/or parents can call the researcher if there are questions or concerns.  The participant is informed that he/she can withdraw from the study at any time without penalty.  The participant and/or parent signs two forms: one to be returned to the researcher and one to keep.

In many cases, consent must be obtained from other sources as well.  If research is conducted within a school, the principal may have to give consent for the study to take place in his/her school.  Some school districts also require clearance through their own administrative offices.  Faculty and students at a university also must seek permission from the university’s Institutional Review Board (IRB).  The IRB is responsible for determining whether the researchers have prepared a study with the highest ethical standards.

Parents or guardians must give consent for their children under 18 years of age.

3) The researcher must maintain confidentiality of collected data.  This can mean many things.  First, when research is published, names of participants are not used (though in some circumstances the participants may give consent for their names to be used, which is VERY RARE).  Second, the researcher does not discuss participants’ data with individuals who are not involved in the study.  When I conduct a study at a school, it is unethical for me to discuss how individual students responded to surveys or scored on tests.  I may not even discuss this with teachers unless given permission by parents or non-minor participants.

4) The researcher must not use deception unless it is necessary.  You may be curious about why a researcher would ever use deception, but it is actually quite common.  One thing a researcher must consider is something called the Hawthorne Effect.  The Hawthorne Effect occurs when research participants change their behavior because they know they are in a research study.  This is a particular problem when participants act in ways they think they are expected to act.  We know we are bound by ethics to inform participants about the nature and purpose of the study (see point 2 above!).  However, if I tell participants that I am conducting a study to determine whether writing achievement can be increased, some individuals may work harder at their writing simply because they think they need to because they are in a study.  Sometimes it is better to give participants the least amount of information possible.  If we do this, we are deceiving participants to a certain extent, and it is our duty to debrief all participants at the end of the study so they know its true purpose and the reasons we were not completely forthcoming with information at the beginning of the study.

One thing that you might be considering is this:  If a researcher must get informed consent (meaning we have explained the purpose of the study as well as what participation means), then how can we ethically deceive participants?  It is a difficult question, and at times it can be difficult for a researcher to find balance between these two concerns.