I can more critically evaluate research now. I pay more attention to the procedures chosen, the results, and most importantly the practical significance. As for the procedures, I can differentiate between quantitative, qualitative, and mixed-methods when reading studies and understand why the researchers chose that particular method. Similarly, I understand the different types of experimental vs. non-experimental designs which are are selected for particular reasons. Specifically, I will be able to critically evaluate published research articles to provide a foundation of information to support any programs, groups, or lessons. For example, if my future school needs a group on traumatic grief counseling, I can look into the existing research to see which programs and interventions worked the best as I plan my own intervention.

I understand the ethical principles and obligations pertaining to research involving human participants. I will keep that in mind as I become a counselor and run my own programs and groups. Some of the ethical principles hold true for school counseling. Confidentiality is obviously paramount when working with children in a school setting to avoid their information spreading throughout the school. It is also important to make sure the groups, programs, etc. do not cause more harm than benefit to the children (because what would be the point then?). Disclosure and (parental) consent are often required before beginning any significant counseling as well.

I’d love to actually run my study for Latino students, if only at my high school as a needs assessment/survey. Those results, while not generalizable, could be helpful in motivating the particular population of students at my school. I can take the results and explain the significance of the statistics (primarily central tendency) and then communicate them to the teachers, administration, school board, or whoever to receive additional support for my interventions.This class has taught me how to explain descriptive statistics like central tendency, standard deviation and correlation. It will be important to clearly communicate results and significance when defending my programs, groups, lessons, etc. against budget cuts. I can show my personal impact through clear, concise data.

Depending on my population and my workload, I can run abridged research studies within my own school using the same principles of real research designs. I can identify a research problem and formulate appropriate research questions based on a needs assessment and my own observations (like my Latino study). I can also compile research to support the program or group I hope to run. With a literature review concisely describing the significance and purpose of my program/group, I would have a more legitimate argument for it. While I don’t foresee myself writing a true research proposal for each of these programs or groups, the skills involved can be transferred to a proposal or grant to start the program or group.

__

When I started this blog, I think I was more pessimistic about being able to tie all these objectives into my school counseling career. As I wrote, however, I discovered that they are much more relevant (in smaller quantities) to supporting my legitimacy as a counselor and the changes I hope to bring to my school and its students. This class has helped me to become more critical of existing research as well as anything I might do on my own. It also reaffirms the need for solid data (in whatever form) to support claims and develop future theories.

-    “What is the essence/meaning of [the experience]?” preliminary, formative questions that are likely to change once the study is underway Selecting Participants -    Usually from a single site but not necessarily

-    About 5-25 individuals selected for following reasons:

Have lived the experience, willing to talk, and **able to articulate their thoughts**

• find a site that exhibits the culture/group you want to study and gain access/permission to the site/individuals
• After some general observations, the researcher purposefully samples a few individuals who will be able to give more detailed insight through interviews.

semi- or un-structured, LONG, multiple sessions possibly

-    Tape-record interviews

-    Primary data reliance on interviews, fewer outside sources

• Observation & Field Notes

(Passive, moderate, active, or complete participation on behalf of the researcher-observer, depending on the situation/culture)

• Interviews

(key informant or focus group)

• Document Analysis

-    Use the researcher’s experience, participant statements (using participant language verbatim), descriptions of actual experience and its construed meaning

• Coding, Coding, Coding …

of all data (major- & sub-codes)

… into categories or patterns

• Look for relationships, draw inferences, use specific examples when possible

(done throughout collection and at the end)
]]> http://lauseme.wordpress.com/2010/04/06/phemenological-vs-ethnographic-designs/feed/ 1 lauseme http://lauseme.wordpress.com/2010/03/28/qualitative-research/ http://lauseme.wordpress.com/2010/03/28/qualitative-research/#comments Sun, 28 Mar 2010 13:55:07 +0000 lauseme http://lauseme.wordpress.com/?p=67 ]]> There are seven key characteristics of qualitative research:

-Natural Setting: the research is conducted in a naturally-occurring setting and observes naturally-occurring behavior. Nothing is directly manipulated by the researcher. Schempp’s study took place in the school where he was observing PE teacher Bob. He followed him to classes and observed Bob’s various duties/behaviors.

-Direct Data Collection: the data comes directly from the source to the researcher. The researcher is the instrument, tool, etc. Schempp was the researcher collecting data from the interviews with Bob, observing Bob, and finding hard evidence to corroborate. Schempp was in the school for approximately a semester, on and off.

-Rich Narrative Descriptions: descriptions are narrative (telling a complete story) with in-depth details that cover the behaviors and contexts surrounding them. Throughout the study, Schempp details various interviews with direct quotations, physical observations, and background knowledge. The descriptions provide a complete picture into Bob’s teaching style and personal beliefs.

-Process orientation: everything is focused on the “why” and “how” of the particular behaviors. The researcher wants to know the connections between behaviors, motivations for behaviors, and the extent of the behaviors. Schempp focused on what made a good teacher, why Bob did so many other duties, why/how Bob chose his lessons, and how Bob structured his classes.

-Participant perspectives: the researcher is primarily concerned with the participants’ points of view, their understanding (of the phenomenon, behavior), and what it all means to them specifically … rather than a broad understanding of the researcher. In the interviews, Schempp focused on Bob’s perspective of class structure, view of his duties, etc. This took place primarily in the interviews and is illustrated by the questions and direct quotations included in the report.

-Inductive data analysis: analyses and generalizations are based on consolidating and synthesizing the collected data. The researcher is looking for similarities and corroborating evidence since the data form is not all identical (unlike test scores). Schempp used different types of data to pull the data from. He used interviews with Bob, observations of Bob, interviews with other colleagues, and document analysis. As it was collected, the different types of data were analyzed for themes and consistent categories.

-Emergent research design: the design is changing as the study occurs, based on the information being collected and learned. The categories ultimately determined by Schempp were not chosen until the research/data collection had begun. The categories were illustrated in the different types of data and what Schempp was learning as the study progressed.

The study design was a survey aimed to develop a student-run mentoring program in an Oregon school system.

The participants were from three samples:

- Latino/non-Latino high students (314), grades 6-12, from Lane County, Oregon

-Latino/non-Latino parents (130), from Lane County, Oregon

-Student attendees of the Oregon State Latino Youth Summit (116)

The researchers collected the survey data by different methods based on the sample group. The researchers administered the survey to Lane County students in school classrooms. No names or identifying information were taken. The researchers administered the survey to the OSLY Summit participants during the Summit. Parent surveys were collected over telephone. All surveys were available in English and Spanish and consisted of open/forced questions and Likert-types.

The data was analyzed primarily with descriptive statistics. Mean survey answers were recorded. The average scores were compared  between 2 of the groups as well as all three groups combined.

The primary conclusions were that Latino students and parents experience/perceive many institutional barriers within the school system; in particular, barriers to the students’ participation in school (compared to non-Latino). It also concluded that academic and extracurricular encouragement by parents and teachers serves as a protective factor promoting school success for Latino students.

The conclusions are fairly valid based on the survey. The perception questions can be considered valid because it is opinion. Without seeing specific questions (and knowing the students’ current academic success), the study may stretch a little far to suggest how encouragement is a protective factor. Further research would be helpful to solidify these conclusions. The study is a survey, so there is a small sampling bias concerning the students/parents who were willing to complete the survey. They may have different circumstances/attitudes about school compared to those who did not participate. For the most part, however, this study’s conclusions are valid because they are not drawing far-fetched conclusions and they recognize the limitations of survey research.

–

Can a Culturally Informed After-School Curriculum Make a Difference in Teen Pregnancy Prevention?

The study design is a causal-comparative study, but there was no control group.  The results were based on the comparison of pre- and post-”treatment” scores and pre/post comments by parents and teachers.

The study’s participants were the same participants of this trial after-school program. Of the 106 program participants, there is data for approximately 52% of them. Those 52% are the study’s participants. The participants were Latino girls who were identified as at-risk by the participating schools. About 75% of the total participants were in 5th or 6th grade. About 20% were in third or fourth grade.

Data collection consisted of attitudinal and perceptional information from female participants, parents, and teachers. This data was collected through questionnaires, which included open-questions and indexes (like self-esteem). The participants’ 1st/4th quarter reading and math scores were also collected by the researchers for a pre/post comparison.

Data analysis was primarily comparative. Perceptional and attitudinal data was scored and the frequency of each response type were compared. It is not specified how the non-numerical data (parent/teacher comments) was analyzed.  Pre- and post-assessment data was compared for all data collected.

The researchers concluded that the program had positive results on participants’ self-esteem but mixed results on overall program efficacy. The researchers concluded that more structured research will be required to more definitely support culturally informed programs such as this one.

The conclusions in this study are valid because the researchers make so few definite conclusions. They recognize the multiple limitations of their study sample and data collection. They also realize that without a control group or more diverse sample, they cannot draw many generalizations. Perceptional data can be highly influenced by personal feelings, day-to-day events, etc. which can provide less solid information. This study’s conclusions, however, do give some support to after-school programs that include cultural awareness components and the need for future research in this area.

–

References

Martinez, C., DeGarmo, D., & Eddy, J. M. (2004). Promoting academic success among Latino youths. Hispanic Journal of Behavioral Sciences, 26(2), 128.

Mendez-Negrete, J., Saldana, L., & Vega, A. (2006). Can a culturally informed after-school curriculum make a difference in teen pregnancy prevention? Preliminary evidence in the case of San Antonio’s escuelitas. Families in Society, 87(1), 95.

For this exercise, it’ll be easiest to imagine a study in which students are placed into an after-school program with some previously-studied characteristics (time commitment, money/fees, Spanish-speaking, membership includes many non-Latino students, etc.) – my survey study.

Threats & Example:

1. Diffusion of Treatment: Treatment is given to the control group. This could occur if one assigned student is from the same household/neighborhood as a non-assigned/control student. The included student shares the activity/knowledge learned, possibly influencing how the other feels about school, study skills, cultural knowledge, etc. and then influencing the results. Now the control subject has the same knowledge and skills as the treatment subject (no accurate comparison).

2. Instrumentation: Poor technical quality/changes in the instrumentation. If the satisfaction surveys change or the researcher poorly monitors the students’ attendance/grades, then we would have less exact numbers to compare after the study concludes.

3. Selection: groups are unequal due to subject differences in the groups. If students are selected based on a particular class they take, those students may be more/less motivated to complete high school regardless of after-school activities. The treatment group may be more economically disadvantaged. The treatment group has an unfair advantage (more motivation) or disadvantage (less motivation, outside barriers) than the comparison group.

4. Hisory: an extraneous event occurs that affects (in some way) the subjects’ performance. If there was another natural disaster in the US or their home country, the students may be distracted or stressed and not focus on school at all – regardless of the extracurricular. Their (dis)engagement would be based on the event, not our program/study, providing inaccurate results.

5. Maturation: subjects’ maturation over the course of the study. High schoolers are often changing on a day-to-day or week-to-week basis, so satisfaction studies may be especially irrelvant/threatened by maturation. Also if the surveys or the program are boring, students are likely to disengage and influence the results by giving poor/inaccurate responses.

6. Attrition: loss of subjects from groups (death, moving, non-return). If the students were to move then their results would be incomplete.  We would not be able to use their previous data because of a lack of a final post-test or measurement (high school graduation).

7. Testing: having taken a pretest. If the students/subjects take a pretest on their perceived likelihood to graduate high school or on their current satisfaction with school, they may be more influenced and likely to pay attention to their motivations, goals, satisfactions, etc. This may influence how they behave during/after the study and be a reason for their high school completion/lack of.

8. Statistical regression: natural movement of extreme scores to the mean. If the subjects take a pre-test on perceived likelihood to graduate (or current satisfaction with school), when they take a post-test they are likely to have a more average score. Maybe they are less stressed,  their social life has calmed down, or they are in a better mood and less likely to report extreme responses on the post-test.

• Cannot determine the direction of the relationship (which variables “causes” the other)
• There may be a third (or fourth) variable outside those studied that has an effect on the overall relationship
• If the measuring instrument isn’t very reliable, then it may not be accurately identifying relationships
• Variability – too large of a range or too many variables can create a false relationship; too little variability may not demonstrate a relationship even if one exists

Demonstrated in Legette’s study …

There is no way to absolutely determine the direction of the relationship. Considering school level and attributions … does the student’s school type determine the environment or does the environment impact the type of school/those drawn to work there?

There were 3 independent variables and 4 dependent variables. With that many variables that there was bound to be some type of relationship between a combination of the characteristics.  “Background” is particularly vague and could mean different things to different students – previous knowledge, parental support/experience, etc.

There may also be outside variables that effect the relationship. With the city/county schools, there may be outside variables like funding and available resources that influence how a student perceives their classroom or any ability (less money for private lessons = less ability?).  Does that have an effect on how they respond? (maybe) It is completely accurate to make causal interpretations? (no)

The sample variability within the study is also interesting. Is 1,100 students a large enough sample to generalize to? Specifically, over 50% of the students are elementary and less than 25% are high school. That may not have an overwhelming effect, but one should keep that in consideration. Since it was convenience sampling, the participants may not be very representative of the population.

#4: Subjects must be protected from physical and mental discomfort, harm, and danger. Being in a research study should not be detrimental to an individual. If participants are caused excessive discomfort or harm, there is likely to be more backlash that progress due to this study. Especially with minors, there should be minimal discomfort of any kind. It is our job to improve education, not hinder it by hurting those involved.

#3: The investigator should be as open and honest with the participants as possible. Obviously the researcher is not going to explain the hypotheses explicitly to participants, but the participants should have some awareness. If the researchers are open/honest, the participants are more likely to trust them and hopefully be more relaxed in the setting.  I think this is especially crucial for qualitative research like ethnographies.

#5: Informed consent must be obtained… Parental consent is required. For many of us, we will be working in schools with minors. The researcher can be open with the participants by outlining the study and discussing any benefits/risks. Without permission, participants could feel coerced into the study. In that case, are they in any better situation than lab rats?  Children and teenagers should assent to participating as individuals because they should also feel some control in the situation. Teenagers who are forced often don’t provide the best behavior, results, answers, data, etc.

#8: Data should be  held confidential. Confidentiality is an ethical tenet of school counseling as well because it protects the client. Many of these studies involve personal information. If participants feel like their information will be self-identifying or overly spread, they are more likely to be uncomfortable and possible provide less accurate information/data.

Since I’ll likely be using a survey for this study, I will need a larger pool of individuals. This was my process before looking at the sampling method options …

Target Population: High School Latino Students

Survey Population: High School Latino Students in Northern Virginia (highest concentration of Latino students in the state)

Sample Size: 250-350 students … High School Latino Students from 10 area high schools

The best fit for this particular sample is a cluster sampling method.  Cluster sampling provides a more representative sample group, which is important. I think that generalizability is key to this study.This study is a preliminary study that would hopefully lead to later studies/experimental programs using the data collected. It is important to have the most representative group as possible (or can be expected when working with adolescents, legal/illegal immigrants or citizens).

The schools are already defined in cluster groups, making it easier to sample from the various schools selected. In Northern Virginia, there are five main public school districts – Arlington, Alexandria, Fairfax, Loudoun, and Falls Church.  I could select schools/participants from 2 high schools in each district and increasing the likelihood of diversity in the sample.

Since the study would be conducted with minors at schools or with schools’ permission, it would be important to have a large survey population from which to choose. Cluster sampling would provide the opportunity to reach more individuals.

The first step is to identify a topic and key terms. Since I’m studying Latinos, extracurricular activities, and educational aspirations (specifically dropout tendencies) those were my initial key terms. “Latinos” is a pretty straight forward term, but “extra curricular activities” and “educational aspirations” required more flexibility on research-ability. I made a chart with related words that I could use interchangeably in the databases. I added to the list when I found a new term.

After having a preliminary game plan, I went to find my databases. From advice in class, I chose databases that covered different areas. My topic is not specifically psychological or educational, so I needed research from both fields. I also found helpful articles from sociological databases.

So I started putting in my research terms and I many results – too many to all be relevant. So I began to narrow my search by the terms on the side. I found many relevant articles in the school section (not to my surprise). I ran a few slightly different searches to encompass everything I thought was relevant.

All the sources I found on this day were primary sources. Some were actual research studies that were conducted. A few were analysis of data collected. Each one of these studies looked at a different aspect of Latinos and educational status. They are not the most substantive information but it was good for a preliminary background. It gave me ideas on how to design my research study by actually conducting research on their concepts and findings.

So after a few hours of reading through abstracts online …

… I started to read through the articles I found most useful. And my desktop looked something like this. And I didn’t just rely on the database searches. As I read some articles, I pulled up a few articles that were previously referenced. Sometimes they were useful but not always.

I created a research matrix similar to the one shown in class to organize all the information I was reading. I found it really helpful to see where my research was focusing. I could also compare and contrast the methods, results, etc.

At the beginning of my lit review research, I was feeling overwhelmed by all the articles and information. I wasn’t sure how to fit my study into existing research and make it unique. Afterward, I ‘m feeling more at ease. I’m starting to think of what I’ll specifically study and how to add to the existing research. Lit reviews are exhaustive (and exhausting). Just for a frame of reference, I probably spent about 4 hours on round 1 of research. So far, I have about 10-12 articles that will be helpful in my literature review. More to come!