Educational Research Report
Allan Feldman
University of Massachusetts Amherst
The educational research component of the STEM RAYS project draws upon literature from activity theory, identity theory, and apprenticeships. Using that literature we developed the following hypotheses that serve as the foundation of the STEM RAYS implementation model and research program:
Hypothesis 1: For students to take on the identity of "scientist" they must act as legitimate participants in authentic science activities.
Hypothesis 2: In order for students to gain the interest in science and the motivation to pursue STEM careers, they must take on the identity of "scientist."
Hypothesis 3: Participation in STEM RAYS activities will have a positive effect on students' science learning.
Hypothesis 4: For the OST activities to be authentic science, teachers and other after-school staff must take on the identity of "scientist" through their own legitimate participation in authentic science activities.
The following research questions are based on these hypotheses:
1) Do students who participate in STEM RAYS programs take on the identity of "scientist?" (Hypothesis 1)
2) If the students do take on the identity of "scientist," how does this affect their interest in science and the motivation to pursue STEM careers? (Hypothesis 2)
3) Does participation in STEM RAYS have a positive effect on students science learning? (Hypothesis 3)
4) Do teachers and other after-school staff take on the identity of "scientist" through their own legitimate participation in authentic science activities? (Hypothesis 4)
5) If the teachers and other after-school staff take on the identity of "scientist," how does this affect their ability to develop and implement authentic science OST activities? (Hypothesis 4)
Questions 1 and 2 are being addressed by measuring (a) students' career interests, (b) their motivation to study STEM subjects, (c) their understanding of the nature of science, and (d) by determining any changes in their use of science language [26]. For (a) and (c) we developed two new instruments. The first is a Career Interest Survey that we based on an instrument provided to use by the Franklin-Hampshire Regional Employment Board. The second is based on the VNOS-E interview protocol (J. S. Lederman & Lederman, 2004). During the pilot year, we administered a modified version of the VNOS-E to all STEM RAYS student participants in focus groups of 4-6 students. We audio-recorded the focus groups and transcribed them. We then coded their responses using the nature of science framework developed by Lederman et al. (2002)We used the student responses to develop a survey that asks students about their agreement with the set of statements that we pulled from the transcripts. We administered this survey to students in the STEM RAYS summer camp and at the beginning of the 2007-08 academic year. For (a) and (b) we are also using the data that we collect for the Program-wide evaluation being conducted by The Exploratorium.
Question 3 is being addressed in two ways. First, students' use of science language is being studied by examining students' products and videotapes of their participation in the STEM RAYS fair. This data has been collected from the pilot sample. Second, at the end of the full implementation year, we will make statistical comparisons of students' scores on the Massachusetts Comprehensive Assessment System (MCAS) examinations. Comparisons will be made among these five groups of students: Students in the after-school programs (self-selected); Students not in the after-school programs (randomly selected); Students in the after-school programs who are in classes taught by the STEM RAYS teachers (self-selected); Students in the after-school programs who are not taught by STEM RAYS teachers (self-selected); and Students who are taught by the STEM RAYS teachers who are not participants in the after-school program (randomly selected).
Questions 4 and 5 is being addressed by (a) measuring teachers' understanding of the nature of science, and (b) determining any changes in their use of science language. We have administered the Views of the Nature of Science/Teaching the Nature of Science (VNOS/TNOS) instrument developed by co-PI Feldman twice to pilot teachers and once to new teachers at the beginning of the 2007-08. Pilot teachers have been interviewed twice and an initial interview has been done of new teachers. All teacher meetings have been either audio- or video-recorded, and transcribed. A new instrument that asks teachers to self-assess their research skills has been developed and has been used at the beginning of the 2007-2008 academic year. It will be repeated at the middle and end of the academic year to track changes in teachers self-evaluation of their skills. Finally, we have asked teachers to complete a Session Report so that we can track what they did in all of the after-school sessions.
All data is being analyzed using predetermined categories based on the hypotheses and goals of the STEM RAYS project. In addition, we use grounded theory to develop emergent categories (Strauss & Corbin, 1990). Data is being coded using HyperResearch software and organized using the methods described by Miles and Huberman (1994). In the final year of the project cases will be constructed according to Yin (2003). The validity of the hypotheses will be examined through cross case analysis.
Preliminary findings:
At this time we have only taken a cursory look at the data and the following findings are speculative. In addition, because the pilot year was used to construct instruments for Questions 1 and 2, there is little that we can say about students' identity formation or motivation to pursue scientific careers. However, it is quite clear that the students learned science as a result of their participation in STEM RAYS. This is evident in their research products at the STEM RAYS fair and the videos of their presentations. Because we have not yet looked at MCAS scores, we cannot say anything about how STEM RAYS participation affects their performance on that examination.
It also clear that some of the pilot teachers learned a great deal of science as a result of their participation (Question 4). This is most evident in the arsenic group. In general the teachers in that group had the weakest academic backgrounds in science. However, by the end of the academic year, they were able to test for arsenic in the environment, to develop hypotheses and experiments to test those hypotheses, and report their findings in a way that was useful for Prof. Tyson's line of research. In addition, the session reports and the students' performance at the STEM RAYS fair suggests that the pilot teachers have developed the ability to develop and implement authentic science OST activities (Question 5).
Lederman, J. S., & Lederman, N. G. (2004, April). Early Elementary Students and Teachers Understandings of Nature of Science and Scientific Inquiry: Lessons Learned From Project ICAN. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, Vancouver, British Columbia.
Lederman, N. G., Abd-El-Khalick, F., Bell, R. L., & Schwartz, R. S. (2002). Views of Nature of Science Questionnaire: Toward Valid and Meaningful Assessment of Learners' Conceptions of Nature of Science. Journal of Research in Science Teaching, 39(6), 497-521.
Miles, M., & Huberman, M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage Publications.
Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage Publications.
Yin, R. (2003). Case study research: Design and methods (3rd ed.). Newbury Park, CA: Sage publications.