| |
Excerpt 1
[Robotics & CPS/CIS in 4-H: Workforce Skills for the 21st Century, University of Nebraska]
|
|
Quantitative
Analysis:
Quantitative analyses of student learning pre-post test differences for treatment and control
|
Our initial research within the project supported that robotics, in this context of 4-H after-school programs and 4-H clubs, is a promising approach for supporting STEM-related learning as reflected on the robotics test. The ANCOVA analysis related to this student learning examined the effect of the robotics intervention using the posttest score of the concepts test as the dependent variable, and using the pretest score and age as covariates. Gender was also entered into the analysis as an additional fixed factor. The main effect for the study group was significant (F(1, 141) = 11.04, p = .001), with the robotics intervention group scoring higher than the control group (m = 11.09, sd = 3.80; and m = 10.68, sd = 3.93 respectively). The pretest was a useful covariate (F(1, 141) = 47.22, p = .001), with an initial score advantage suggested for the smaller control group of approximately two questions. Age was shown to not be a significant covariate (F(1, 141) = 2.31, p = .13) and gender was not a significant factor (F(1, 141) = 8.33, p = .478).
ANCOVA Analysis of Concepts Posttest (as Dependent Variable)
Source |
Sum of
Squares |
df |
Mean
Squares
|
F |
Sig |
Partial
Eta
Squared
|
|
Corrected Model |
782.466 |
7 |
111.781 |
11.467 |
.000 |
.363 |
|
Intercept |
3.737 |
1 |
3.737 |
.383 |
.537 |
.003 |
|
Pretest (SET Concepts) |
460.354 |
1 |
47.223 |
47.223 |
.000 |
.251 |
|
Age |
22.516 |
1 |
22.516 |
2.310 |
.131 |
.016 |
|
Group |
107.630 |
1 |
107.630 |
11.041 |
.001 |
.073 |
|
Gender |
24.355 |
1 |
8.118 |
.833 |
.478 |
.017 |
|
Group by Gender |
11.085 |
1 |
9.748 |
1.137 |
.288 |
.008 |
|
Error |
1374.528 |
141 |
|
|
|
|
|
Total |
20208.000 |
149 |
|
Corrected Total |
2156.993 |
148 |
R Squared = .363 (Adjusted R Squared = .331) |
| |
Excerpt 2
[Robotics & CPS/CIS in 4-H: Workforce Skills for the 21st Century, University of Nebraska] |
|
Quantitative
Analysis
Quantitative analyses of pre-post differences for treatment and control on attitudinal survey and comparison |
In contrast to the learning results as represented by the robotics concepts test, the total score attitudinal analysis showed no significant difference between treatment (m =
142.67, sd 15.03) and control (m = 137.22, sd = 12.34) conditions F (1,75) = .55, p = .46. This lack of significance held for both the total attitudinal score, as well as each of the six underlying scales. Clearly, participation in the robotics program did not impact attitudes towards science, as measured by the SAI in this study's context.
Source |
Sum of
Squares |
df |
Mean
Squares |
F |
Sig |
Partial Eta Squared |
|
Corrected Model |
6488.816 |
2 |
3244.408 |
24.744 |
.000 |
.398 |
|
Intercept |
814.490 |
1 |
814.490 |
6.212 |
.015 |
.076 |
|
Pre (Survey) |
6078.388 |
1 |
6078.388 |
46.357 |
.000 |
.382 |
|
Group |
71.988 |
1 |
71.988 |
.549 |
.461 |
.007 |
|
Error |
75 |
131.121 |
|
|
|
|
|
Total |
1576078.000 |
78 |
|
Corrected Total |
16322.872 |
77 |
|
|
| |
Excerpt 3
[SRI Build IT] |
| Describes qualitative and quantitative analyses performed.
|
Data Analyses
We use descriptive and inferential techniques in order to compare the
intervention and comparison group results. Inferential techniques include
chi-square test of significance for contingency tables (e.g. testing
whether the intervention and comparison groups differ significantly in
their frequency of categorical responses such as ‘yes’ versus ‘no’),
t-test comparison of means, and log-linear regression models. The
log-linear regression models generally take the form:
ln(CHANGE + k) =$ + X 'â +# ln(BASELINE + k) +"BI +! , (1)
where,
ln(CHANGE k) + is the natural logarithm of the post-pre
difference (CHANGE) in the outcome measure for respondent i,
transformed by k units in order to avoid zero and negative values of
CHANGE.
i
X is a matrix of independent variables including for ethnicity,
language spoken at home, age of first computer contact, and computer
engagement and computer course plans as reported at baseline,
ln (BASELINE k)
+ is the natural logarithm of individual i'’s baseline score for the CHANGE
variable, scaled by k units—this controls for potential ceiling and basal
effects in change due to starting scores—were appropriate we replace the log of the baseline with
categorical baseline scores, and
iBI is an indicator that equals 1 if individual i is a Build IT participant, and zero otherwise.
The use of log-transformed models permits the interpretation of model coefficients as percent change in
the outcome measure given the independent variable. Put differently, we could say that, after
considering baseline scores and other salient characteristics, participation in Build IT will
increase the CHANGE+k outcome by 100*?%. Large positive values of ? indicate a positive
influence of Build IT on technology-related development, after accounting for differences in
starting scores.
In addition to these group comparisons, we also assess the whether there were significant
differences between the Build IT participants who completed the entire program and those who
left early. We also examine whether there are potential differences between the EXPLORE and Muir
Build IT sites. All quantitative data was analyzed using the R statistical software. All interview
data was coded and analyzed by HTA using Atlas-Ti qualitative software. The emerging themes and
concepts from the qualitative data were used to develop a theoretical framework for understanding
ALL STARS’ capacity with respect to Build IT implementation. We discuss the results in terms of
this framework.
|
|
|
