
Excerpt 1
[Robotics & CPS/CIS in 4H: Workforce Skills for the 21^{st} Century, University of Nebraska]

Quantitative
Analysis:
Quantitative analyses of student learning prepost test differences for treatment and control

Our initial research within the project supported that robotics, in this context of 4H afterschool programs and 4H clubs, is a promising approach for supporting STEMrelated learning as reflected on the robotics test. The ANCOVA analysis related to this student learning examined the effect of the robotics intervention using the posttest score of the concepts test as the dependent variable, and using the pretest score and age as covariates. Gender was also entered into the analysis as an additional fixed factor. The main effect for the study group was significant (F(1, 141) = 11.04, p = .001), with the robotics intervention group scoring higher than the control group (m = 11.09, sd = 3.80; and m = 10.68, sd = 3.93 respectively). The pretest was a useful covariate (F(1, 141) = 47.22, p = .001), with an initial score advantage suggested for the smaller control group of approximately two questions. Age was shown to not be a significant covariate (F(1, 141) = 2.31, p = .13) and gender was not a significant factor (F(1, 141) = 8.33, p = .478).
ANCOVA Analysis of Concepts Posttest (as Dependent Variable)
Source 
Sum of
Squares 
df 
Mean
Squares

F 
Sig 
Partial
Eta
Squared

Corrected Model 
782.466 
7 
111.781 
11.467 
.000 
.363 
Intercept 
3.737 
1 
3.737 
.383 
.537 
.003 
Pretest (SET Concepts) 
460.354 
1 
47.223 
47.223 
.000 
.251 
Age 
22.516 
1 
22.516 
2.310 
.131 
.016 
Group 
107.630 
1 
107.630 
11.041 
.001 
.073 
Gender 
24.355 
1 
8.118 
.833 
.478 
.017 
Group by Gender 
11.085 
1 
9.748 
1.137 
.288 
.008 
Error 
1374.528 
141 




Total 
20208.000 
149 
Corrected Total 
2156.993 
148 
R Squared = .363 (Adjusted R Squared = .331) 

Excerpt 2
[Robotics & CPS/CIS in 4H: Workforce Skills for the 21^{st} Century, University of Nebraska] 
Quantitative
Analysis
Quantitative analyses of prepost differences for treatment and control on attitudinal survey and comparison 
In contrast to the learning results as represented by the robotics concepts test, the total score attitudinal analysis showed no significant difference between treatment (m =
142.67, sd 15.03) and control (m = 137.22, sd = 12.34) conditions F (1,75) = .55, p = .46. This lack of significance held for both the total attitudinal score, as well as each of the six underlying scales. Clearly, participation in the robotics program did not impact attitudes towards science, as measured by the SAI in this study's context.
Source 
Sum of
Squares 
df 
Mean
Squares 
F 
Sig 
Partial Eta Squared 
Corrected Model 
6488.816 
2 
3244.408 
24.744 
.000 
.398 
Intercept 
814.490 
1 
814.490 
6.212 
.015 
.076 
Pre (Survey) 
6078.388 
1 
6078.388 
46.357 
.000 
.382 
Group 
71.988 
1 
71.988 
.549 
.461 
.007 
Error 
75 
131.121 




Total 
1576078.000 
78 
Corrected Total 
16322.872 
77 



Excerpt 3
[SRI Build IT] 
Describes qualitative and quantitative analyses performed.

Data Analyses
We use descriptive and inferential techniques in order to compare the
intervention and comparison group results. Inferential techniques include
chisquare test of significance for contingency tables (e.g. testing
whether the intervention and comparison groups differ significantly in
their frequency of categorical responses such as ‘yes’ versus ‘no’),
ttest comparison of means, and loglinear regression models. The
loglinear regression models generally take the form:
ln(CHANGE + k) =$ + X 'â +# ln(BASELINE + k) +"BI +! , (1)
where,
ln(CHANGE k) + is the natural logarithm of the postpre
difference (CHANGE) in the outcome measure for respondent i,
transformed by k units in order to avoid zero and negative values of
CHANGE.
i
X is a matrix of independent variables including for ethnicity,
language spoken at home, age of first computer contact, and computer
engagement and computer course plans as reported at baseline,
ln (BASELINE k)
+ is the natural logarithm of individual i'’s baseline score for the CHANGE
variable, scaled by k units—this controls for potential ceiling and basal
effects in change due to starting scores—were appropriate we replace the log of the baseline with
categorical baseline scores, and
iBI is an indicator that equals 1 if individual i is a Build IT participant, and zero otherwise.
The use of logtransformed models permits the interpretation of model coefficients as percent change in
the outcome measure given the independent variable. Put differently, we could say that, after
considering baseline scores and other salient characteristics, participation in Build IT will
increase the CHANGE+k outcome by 100*?%. Large positive values of ? indicate a positive
influence of Build IT on technologyrelated development, after accounting for differences in
starting scores.
In addition to these group comparisons, we also assess the whether there were significant
differences between the Build IT participants who completed the entire program and those who
left early. We also examine whether there are potential differences between the EXPLORE and Muir
Build IT sites. All quantitative data was analyzed using the R statistical software. All interview
data was coded and analyzed by HTA using AtlasTi qualitative software. The emerging themes and
concepts from the qualitative data were used to develop a theoretical framework for understanding
ALL STARS’ capacity with respect to Build IT implementation. We discuss the results in terms of
this framework.


