Abstract
Citation
Benjamin SE, Neelon B, Ball SC, Bangdiwala SI, Ammerman AS, Ward DS. Reliability and validity of a nutrition and physical activity environmental self-assessment for child care. Int J Behav Nutr Phys Act 2007 Jul 5;4:29.
Abstract
BACKGROUND: Few assessment instruments have examined the nutrition and physical activity environments in child care, and none are self-administered. Given the emerging focus on child care settings as a target for intervention, a valid and reliable measure of the nutrition and physical activity environment is needed. METHODS: To measure inter-rater reliability, 59 child care center directors and 109 staff completed the self-assessment concurrently, but independently. Three weeks later, a repeat self-assessment was completed by a sub-sample of 38 directors to assess test-retest reliability. To assess criterion validity, a researcher-administered environmental assessment was conducted at 69 centers and was compared to a self-assessment completed by the director. A weighted kappa test statistic and percent agreement were calculated to assess agreement for each question on the self-assessment. RESULTS: For inter-rater reliability, kappa statistics ranged from 0.20 to 1.00 across all questions. Test-retest reliability of the self-assessment yielded kappa statistics that ranged from 0.07 to 1.00. The inter-quartile kappa statistic ranges for inter-rater and test-retest reliability were 0.45 to 0.63 and 0.27 to 0.45, respectively. When percent agreement was calculated, questions ranged from 52.6% to 100% for inter-rater reliability and 34.3% to 100% for test-retest reliability. Kappa statistics for validity ranged from -0.01 to 0.79, with an inter-quartile range of 0.08 to 0.34. Percent agreement for validity ranged from 12.9% to 93.7%. CONCLUSION: This study provides estimates of criterion validity, inter-rater reliability and test-retest reliability for an environmental nutrition and physical activity self-assessment instrument for child care. Results indicate that the self-assessment is a stable and reasonably accurate instrument for use with child care interventions. We therefore recommend the Nutrition and Physical Activity Self-Assessment for Child Care (NAP SACC) instrument to researchers and practitioners interested in conducting healthy weight intervention in child care. However, a more robust, less subjective measure would be more appropriate for researchers seeking an outcome measure to assess intervention impact.
Full Text
The full text is available at https://dx.doi.org/10.1186/1479-5868-4-29
At A Glance
Food Environment Variables
# | Type of Environment/Institution |
---|---|
69 | Total Environments/Locations |
69 | Child Care (Daycare, Pre-school) |
Measure | objective | perceived |
---|---|---|
Availability/Access | ✘ | ✔ |
Food Quality | ✘ | ✔ |
Policy/Practice | ✘ | ✔ |
Provision of nutrition education; Supportive healthy eating practices | ✘ | ✔ |
Food Group/Type of Food |
---|
Fruits and vegetables |
Low-fat dairy |
Whole grains |
Foods of minimal nutritional value |
Sweetened beverages |
Fried foods and high-fat meats |
Physical Activity Environment Variables
# | Type of Environment/Location |
---|---|
69 | Total Environments/Locations |
69 | Child Care |
# | Scale |
---|---|
- | Building |
Measure | objective | perceived |
---|---|---|
Policy | ✘ | ✔ |
Crime/Safety | ✘ | ✔ |
Facility Adequacy/Appeal or Quality | ✘ | ✔ |
Aesthetics/Beautification | ✘ | ✔ |
Domain(s)
Food Environment
Physical Activity Environment
Measure Type
Questionnaire
Measure Availability
Measure included in article
Download measure from riskfactor.cancer.gov
Number of Items
56 Reported
Study location
NC, USA
Child care centers across North Carolina
Languages
Not applicable
Information about Development of Measure
Nothing to add
Study Design
Study Participants
Age
2 - 5 Years
Sex
Not reported
Race/Ethnicity
Hispanic
White
Black/African American
Asian
American Indian/Alaska Native
Predominantly Low-income/Low-SES
Not reported
Sample Size
69
Study Design
Design Type
Validation/Reliability
Health Outcomes Assessed
None
Obesity Measures
Not applicable
BMI Measured or Self-reported
Not applicable
Covariates
Not reported
Data Reported on Race/Ethnicity
Quantitative data for community or area
Data Reported on SES
Not applicable
SES-related Variables
Not applicable
How To Use
Administration
Who Administered
Self-administered
How Administered
Email/postal mail
Time Required
Not reported
Training Required
Not reported
Instructions on Use
Not reported
Data Analysis
Data Collection/Analysis Costs
Not available
Data Collection/Protocol
Not available
Instructions on Data Analysis
Not reported
Validity (7)
Type of validity | Construct/subscale assessed | Criterion measure used | Test/statistic used | Result |
---|---|---|---|---|
Face | See paper | Face validity established | ||
Content | See paper | Content validity established | ||
Construct | NAP SACC | National expert review | Found to be accurate and comprehensive | |
Criterion | Nutrition construct (30 of 38 questions) | Environmental and Policy Assessment and Observation (EPAO) system | Percent Agreement | 12.9 to 93.7 |
Criterion | Physical Activity construct (14 of 18 questions) | Environmental and Policy Assessment and Observation (EPAO) system | Percent Agreement | 0.0 to 90.6 |
Criterion | Nutrition construct (30 of 38 questions) | Environmental and Policy Assessment and Observation (EPAO) system | Kappa statistic | -0.01 to 0.78 |
Criterion | Physical Activity construct (14 of 18 questions) | Environmental and Policy Assessment and Observation (EPAO) system | Kappa statistic | 0.00 to 0.77 |
Reliability (8)
Type of reliability | Construct/subscale assessed | Test/statistic used | Result |
---|---|---|---|
Inter-rater | Physical Activity construct | Kappa statistic | 0.44 to 0.85 |
Inter-rater | Nutrition construct | Kappa statistic | 0.20 to 1.0 |
Inter-rater | Nutrition construct | Percent agreement | 52.6 to 100.0 |
Inter-rater | Physical Activity construct | Percent agreement | 52.6 to 92.1 |
Test-retest | Physical Activity construct | Percent agreement | 37.1 to 87.4 |
Test-retest | Nutrition construct | Kappa statistic | 0.07 to 1.00 |
Test-retest | Physical Activity construct | Kappa statistic | 0.17 to 0.70 |
Test-retest | Nutrition construct | Percent agreement | 34.3 to 100.0 |