Turkish version of the SPAN questionnaire for high-school students: reproducibility and validity

PDF version

F. Öz ¹, S. Metintas¹, R. Aydın¹ and Ö. Özay ¹

النسخة التركية لاستبيان النشاط البدني المدرسي والتغذية الخاص بطلاب المدارس الثانوية: التناتُج والمصدوقية

فاتح أوز، سلمى متين طاش، رشاد آيدن، أوزكان أوزاي

الخلاصة: هدفت هذه الدراسة إلى إيجاد نسخة تركية لاستبيان النشاط البدني المدرسي والتغذية (2009) الخاص بطلاب المدارس الثانوية وإلى اختبار مصدوقيته وتناتجه. فتُرجم الاستبيان ثم تُرجم عكسياً وقيِّمت مصدوقية اللغة. واختُبر وأُعيد اختباره على 318 طالباً من طلاب المدارس الثانوية لتقييم مصدوقيته وتناتجه. وقد حُدِّدت مصدوقيته بمقارنة أجزاء الاستبيان المتعلقة بما أُكِل في اليوم السابق مع ما هو موجود في قوائم استهلاك الغذاء على مدار 24 ساعة. ثم أجري تحليل إحصائي باستخدام تحليل ارتباط سبيرمان وإحصائيات كابا والنسبة المئوية للتوافق. فكانت أعلى نتائج الاختبار وإعادة الاختبار للأغذية المستهلكة في اليوم السابق، حيث كانت النسبة المئوية للتوافق تتراوح ما بين %56 و%96، وكابا تتراوح ما بين 0.43 و0.95 وتراوح التوافق على البنود المتعلقة بالنشاط البدني ما بين %38 و%88، وتراوحت كابا ما بين 0.24 و0.82 إن تناتج ومصدوقية النسخة التركية لهذا الاستبيان مقبولة، وهي أداة سريعة سهلة الاستخدام لتقييم التغذية والنشاط البدني لدى طلاب المدارس.

ABSTRACT This study aimed to generate a Turkish version of the School Physical Activity and Nutrition (SPAN) (2009) questionnaire for high-school students and to test its validity and reproducibility. The questionnaire was translated and back translated and the language validity was assessed. It was tested and re-tested on 318 high-school students to evaluate its validity and reproducibility. Validity was determined by comparing the parts of the questionnaire about what was eaten in the previous day with that of 24-hour food consumption lists. Statistical analysis was performed by using Spearman correlation analysis, kappa statistics and percentage of agreement. The highest test−re-test results were on foods consumed the day before, with an agreement percentage range of 56%−96% and a kappa range of 0.43−0.95. The items related to physical activity had an agreement range of 38%−88% and a kappa range of 0.24−0.82. The reproducibility and validity of the Turkish version of SPAN questionnaire is acceptable and it is a rapid, easily used tool for the assessment of nutrition and physical activity in schoolchildren.

Version turque du questionnaire SPAN pour les élèves du secondaire : reproductibilité et validité

RÉSUMÉ La présente étude avait pour objectif de générer une version turque du questionnaire SPAN (School Physical Activity and Nutrition) [activité physique et nutrition en milieu scolaire  (2009) pour les élèves du secondaire et de tester sa validité et sa reproductibilité. Le questionnaire a été traduit puis a fait l’objet d’une rétro-traduction, et la validité de la langue a été évaluée. Il a été testé puis re-testé sur 318 élèves du secondaire afin d’évaluer sa validité et sa reproductibilité. La validité a été déterminée en comparant les parties du questionnaire sur les aliments consommés le jour d’avant avec celle comprenant des listes relatives à la consommation alimentaire sur une durée de 24h. L’analyse statistique a été réalisée à l’aide de l’analyse de corrélation de Spearman, du test du kappa et du pourcentage de concordance. Les résultats test-retest les plus élevés concernaient les aliments consommés le jour d’avant, avec un pourcentage de concordance compris entre 56 % et 96 % et un coefficient kappa allant de 0,43 à 0,95. Les items liés à l’activité physique avaient un pourcentage de concordance de 38 à 88 % et un coefficient kappa de 0,24 à 0,82. La reproductibilité et la validité de la version turque du questionnaire SPAN est acceptable et constitue un outil rapide et simple d’utilisation pour évaluer la nutrition et l’activité physique chez les enfants scolarisés.

¹Department of Public Health, Eskisehir Osmangazi University Medicine School, Eskişehir, Turkey (Correspondence to: F. Öz: This email address is being protected from spambots. You need JavaScript enabled to view it.).

Received: 05/06/14; accepted: 18/07/16

Introduction

Obesity in adolescents is a major problem in developed as well as in developing countries (1). According to the results of the HBSC (Health Behavıour in School-Aged Chıldren) survey conducted in Turkey, 14% of the males and 5% of the females at age of 15 years in 2001-2014) (5). It was stated in the action plan that interventions to address adolescent obesity require interactive educational and behavioural support regarding healthy eating and physical activity. The interventions themselves, as well as the measurability and ability to monitor the interventions, have an important role in the programme. The ability to measure and monitor the interventions allows the assessment of the programme to be assessed. Therefore, surveillance tools should be available to monitor the eating habits and physical activity levels of the individuals.

School Physical Activity and Nutrition (SPAN-2009) questionnaire was developed by the School-Based Nutrition Monitoring (SBNM) project funded by the Centers for Disease Control and Prevention (CDC) and United States Department of Agriculture (USDA) and it is a well-accepted surveillance tools. There are two forms of SPAN questionnaire developed for using on primary and high school students (6).

This study aimed to generate a Turkish version of the SPAN (2009) questionnaire for high-school students and to test its validity and reproducibility.

Methods

Translation of the SPAN questionnaire

Permission from Deanna M. Hoelscher was received to generate the Turkish version of SPAN-questionnaire. The questionnaire was translated and back translated by English to Turkish and Turkish to English linguists and the language validity of the questionnaire was assessed. Taking account of the sociocultural norms in Turkey and some other factors (e.g. religious, economic, climatic, access to enough food), a committee of three experienced individuals (one epidemiologist, one medical doctor, one linguist) made some changes in the questionnaires in order to assure the cultural adaptation. For this purpose, the terms “hot dog and pig bacon” in 10th item were replaced by “beef bacon and meat balls”, the term “hot or cold cereal” in 33rd item was replaced by “corn flakes” and the “frozen yoghurt-ice cream” was replaced by “cold sweet”. Items 39 (Last week, were the following available in your home?), 46 (How often does your family buy or get fruits/vegetables from…) and 50 [Do you use food labels (nutrition facts) to make your food choices?] in the SPAN survey were not included in the Turkish version. These items were removed because these habits were only likely to apply to a small minority with high incomes in Turkey. Items 58 and 59 were integrated and transformed into one question as “How many teams did you play in during last year?” Finally, the items between 65 and 69 (questions about attitudes of parents towards their children) and between 81 and 91 (questions about psychological situation) were also excluded from the Turkish version of the questionnaire.

The newly created Turkish version of the SPAN Questionnaire consisted of 15 questions on sociodemographic characteristics and 61 questions in six sub-sections (“Did You Consume These Foods Yesterday?”, “What foods/meals do you usually consume?”, “Nutrition knowledge questions”, “Nutrition attitude questions”, “Weight behaviour questions” and “Physical activity questions”). In total the survey consisted of 76 questions.

Testing the questionnaire

The sample size for testing the questionnaire was calculated to be 386 with a margin of error of 5% and considering the reliability of questionnaire as 50%. The questionnaire was tested among 370 students out of 419 studying in 1st or 2nd class of 4 high schools in the region of Sivrihisar located between the capital city Ankara and the city of Eskisehir in Central Anatolia Region. The 4 schools were elected by simple random sampling from the 8 regional high schools with different educational programmes.

The questionnaire was completed by the students themselves under supervision. The questionnaire was applied during the school hours in the morning from Tuesday to Friday because eating habits may change in the weekend. The re-test of “Did You Consume These Foods Yesterday?” sub-section was applied on the same day during the school hours in afternoon, while the re-test of the other sub-sections was applied 7 days after during the morning school hours.

A total of 52 students (14%) who were not at school at the time of study or who gave incomplete or false information or who did not complete the re-test were excluded from the study. The students excluded from the study were those not completing more than 10% of the questionnaire and those who displayed deficient attention during the 20 minutes required to fill the questionnaire.

The 24-hour food consumption lists were obtained following the 2-hour re-test. Researchers followed a standard dialogue with the students and recorded foods on the 24-hour food consumption lists. These lists were entered in the computer. The list of the foods eaten the previous day was adapted from the foods included in the SPAN food consumption questionnaire. The students reporting calorie intake of less than 500 kcal or more than 5000 kcal in 24 hours were also excluded since these were highly improbable intakes and were likely incorrect. Students’ self-reported height and weight were recorded. Body Mass Index (BMI) was calculated and reference norms published by WHO were used (7,8).

Ethical clearance

Eskisehir Osmangazi University Medical Faculty Non Drug Clinical Research Ethical Committee approved the study (ethical committee report: 80558721/244). All necessary permissions from the institutions and oral informed consent from all participants was obtained.

Data analysis

Data were analysed with SPSS, version 15.0. There is no consensus in the literature about the best method of statistical analysis to assess the reliability and validity of estimating if the nutrition is good or bad. Spearman correlation coefficient, kappa coefficient and the percentage of agreement were used because of nonparametric data (9).

The analysis was performed in two-steps. In the first step assessing the validity, the 24-hour food consumption lists were compared with the list in the SPAN questionnaire on the foods consumed the previous day. In the second step, the “Did You Consume These Foods Yesterday?” sub-section of the questionnaire was assessed by calculating the internal consistency, Cronbach α and intraclass correlation coefficient. The Cronbach α of 0.40−0.60, 0.60−0.80 and > 0.80 indicates low, moderate or high reliability for the questionnaire respectively.

Because the other sub-sections consisted of very heterogeneous answers, the Cronbach α test was not performed. Spearman correlation coefficient, kappa coefficient and the percentage of agreement were used for reliability for these sub-sections.

Spearman correlation coefficient, kappa coefficient and the percentage of agreement were used to assess the 24-hour food consumption list and test−re-test agreement for each item.

Overall, the level of compliance indicated by the Spearman correlation coefficient ranges as follows: 0.80 strong relationship. The kappa statistic is used to examine the agreement between the test and re-test scores for each item. This statistical method yields reliable statistical data in terms of eliminating the effect of chance. The level of compliance indicated by kappa coefficient ranges as follows: < 0.20 inadequate; 0.21−0.40 weak; 0.41−0.60 moderate; 0.61−0.80 strong; and 0.81−1.00 excellent compliance. The kappa statistic and adjusted kappa statistic, adjusted for the extraordinary variables, were determined using pre-prepared tables at: http://vassarstats.net/kappa.html.

Agreement percentage was calculated as the percentage giving the same answer to the same question both in the test and re-test (10).

Results

The study group consisted of 318 students in the age group of 14−15 years, with a mean age and standard deviation (SD) of 14.7 (SD 0.5) years. There were slightly more girls [168 (52.8%)] than boys. Based on BMI, 55 (17.2%) students were overweight or obese.

The Cronbach α intraclass correlation coefficient was 0.71 for the question of “Did you consume these foods yesterday?”

The reproducibility with the question “Did you consume these foods yesterday?” and validity analysis with the 24-hour food consumption lists are shown in Table 1.

The highest results in the test−re-test comparison was in the part including questions of “Did you consume these foods yesterday?” The percentage of agreement and kappa coefficient for this part ranged from 61% to 93% and from 0.46 to 0.91 respectively (Table 1). The items with the highest agreement percentage and kappa coefficient were the eating “fried chicken-fish” and “dining out”, although the latter did not have a high kappa value, while those with lowest were eating a “snack” and “consuming chocolate”. The Spearman correlation coefficient ranged from 0.40 to 0.84, the highest being for the item “nuts” (0.84) and lowest for the item “orange-coloured vegetables” (0.40).

For the questions related to the consumption of fruits and vegetables, the Spearman correlation coefficient showed moderate reliability, while agreement percentages and kappa coefficients showed high reliability.

Because mean correlation did not differ between females and males, the results shown in tables were given for the whole study sample.

The validity assessment of the questionnaire was performed only for the part “Did you consume these foods yesterday?” (Table 1). For this purpose, the 24-hour food consumption lists converted into comparable data with SPAN results. The agreement percentage and kappa statistic ranged from 56% to 98% and from 0.43 to 0.95 respectively. The first five items with highest values in both analyses included “fruit juice”, “dining out” and “cornflakes consumption” in common and those with lowest values included “snacks”, “rice-pasta consumption” and “tea-coffee consumption” in common. The Spearman correlation coefficient ranged from 0.11 to 0.89, being highest for the item “fruit juice” (0.89) and lowest for the item “snacks” (0.11).

The reproducibility analysis for the items related to usual food consumption and the nutritional knowledge and attitudes is show in Table 2.

The agreement percentage and kappa coefficient for the items of “What foods/meals do you usually consume?” ranged from 53% to 89% and from 0.39 to 0.84 respectively. The highest and lowest values common in both analysis were for “vitamin-mineral intake” and “type of milk”, respectively. The Spearman correlation coefficient ranged between 0.25 and 0.52.

The part of questionnaire about nutritional knowledge and attitudes had the lowest agreement percentage, kappa coefficient and Spearman correlation coefficient, with ranging from 46% to 73%, from 0.33 to 0.59 and from 0.19 to 0.33 respectively (Table 2). Among these items, the highest result was obtained for “If I am overweight I am more likely to have more health problems like cancer or heart disease”, while the lowest result was for “How many total cups of fruits should you eat each day?”

The agreement percentage, kappa coefficient and Spearman correlation coefficient for the items about nutritional attitudes ranged from 46% to 55%, from 0.29 to 0.41 and from 0.25 to 0.48 respectively. The rank of the items was similar in all three analyses.

The agreement percentage, kappa coefficient and Spearman correlation coefficient for the weight behaviour items ranged from 66% to 75%, from 0.41 to 0.63 and from 0.34 to 0.48 respectively. The rank of the items was also similar in all three analyses.

The agreement percentage, kappa coefficient and Spearman correlation coefficient for items about physical activity ranged from 38% to 88%, from 0.24 to 0.82 and from 0.33 to 0.86 respectively. The scores on the items related to exercises were better than those related to the inactive time spent.

No significant gender and BMI differences were detected for any survey questions in terms of reproducibility and validity.

Discussion

Reliability is defined as the extent to which a measurement tool measures the intended feature and the ability to produce consistent results. There are two main criteria for the assessment of the reliability of a tool. The first one is the consistency between the responses obtained at the same time. The reliability coefficient, Cronbach α is used for this purpose. The higher the internal consistency coefficient, the higher consistency between the items. The second criterion is the consistency between the results obtained at different time points, which is assessed by the test−re-test method. The test−re-test reliability analysis assesses the tool in terms of consistency of results on repeated uses and invariability with the time. For this purpose, the questionnaire is re-applied some time after the period of initial application (11).

Overall, the test−re-test reliability for the question of “Did you consume these foods yesterday?” was found to be good. The Kappa and Spearman correlation coefficients were higher in less frequently consumed foods. Because it is possible that the frequently consumed foods could not be stated as the exact number of daily consumption, the reliability might be underestimated. Furthermore, the low reliability of the items related to vegetables might be due to the difficulty in remembering the vegetables in mixed foods. Hoelscher et al. have also reported low reliability for frequently consumed foods and high reliability for less frequently consumed foods (12). Thiagarajah and colleagues have also reported similar results (13).

The results of present study revealed no difference in reproducibility between the males and females. There are other previous studies reporting no difference in reproducibility and validity of the questions about food consumption frequency in terms of gender (12,13).

The validity of the items within “Did you consume these foods yesterday?” assessed by the 24-hour nutritional consumption list was found to be good overall. The low validity for the items “snacks” and “tea-coffee consumption” might have resulted from difficulty in remembering how many times these foods are eaten/drunk due to their frequent consumption. Furthermore, snacks might not be considered as a food by some individuals. Thiagarajah and colleagues also reported low validity for the frequently consumed foods (13).

In Turkey, there are a limited number of validity and reliability studies for the nutrition in adolescents. The study conducted by Demirezen et al. used a 5-point Likert scale including the statements ranging from “never” to “always” for the assessment of food consumption (14). Arikan et al. used the translated scale of the “Adolescent Food Habit Checklist”, which assessed the amount and frequency of the consumption of foods (15). On the other hand, because the translated SPAN questionnaire has a Cronbach α coefficient of 0.71 with an acceptable reliability and because the part of the questionnaire assessing foods eaten the previous day has the properties of other existing scales, the translated SPAN questionnaire can be considered appropriate for using in epidemiological studies on the nutritional habits of adolescents in Turkey. Furthermore, the SPAN questionnaire has been reported to be a rapid and easily used measurement tool for the assessment of nutrition in school classes (12,13,16).

The lowest reliability was found for the item about the “type of milk” among the items of “What foods/meals do you usually consume?” According to the data from National Dairy Council, 45% of the milk produced in Turkey is raw milk, compared to under 1% in the United States of America (17). As a result, the reproducibility for the “type of milk consumed” might be underestimated due to the fact that many individuals were consuming raw milk. So raw milk could be added to the Turkish version of SPAN questionnaire. Furthermore, the low reliability might be due to the fact that milk is usually bought by the parents and that children are not usually concerned about the type of milk bought or consumed. Thiagarajah and colleagues also reported that students had difficulty in remembering what kind of milk they were consuming (13).

In the present study, the part including items about nutritional knowledge and attitudes had the lowest reproducibility. It might be that the students were interested in the questions about nutritional knowledge and tried to learn the correct answers, which may have led to the inconsistency in the reproducibility of items about nutritional knowledge. Furthermore the items about attitudes included no specified time interval, such as asking about the foods eaten the previous day, so students may have had difficulty in remembering these foods, resulting in the underestimation of reliability. Hoelscher et al. also reported similar results in their study (12).

Although the reproducibility of the weight behaviour items and physical activity items was at a moderate level, it was better than that of nutritional knowledge and attitudes items. The low reproducibility found for the physical activity items assessing the inactive time spent might be a result the students having difficulty in remembering the time they have spent inactively. Hoelscher et al. reported an acceptable level of reliability for the items about physical activity (12).

There are a few limitations with regard to the SPAN questionnaire and our Turkish version. Because the part of the SPAN questionnaire assessing the eating habits in the previous day measures just one day, it does not represent the overall food consumption. Furthermore, the questionnaire is appropriate for the evaluation of groups rather than individuals. Finally, because the Turkish version was used on students in a limited region, the findings may not represent the whole geographical region of Turkey and needs to be tested on larger and different samples.

Conclusions

The Turkish SPAN questionnaire was found to be a rapid, easily used and reproducible tool for the assessment of nutrition and physical activity in schools. However, it should be used on larger and different study samples repeatedly in order to obtain additional data about the reliability and validity of the questionnaire.

Funding: None.

Competing interests: None declared.

References

1. Population-based approaches to childhood obesity prevention. Geneva: World Health Organization; 2012 (http://apps.who.int/iris/bitstream/10665/80149/1/9789241504782_eng.pdf?ua=1, accessed 29 September 2016).
2. Inequalities in young people's health. HBSC international report from the 2005/2006 survey. Copenhagen: WHO Regional Office for Europe; 2008 (http://www.euro.who.int/__data/assets/pdf_file/0005/53852/E91416.pdf, accessed 29 September 2016).
3. Social determinants of health and well-being among young people : Health Behaviour in School-Aged Children (HBSC) study : international report from the 2009/2010 survey. Copenhagen: WHO Regional Office for Europe; 2012 (http://www.euro.who.int/__data/assets/pdf_file/0003/163857/Social-determinants-of-health-and-well-being-among-young-people.pdf?ua=1, http://www.euro.who.int/__data/assets/pdf_file/0003/163857/Social-determinants-of-health-and-well-being-among-young-people.pdf?ua=1).
4. Reed DB, Patterson PJ, Wasserman N. Obesity in rural youth: looking beyond nutrition and physical activity. J Nutr Educ Behav. 2011 Sep-Oct;43(5):401–8. PMID:21906552
5. Türkiye Cumhuriyeti Sağlık Bakanlığı sağlıklı beslenme ve hareket programı [Healthy eating and movement life programme]. Ankara, Ministry of Health, 2013 (http://www.beslenme.gov.tr/content/files/home/turkiye_saglikli_beslenme_ve_hareketli_hayat_programi.pdf, accessed 10 October 2016).
6. School Physical Activity and Nutrition (SPAN) Project: questionnaire. (https://sph.uth.edu/dotAsset/4d09697e-e692-4e66-816e-d9d6ccb35499.pdf, accessed 10 October 2016).
7. World Health Organization. BMI-for-age. Boys: 5-World Health Organization. BMI-for-age. Girls: 5-19 years(z-scores) (http://www.who.int/growthref/bmifa_girls_5_19years_per.pdf?ua=1 accessed 10 October 2016)
8. Burema J, Van Staveren WA, Feunekes GIJ. Guidelines for reports on validation studies. [Letter to the Editor]. Eur J Clin Nutr. 1995 Dec;49(12):932–3. PMID:8925796
9. Şencan H. Sosyal ve davranışsal ölçümlerde güvenilirlik ve geçerlilik [Reliability and validity in the social and behavioral measures]. Ankara: Seckin Yayıncılık Sanayi ve Ticaret AŞ; 2005:499–559.
10. Norman GR, Streiner DL. Biostatistics: the bare essentials. St. Louis (MO): Mosby; 1994
11. Hoelscher DM, Day RS, Kelder SH, Ward JL. Reproducibility and validity of the secondary level School-Based Nutrition Monitoring student questionnaire. J Am Diet Assoc. 2003 Feb;103(2):186–94. PMID:12589324
12. Thiagarajah K, Fly AD, Hoelscher DM, Bai Y, Lo K, Leone A, et al. Validating the food behavior questions from the elementary school SPAN questionnaire. J Nutr Educ Behav. 2008 Sep-Oct;40(5):305–10. PMID:18725149
13. Demirezen E, Coşansu G. Adölesan Çağı Çocuklarda Beslenme Alışkanlıklarının Değerlendirilmesi [Evaluating dietary patterns in adolescence]. Sted. 2005;14(8):174–8.
14. Arıkan İ, Aksu AE, Metintaş S, Kalyoncu C. Adölesan Beslenme Alışkanlıkları Kontrol Listesi’nin Türk Adölesanlarına Uyarlanması [Adaptation of the adolescent food habit checklist to Turkish adolescents. TAF Prev Med Bull. 2012;11(1):49–56.
15. Penkilo M, George GC, Hoelscher DM. Reproducibility of the School-Based Nutrition Monitoring Questionnaire among fourth-grade students in Texas. J Nutr Educ Behav. 2008 Jan-Feb;40(1):20–7. PMID:18174100
16. [National Dairy Council. 2012 world milk production data]. (http://www.ulusalsutkonseyi.org.tr/ifcn-2012-yili-ozet-raporu/, accessed 10 October, 2016).
17. [National Dairy Council. 2012 world milk production data]. (http://www.ulusalsutkonseyi.org.tr/ifcn-2012-yili-ozet-raporu/, accessed 10 October, 2016).