Skip to main content
SpringerLink
Log in

Search-Based Testing for Scratch Programs

  • Conference paper
  • First Online:
Search-Based Software Engineering (SSBSE 2020)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 12420))

Included in the following conference series:

Abstract

Block-based programming languages enable young learners to quickly implement fun programs and games. The Scratch programming environment is particularly successful at this, with more than 50 million registered users at the time of this writing. Although Scratch simplifies creating syntactically correct programs, learners and educators nevertheless frequently require feedback and support. Dynamic program analysis could enable automation of this support, but the test suites necessary for dynamic analysis do not usually exist for Scratch programs. It is, however, possible to cast test generation for Scratch as a search problem. In this paper, we introduce an approach for automatically generating test suites for Scratch programs using grammatical evolution. The use of grammatical evolution clearly separates the search encoding from framework-specific implementation details, and allows us to use advanced test acceleration techniques. We implemented our approach as an extension of the Whisker test framework. Evaluation on sample Scratch programs demonstrates the potential of the approach.

Authors listed in alphabetical order.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://scratch.mit.edu/statistics/, last accessed 9.6.2020.

  2. 2.

    https://pptr.dev/, last accessed 9.6.2020.

References

  1. Aivaloglou, E., Hermans, F.: How kids code and how we know: an exploratory study on the scratch repository. In: Proceedings of ICER, pp. 53–61 (2016)

    Google Scholar 

  2. Anjum, M.S., Ryan, C.: Seeding grammars in grammatical evolution to improve search based software testing. In: Hu, T., Lourenço, N., Medvet, E., Divina, F. (eds.) EuroGP 2020. LNCS, vol. 12101, pp. 18–34. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-44094-7_2

    Chapter  Google Scholar 

  3. Arcuri, A.: It really does matter how you normalize the branch distance in search-based software testing. Softw. Test. Verif. Reliab. 23(2), 119–147 (2013)

    Article  Google Scholar 

  4. Boe, B., Hill, C., Len, M., Dreschler, G., Conrad, P., Franklin, D.: Hairball: lint-inspired static analysis of scratch projects. In: Proceedings of SIGCSE, pp. 215–220 (2013)

    Google Scholar 

  5. Chang, Z., Sun, Y., Wu, T.Y., Guizani, M.: Scratch analysis tool (SAT): a modern scratch project analysis tool based on ANTLR to assess computational thinking skills. In: Proceedings of IWCMC, pp. 950–955. IEEE (2018)

    Google Scholar 

  6. Fraser, G., Arcuri, A.: Whole test suite generation. IEEE Trans. Softw. Eng. 39(2), 276–291 (2012)

    Article  Google Scholar 

  7. Fraser, G., Arcuri, A.: Handling test length bloat. Softw. Test. Verif. Reliab. 23(7), 553–582 (2013)

    Article  Google Scholar 

  8. Frädrich, C., Obermüller, F., Körber, N., Heuer, U., Fraser, G.: Common bugs in scratch programs. In: Proceedings of ITICSE, pp. 89–95 (2020)

    Google Scholar 

  9. Gross, F., Fraser, G., Zeller, A.: Search-based system testing: high coverage, no false alarms. In: Proceedings of ISSTA, pp. 67–77 (2012)

    Google Scholar 

  10. Hermans, F., Aivaloglou, E.: Do code smells hamper novice programming? A controlled experiment on scratch programs. In: Proceedings of ICPC, pp. 1–10. IEEE (2016)

    Google Scholar 

  11. Hermans, F., Stolee, K.T., Hoepelman, D.: Smells in block-based programming languages. In: Proceedings of VL/HCC, pp. 68–72. IEEE (2016)

    Google Scholar 

  12. Johnson, D.E.: Itch: individual testing of computer homework for scratch assignments. In: Proceedings of SIGCSE, pp. 223–227 (2016)

    Google Scholar 

  13. Korel, B.: Automated software test data generation. IEEE Trans. Softw. Eng. (TSE) 16, 870–879 (1990)

    Article  Google Scholar 

  14. Mahmood, R., Mirzaei, N., Malek, S.: Evodroid: segmented evolutionary testing of android apps. In: Proceedings of ESEC/FSE, pp. 599–609 (2014)

    Google Scholar 

  15. Maloney, J., Resnick, M., Rusk, N., Silverman, B., Eastmond, E.: The scratch programming language and environment. TOCE 10(4), 1–15 (2010)

    Article  Google Scholar 

  16. Mao, K., Harman, M., Jia, Y.: Sapienz: multi-objective automated testing for android applications. In: Proceedings of ISSTA, pp. 94–105 (2016)

    Google Scholar 

  17. Meerbaum-Salant, O., Armoni, M., Ben-Ari, M.: Habits of programming in scratch. In: Proceedings of ITICSE, pp. 168–172 (2011)

    Google Scholar 

  18. Moreno-León, J., Robles, G.: Dr. scratch: a web tool to automatically evaluate scratch projects. In: Proceedings of WIPSCE, pp. 132–133 (2015)

    Google Scholar 

  19. O’Neill, M., Ryan, C.: Grammatical evolution. IEEE Trans. Evol. Comput. 5(4), 349–358 (2001)

    Article  Google Scholar 

  20. Panichella, A., Kifetew, F.M., Tonella, P.: Reformulating branch coverage as a many-objective optimization problem. In: Proceedings of ICST, pp. 1–10 (2015)

    Google Scholar 

  21. Price, T.W., Dong, Y., Lipovac, D.: iSnap: towards intelligent tutoring in novice programming environments. In: Proceedings of SIGCSE, pp. 483–488 (2017)

    Google Scholar 

  22. Stahlbauer, A., Kreis, M., Fraser, G.: Testing scratch programs automatically. In: Proceedings of ESEC/SIGSOFT FSE, pp. 165–175. ACM (2019)

    Google Scholar 

  23. Techapalokul, P., Tilevich, E.: Quality hound-an online code smell analyzer for scratch programs. In: Proceedings of VL/HCC, pp. 337–338. IEEE (2017)

    Google Scholar 

  24. Techapalokul, P., Tilevich, E.: Understanding recurring quality problems and their impact on code sharing in block-based software. In: Proceedings of VL/HCC, pp. 43–51. IEEE (2017)

    Google Scholar 

  25. Techapalokul, P., Tilevich, E.: Code quality improvement for all: Automated refactoring for scratch. In: Proceedings of VL/HCC, pp. 117–125. IEEE (2019)

    Google Scholar 

  26. Wegener, J., Baresel, A., Sthamer, H.: Evolutionary test environment for automatic structural testing. Inf. Softw. Tech. 43(14), 841–854 (2001)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by EPSRC project EP/N023978/2 and DFG project FR 2955/3-1 “TENDER-BLOCK: Testing, Debugging, and Repairing Blocks-based Programs”.

Author information

Authors and Affiliations

  1. University of Passau, Innstr. 33, 94032, Passau, Germany

    Adina Deiner, Christoph Frädrich, Gordon Fraser, Sophia Geserer & Niklas Zantner

Authors
  1. Adina Deiner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christoph Frädrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gordon Fraser
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sophia Geserer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Niklas Zantner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gordon Fraser .

Editor information

Editors and Affiliations

  1. Monash University, Melbourne, VIC, Australia

    Aldeida Aleti

  2. Delft University of Technology, Delft, The Netherlands

    Annibale Panichella

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Deiner, A., Frädrich, C., Fraser, G., Geserer, S., Zantner, N. (2020). Search-Based Testing for Scratch Programs. In: Aleti, A., Panichella, A. (eds) Search-Based Software Engineering. SSBSE 2020. Lecture Notes in Computer Science(), vol 12420. Springer, Cham. https://doi.org/10.1007/978-3-030-59762-7_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59762-7_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59761-0

  • Online ISBN: 978-3-030-59762-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation