RESEARCH ARTICLE


The Effectiveness of Learning Implementation Plan Tool Through Design-Based Research



Muhammad Zaini*
Lambung Mangkurat University, Banjarmasin South Kalimantan, Banjarmasin, Indonesia


Article Metrics

CrossRef Citations:
0
Total Statistics:

Full-Text HTML Views: 399
Abstract HTML Views: 183
PDF Downloads: 118
ePub Downloads: 90
Total Views/Downloads: 790
Unique Statistics:

Full-Text HTML Views: 157
Abstract HTML Views: 118
PDF Downloads: 89
ePub Downloads: 70
Total Views/Downloads: 434



© 2018 Muhammad Zaini.

open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: (https://creativecommons.org/licenses/by/4.0/legalcode). This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

* Address correspondence to this author at the Lambung Mangkurat University,Banjarmasin South Kalimantan, Banjarmasin, Indonesia; Tel: +62 85102119105; E-mail: muhammadzaini@unlam.ac.id


Abstract

Background:

Research of Program for International Student Assessment (PISA) focusing on reading material, mathematics, and Natural Science shows that Indonesia ranks 10th of the lowest from 65 countries. Research of Trends International Mathematics and Science Study (TIMSS) shows the Indonesian students capability at very low rank in 1) understanding complex information, 2) theory, analysis, and problem solving, 3) use of tools, procedures, and problem solving, and 4) investigation. One possible way to overcome the problem of Indonesian students’ weakness is to improve the Lesson Plan.

Materials and Methods:

This research intended to determine the effectiveness of Science-Biology Lesson Plan through design-based research. The research used Tessmer Model with the stages of self-evaluation, expert reviews, one by one, small group, and field test as the research focus. The research subject was VII A students amounted to 40 students. The data included 1) cognitive learning outcomes, 2) performance skills, 3) character behavior, 4) social skills, and 5) critical thinking skills. Learning outcomes was obtained through test and analyzed based on Minimum Mastery Criteria. Performance skills were assessed through observation using performance task detail rubric. Character behavior and social skill were assessed through observation using characteristic behavior rubric. Critical thinking skills were ssessed through student worksheets with quantitative rubric.

Results:

The research result shows that Lesson Plan is effective to be used based on (1) student learning outcomes (product and process) that have exceeded the classical completeness, (2) student performance skills are very good, (3) student character behavior (discipline and responsibility) is very good, (4) student social skills (collaborating and contributing ideas) are mostly very good, and (5) student critical thinking skills are good.

Keywords: Lesson plan, Effectiveness, Design-based research, PISA, TIMSS, NRC.



1. INTRODUCTION

The main key to deal with all 21st century challenges is scientific literacy [1]. Scientific literacy can be meant as knowledge and scientific skill to identify questions, gain new knowledge, explain scientific events, draw conclusion based on the facts, understand the nature of science, awareness of science and technology linkage, as well as have a concern for science issues [2]. Therefore, scientific literacy deals with several scientific skills.

The results of National Research Council (NRC) workshop 2007 on 21st century skills define five skills that needs to be improved namely adaptation, complex communication skills, non-routine problem-solving skills, self-management/self-development, and thinking systems. The next workshop results focus on three skills in learning, namely [1] cognitive skills (non-routine problem solving, critical thinking, thinking systematically), [2] interpersonal skills (complex communication, social skills, teamwork, cultural sensitivity, diversity), [3] interpersonal skills (self-management, time management, self-improvement, self-regulation, adaptability, and executive function).

The research results of Trends International Mathematics and Science Study (TIMSS) on 2015 positioned Indonesia at 44th rank of 47 countries in the field of science [3]. This indicates that the scientific literacy of students is still low. Several factors have significant impact on scientific literacy development of students in learning, namely learning model, facilities, learning resources, teaching materials, and etc. [4]. The important thing is to design the learning and conduct assessments that can stimulate the scientific literacy increase. One way that may facilitate improving scientific literacy is to design contextual learning based on the students’ environmental condition.

The lesson design is compiled into a tool called Lesson Plan Tool (RPP Tool). According to the Regulation of Minister of National Education number 41 of 2007, Lesson Plan tool consists of syllabus, Lesson Plan, Student Worksheet and the answer key, teaching materials, learning media, and evaluation tools. Lesson Plan tool can affect learning outcomes. In addition, several matters related to learning process such as learning strategy, student, and teacher also affect learning outcomes [5].

Learning design is made by referring to 2013 Curriculum. The 2013 Curriculum is designed to improve the competence of knowledge, skill, and attitude comprehensively. Teachers are required to be more creative in teaching, improving teaching skill, and developing pedagogically in the learning process [6]. This is reasonable because Indonesian student’s ability is at low rank in understanding 1) complex information, 2) theory, analysis, and problem solving, 3) use of tools, procedures, and problem solving, as well as 4) investigation [7].

The challenges for teachers are teaching concepts, processes, and thinking skills comprehensively to students. Therefore, a learning should be planned, assessed, and evaluated. A teacher is required to plan varied learning strategies with the principles of learning and empowering students, not just teaching students [8]. Students learn by experiencing themselves, constructing knowledge, then providing explanation to the knowledge [8].

Planning, assessing, and evaluating Lesson Plan tool is a means to improve the product [9]. It can be performed through design-based research [10]. The main purpose is to obtain high quality product and one of them is product effectiveness, namely the prototype of Lesson Plan tool.

Design-based research is relevant to educational practices and policies because the intention is to solve the complex issues on research-based on educational practices or to improve and validate the learning theory [11]. According to Ritland [12], since the early 1990s in educational research, the term “design” is used as an experimental design in educational research. Design-based research is not enough with just a single approach but requires a series of approach [13].

Lesson Plan tool in school needs to be evaluated and revised through development research. This research process is an iterative design focusing on the implementation and development of intervention/product in education [14]. The term ‘development research’ is popularized by Richey & Klein [15] as design and development research namely a systematic research of design, development, and evaluation that intends to generate learning/non-learning product. There are two paradigms in design-based research, namely design to solve problems, and design as reflection in action [16].

Lesson Planning tool development research is an innovation for learning process. By this way, a valid, practical, and effective product will be generated [11]. Product of development results is valid (relevant) due to intervention and design based on state-of-the-art knowledge [11]. Tessmer [17] described the practical term where the users are free to use the product in the learning environment and it is said to be effective if a goal can be achieved according to specific questions and criteria.

Lesson Plan tool development is according to the Regulation of Ministry of National Education Number 65 of 2013 on ICT utilization. This requires the learning to prepare students for global era. The demands of 21st century students’ skills are how to think critically, find solutions, be creative, and gain information and media skills required in learning using CST.

Rotherdam & Willingham [18] described that success of students depends on their ability to think critically, solve problems, communicate and collaborate. Inquiry-based learning provides opportunity to teachers to explore critical thinking and creative thinking skill. Both of the thinking skills are high order thinking skills which are always emphasized in modern science learning. Lohner et al. [19] stated that inquiry learning offers authentic experience by involving students in the process of knowledge construction. Inquiry-based learning in science involves the process of science and thinking skills. This model is also called 'guided discovery'. Teacher guides students’ inquiry until they ‘discover’ the concept of science that has been determined by the teacher [20].

The implementation of inquiry-based learning has succeeded in increasing students’ activity, improving learning and learning outcomes, as well as scientific attitudes [21-23]. Inquiry-based learning is also able to improve character of student and learning activity [24, 25]. This success needs to be continued to determine the effectiveness of different topics and levels through design-based research.

2. RESEARCH METHOD

A good method can lead the researcher in achieving the research objective [26-28], in which this research was a product development research in the form of learning implementation plan tools (syllabus, lesson plan, worksheet, and teaching materials) based on guided inquiry model. The research procedures referred to Tessmer model namely self-evaluation, expert reviews, one by one, small group, and field test as the research focus [9]. One of the qualities that a developed product should possess is effectiveness which is determined based on the field test data.

This research was conducted for a month in one of the junior high schools. The research was conducted in a classroom. Field test stage involved 40 students of VII Grade and a subject teacher of Natural Science as the research partner.

The type of data to determine the effectiveness of learning implementation plan tool includes 1) cognitive learning outcomes, 2) performance skills, 3) character behavior, 4) social skills, and 5) critical thinking skills. Each assessment is referred to the assessment sheet instrument.

  1. Cognitive assessment sheets: (a) cognitive assessment sheet of product was used to measure students’ mastery over the learning materials. This instrument consisted of pretest and post-test in form of 20 multiple-choice questions that performed for less than in 15 minutes; (b) cognitive process assessment sheet was used to measure students’ mastery over the learning activities performed which consisted of pretest and post-test in form of 5 multiple-choice questions that was performed in less than 5 minutes and the validated questions from expert review were majorly used for cognitive assessment [29].
  2. Performance skill assessment sheet. This instrument was provided to assess students’ skills in using tools during teaching and learning activities. The instrument was in a form of observation sheet with assessment rubric. This assessment involved the observer who assessed student psychomotor during the learning process. Instrument format was adapted from Nur [29].
  3. Student’s behavior and character assessment sheet. This instrument was used to assess student’s behavior and character during learning process. The instrument was in a form of observation sheet with assessment rubric. The observer provided score based on the available rubric for assessment format. Instrument format was adapted from Nur [29].
  4. Student’s social skills assessment sheet. This instrument was used to assess student’s social skills during the learning process. The instrument was in the form of observation sheet with assessment rubric. The observer provided a score based on the available rubric for assessment format. Instrument format was adapted from Nur [29].
  5. Student’s critical thinking skill assessment sheet. This instrument was used to assess students’ critical thinking skill through their answer on the worksheet. The observer provided a score based on the available rubric for assessment format. The instrument format was adapted from Herlina [30].

Analysis of effectiveness data obtained from the assessment data during field test was conducted descriptively. Data analysis techniques of each learning outcomes are explained in more detail as follows:

  1. Cognitive assessment result. The result observed from the cognitive assessment is classical completeness. Learning was considered as effective if it exceeded the classical completeness of 85%. Classical completeness was obtained from the number of students who individually fulfilled the criteria. Students were considered to have fulfilled the criteria individually if the value was obtained at least in accordance with minimum mastery criteria of 70. The formula to obtain cognitive result is:

Description:

P = Pretest/Post-test value

N = student score

Based on the individual completeness, then the classical completeness could be determined by formula:

Classical completeness result was analyzed descriptively.

1. The result of performance skill and critical thinking skill assessment of each student was obtained by formula:

The result of all students was average and converted into percent for each meeting. The result was described in descriptively.

2. The result of behavior character and social skills assessment of each student was obtained by formula:

Description:

= average of activities

∑X = sum of activities

n = the number of activities

The result of entire students was averaged and converted into percent for each meeting. The result was described descriptively.

3. RESULT

Table 1 shows cognitive learning result of the product as well as the process that passes beyond the classical completeness at post-test. The results of student performance skills are presented in Table 2 .

Table 1. Cognitive learning outcomes.
Test Students Who Took the Test (People) Students Completing Criteria (Person) Students not Completing Criteria (Person) Classical Completeness (%)
Product Pretest 40 1 39 2.56
Post-test 37 3 92.5
Process Pretest 0 40 0
Post-test 37 3 92.5
Description: individual completeness: Minimum Mastery Criteria = 70; classical completeness 85%.
Table 2. Results of student performance skills.
Average of Student Performance Skill Results Achievement at Meeting of- Average of Entire Meetings Category
1 2 3 4 5
97.3 99.2 96.1 96.9 96.1 97.12 Very Good
Category: Very Good (85-100%); Good (70.01- < 85%); Moderate (50.01-< 70%) Less (< 50%).

Table 2 shows very good student performance skills. The results of student behavior character are presented in Table 3.

Table 3. Results of student behavior character.
Observed Attitude Average Behavior Character Achievement at Meeting of- Average of Entire Meetings Category
1 2 3 4 5
Discipline 4 4 4 4 4 4 Very Good
Responsibility 4 4 4 4 4 4 Very Good
Category: Very Good (4.0); Good (3.0); Moderate (2.0) Less (1.0).

Table 3 shows very good student behavior character (discipline and responsibility). The results of student social skills are presented in Table 4.

Table 4. Results of student social skills.
Observed Attitude Average Social Skills Achievement at Meeting of- Average of Entire Meetings Category
1 2 3 4 5
Collaborating 4 4 4 4 4 4 Very Good
Contributing Ideas 4 4 4 4 4 4 Very Good
Category: Very Good (4.0); Good (3.0); Moderate (2.0) Less (1.0).

Table 4 shows very good student social skills (collaborating and contributing ideas). The results of student critical thinking are presented in Table 5 that shows very good student critical thinking skills.

Table 5. Result of student critical thinking skills.
Average Critical Thinking Skill Results Achievement at Meeting of- Average of Entire Meetings Category
1 2 3 4 5
76.40 87.80 80.00 89.12 87.70 76.40 Good
Category: Very Good (85-100%); Good (70.01- < 85%); Moderate (50.01-< 70%) Less (< 50%).

The research result shows that Lesson Plan is effective to be used based on student learning outcomes (product and process) that have exceeded the classical completeness, students’ performance skills that are above the average, student character behavior (discipline and responsibility) that is above the average, student social skills (collaborating and contributing ideas) that are above the average, and students critical thinking skills that have reached the average.

4. DISCUSSION

The effectiveness indicators of lesson plan tool are cognitive learning outcomes, performance skill, character behaviors, social skills, and critical thinking skills. Learning outcomes of students (product and process) have exceeded the classical completeness. This is in line with previous research [22, 24, 25, 31] that they conducted inquiry-based learning and assisted students in finding their own answers to solve a problem. This learning emphasizes the process, although it does not neglect the product.

Student performance skill is very good because almost all groups can use the equipment based on the details of performance tasks that have been prepared by teacher. Sanjaya [32] stated that direct learning is student’s experience as the results of their activities. Students experienced and felt by themselves everything related to the goal achievement.

Performance skills are very good. These findings are in line with previous research [33, 34] that inquiry-based learning cannot be separated from students’ activities in implementing experiment. Furthermore, the use of student worksheet oriented to approach of environmental-based science process skills has positive effect to students learning outcomes improvement on the performance aspects [33].

The very good student performance skills generate a good student process skill [35, 36]. Blessinger [37] and Sanjaya [38] explained that inquiry-based learning emphasizes on the aspects of cognitive, affective, and psychomotor in balance. Consequently, the learning is considered to be more meaningful.

Character behaviors use the indicator of discipline and responsibility well. This is in line with the previous research [39]. She explained the inquiry-based Lesson Plan tool successfully instilled the character values. Student’s character shows positive result through guided inquiry [40]. Other research reported that the very good characters appeared are independent, curious, tolerance, creative, discipline, collaboration, and responsibility [41].

Inquiry based learning according to Shoimin [42] emphasizes on the development of cognitive aspect, affective, and psychomotor in balance, so the learning is considered to be more meaningful. Moreover, this kind of learning is in accordance with psychological development of modern learning which considers learning a process of behavioral change.

Most of students’ social skills (collaborating and contributing ideas) are very good. Social knowledge cannot be formed from a person action on an object, but it is formed through the interactions with others Wadsworth in Sanjaya [32]. This is the essence of social skills. When student interacts with their friends, then the opportunities to build social knowledge can develop.

The observation result of social skill using indicators of collaboration and contributing ideas is good. This is in line with previous research that the developed Lesson Plan tool shows student competence improvement because inquiry syntax can accommodate competences of spiritual, social, knowledge, and skills [43]. Inquiry-based learning is designed with aims to make students have scientific skills and directly involved in the learning process [44].

Good critical thinking skill is supported by previous research [21, 44-46]. Critical thinking skill in students’ groups of inquiry-based learning is better than the control group [44]. Inquiry-based learning is positively correlated to the student critical thinking ability [21]. Inquiry-based learning has significant positive impact on student critical thinking skills [46]. The student critical thinking skills (formulating problems, formulating hypotheses, collecting data, analyzing data, and drawing conclusion) are also in good category generally [45].

Critical thinking skill is required and must be owned by students [47]. The developed student worksheet is effective to improve student critical thinking skill. Critical thinking skill is one of the learning instrument effectiveness indicators [48].

CONCLUSION

Based on the research results, the effective Lesson Plan tool is based on student outcomes (product and process) that have exceeded classical completeness, student performance skill is very good, student character behavior (discipline and responsibility) is very good, most of student social skills (collaborating and contributing ideas) are very good, and student critical thinking skills are good.

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

Not applicable.

HUMAN AND ANIMAL RIGHTS

No Animals/Humans were used for studies that are the basis of this research.

CONSENT FOR PUBLICATION

Not applicable.

CONFLICT OF INTEREST

The author declares no conflict of interest, financial or otherwise

ACKNOWLEDGEMENTS

On this occasion I would like to express my gratitude and express my highest appreciation to the Dean of the Faculty of Teacher Training and Education at Universitas Lambung Mangkurat who has provided assistance with the cost of writing this article. Not forgetting, I also convey an immeasurable appreciation to Dr. J Dalle who have helped improve this article so that it is worthy of being accepted and published in international journals.

REFERENCES

[1] UNEP. 21 Issues for the 21st Century: Result of the UNEP foresight process on emerging environmental issues, United Nations Environment Programme (unep). Nairobi Kenya 2012.
[2] OECD. Programme for International Student Assessment. PISA 2015 released field trial item cognitive Doc: CY6_TST_PISA 2015FT Released cognitive items 2015.
[3] Martin MO, Mullis IVS, Foy P, Hooper M. TIMSS 2015 international results in science 2016.
[4] Fathurohman A, Zulherma , Kurnia F. Analysis of high school physics grade IX teaching materials in northern Indralayu sub-district based on scientific literacy categories. Jurnal Inovasi dan Pembelajaran Fisika 2014; 1(1): 43-7.
[5] Sugandi A. Learning theory 2008.
[6] Uno HB, Mohammad N. Paikem learning approach 2011.
[7] Nuh M. Teacher training materials (2013 curriculum implementations for junior/senior high school science) 2013.
[8] Syatra NY. Design of effective relationships between teachers and students 2013.
[9] Tessmer M. Planning and conducting formative evaluations: improving the quality of education and training 1993.
[10] Plomp T, Nieveen N. An Introduction to Educational Design Research. Proceedings of the seminar conducted at the East China Normal University, Shanghai (PR China) 2007; 23-26: 9-36.
[11] Plomp T, Nieveen N. educational design research. T Plomp, & N Nieveen (Eds), Part A: An introduction Enschede, the Netherlands SLO 2010.
[12] Ritland BB. The role of design in research: the integrative learning design framework. Educ Res 2003; 32(1): 21-4.
[13] Barab S, Squire K. Design-based research: putting a stake in the ground. J Learn Sci 2004; 13(1): 1-14.
[14] Rawson C, Hughes-Hassell S. Research by design: the promise of design-based research for school library research. School Libraries Worldwide 2015; 21(2): 11-25.
[15] Richey RC, Klein JD. Design and development research: Methods, strategies, and issues 2014.
[16] Kennedy-Clark S. Research by design: design- based research and the higher degree research student. J Learn Des 2013; 6(2): 26-32.
[17] Tessmer M. Planning and conducting formative evaluations 1998.
[18] Rotherham AJ, Willingham D. 21st Century skills: the challenges ahead. Educ Leadersh 2009; 67(1): 16-21.
[19] Lohner S, Van Joolingen RW, Savelsbergh ER, Hout-Wolters B. Students’ reasoning during modeling in an inquiry learning environment. Comput Human Behav 2005; 21(3): 441-61.
[20] McBride WJ, Bhatti IM, Hannan AM, Martin F. Using an inquiry approach to teach science to secondary school science teachers. Procedia Soc Behav Sci 2002; 46: 2327-33.
[21] Rahmawati AN. The application of integrated of sciences with the inquiry learning model on eyes theme at smpn 1 maduran lamongan. PENSA e-Journal 2015; 3(3): 76-91.
[22] Hermawati NWM. The influence of inquiry learning models on mastery of biological concepts and scientific attitudes of senior high school students learning interest. Jurnal Pendidikan IPA 2012; 2(2): 1-30.
[23] Setiawati R. Guided inquiry-based module development to optimize students’ scientific attitudes to dynamic electricity subjects in sman 8 purwerejo calss x academic year 2013/2013. RADIASI: Jurnal Berkala Pendidikan Fisika 2013; 3(1): 24-7.
[24] Asyhari AT, Sunarno WT, Sarwanto S. Development of guided inquiry-based high school physics learning tools integrated with character education. Inkuiri 2014; 3(1)
[25] Sukamsyah S. Efforts to improve learning outcomes by applying type a guided inquiry method to the heat concept at class vii smpn 5 seluma. EXACTA 9(1): 38-44.
[26] Dalle J, Hadi S, Baharuddin , Hayati N. The development of interactive multimedia learning pyramid and prism for junior high school using macromedia authorware. The Turkish Online Journal of Educational Technology 2017; 16(3): 714-21.
[27] Baharuddin, Dalle J. Interactive courseware for supporting learners competency in practical skills. The Turkish Online Journal of Educational Technology 2017; 16(3): 88-93.
[28] Derlina, Dalle J, Hadi S, Ariffin AM, Sumantri C. signaling principles in interactive learning media through expert's walkthrough. Turkish Online Journal Of Distance Education 2018; 19(4): 147-62.
[29] Nur M. Examples of smk lesson plans tools: the basis of vocational competency for the application of basic concepts of electricity and electronics for the first semester of grade x Ministry of National Education 2011. UNESA
[30] Herlina L. The development of a virus concept device uses a problem-based learning model for student learning outcomes and high-level thinking skills in high school Master Thesis of Universitas Lambung Mangkurat 2014. Unpublished
[31] Rosmalina I. Development of inquiry learning tools on the concept of ecosystems towards learning outcomes and high-level thinking skills of students mts darul istiqamah barabai daughter Master Thesis of Pendidikan Biologi Unlam Banjarmasin 2012. Unpublished.
[32] Sanjaya W. Learning strategies oriented to standard educational processes 2006.
[33] Arsih F. Development of student worksheets class viii science biology that oriented science process skills approach. Jurnal Ta’dib 2010; 13(1): 1-10.
[34] Rinarta IN, Yuanita L, Widodo W. Development of a lesson plan inquiry model to train science process skills and mastery of the concept of junior high school students. Fisika 2014; 2(2): 70-88.
[35] Aktamis H, Ergin O. The effect of science process skill education on students’ scientific creativity, science attitudes and academics achievement. Asia-Pacific Forum on Science Learning and Teaching 2008; 9(1): 1-21.
[36] Al Rabadi IGS, Heyam OS. Al Momani, Khetam IS. The effect of using process approach on science achievement and scientific attitudes among jordanian basic stage students. J Educ Pract 2013; 4(20): 136-50.
[37] Blessinger P, John MC. Inquiry-based learning for multidisciplinary programs: a conceptual and practical resource for educators 2015.
[38] Sanjaya W. Learning strategies standardized educational processes 2008.
[39] Heriningsih PD, Agustini R. Development of lesson plan-based character based tools to improve student learning outcomes of junior high school students. Prosiding Seminar Nasional Kimia 2014.
[40] Koksal EA, Giray B. The effect of guided-inquiry instruction on 6th grade turkish students’ achievement, science process skills, and attitudes toward science. Int J Sci Educ 2014; 36(1): 66-78.
[41] Trian AE, Haryani S, Mantini SRS. The development of an integrated ipa module characterizes the theme of environmental management for the class VII junior high school. Unnes Science Education Jurnal 2013; 2(2): 269-73.
[42] Shoimin A. Model Pembelajarn Inovatif dalam Kurikulum 2013 2014.
[43] Henykartikasari T, Indriwati ES, Prabaningtyas S. Development of inquiry-based lesson plan devices on fungal material biology subjects to improve the competency of class x sma brawijaya smart school poor. Jurnal Ilmu Hayati 2015; 1(1): 1-11.
[44] Sutama IN, Aryana PB, Swasta JB. The influence of inquiry learning model on critical thinking skills and scientific performance in the biology class xi science major sman 2 martapura. Jurnal Penelitian Pascasarjana 2014; 4: 1-14.
[45] Zaini M. Guided inquiry-based learning on the concept of ecosystem toward learning outcomes and critical thinking skills of high school students. IOSR Journal of Research & Method in Education (IOSR-JRME) 2016; 6(8): 50-5.
[46] Duran M, İlbilge D. The Effect of the Inquiry-Based Learning Approach on Student’s Critical-Thinking Skills. EURASIA J Math, Sci Tech 2016; 12(12): 2887-908.
[47] Beaumont J. A Sequence of Critical Thinking Tasks. TESOL J 2010; 1(4): 1-22.
[48] Zaini M, Supiati . Developing learning device on environment pollution topic in senior high school. The Social Sciences 2017; 12(12): 2269-76.