European Journal of Educational Research

: This meta-analysis study investigates the effect of computer-based tests on motivation. The random effect model is the basis for calculating the summary effect, and it is found that the 30 articles obtained through the PRISMA method, which consist of four parts, namely identification, screening, eligibility, and included, can be generalized. Data analysis was performed using R studio and OpenMEE. Based on the research results, the p-value <.05 was obtained, so it was concluded that using a computer-based test significantly affected motivation. In the moderator analysis, it was found that computer-based tests were most effective for intrinsic motivation for the motivation variable, university students for the participants variable, and the country of Iran for the region variable. Evaluation of publication bias using the funnel plot and regression method shows no publication bias, proving that the 30 articles analyzed represent all existing studies on computer-based tests


Introduction
Since their introduction, the computer has become essential to applied psychology. The application of computer measurement methods has experienced rapid development in the last few decades. Automated processes using computers have developed in scope and depth of material (Butcher et al., 2000). Computers have become an essential part of life in administrative and educational activities. One of the functions of using a computer is the existence of a computer-based test (CBT) (Carpenter & Alloway, 2019). Practical, fast, easy, efficient in the application, and can be integrated with many school computers is a consideration of the change towards the use of computer technology (Davies & West, 2014). According to Lowyck (2014), technology in education is influenced in many ways as innovation in education. Behaviorist, socio-constructivist, cognitive, and constructivist approaches to learning have also made clear contributions to the use of educational technology.
The function of computers for assessing, determining items, and reporting test scores has become a concern in the field of measurement and assessment. This is what causes the rapid growth in the use of this computer technology. One of the uses of this technology is CBT, where users of CBT occur very quickly. The rapid development of CBT has also been accompanied by extraordinary actions and delivered in various forms (Ockey, 2009). Safaruddin et al. (2021) stated that the advantages of CBT compared to paper-based tests include more time efficiency, assessments can be made directly, and feedback in multiple-choice exam questions.
CBT are promoted to revolutionize Education, providing experiences to students, improving performance, and assessing students' exam results through more modern computerized testing tools. Faster test report results, good security, flexible scheduling, environmental benefits, efficiency, and increased focus are the benefits of CBT (Boevé et al., 2015). CBT can improve assessment in four general categories: (a) increase the depth of skills and knowledge of students taking the test, (b) increase the accuracy of the test results obtained by students, (c) increase the efficiency of the assessment process, and (d) increase the accuracy testing (Dolan & Burling, 2012). According to Luecht (2006), CBT have enabled many new measurement technologies, such as rapid item creation and automated test assembly. The widespread internet has implications for CBT which have developed into very convenient, cost-effective, efficient, and up-to-date assessments.
Testing using a computer is more helpful in expanding access; besides, it is cheaper and more standardized. Web-based testing will allow almost any university or school computer to become a test center (Bridgeman et al., 2003). Istiyono et al. (2020) stated that using CBT could save time, and the results obtained by students come out immediately after completing the test. Apart from being cognitive, CBT also influence the motivation of their users (Chua, 2012;Chua & Don, 2013;Nasri et al., 2021;Nikou & Economides, 2016;Partovi & Razavi, 2019;Sins et al., 2008). According to Dolan and Burling (2012), one additional benefit of CBT regarding delivery and reporting is that it reduces our dependence on limited resources, namely trees for paper and fossil fuels needed for their transportation. In addition, CBT can reduce the carbon footprint associated with the process.
One of the attractive features of CBT is the possibility to offer examinees the opportunity to take the test multiple times (van der Linden, 2006). Testing using computerized programs provides an opportunity to improve psychological testing. Computerization gives test developers greater freedom to increase or decrease the item features and assignments presented to examinees, which allows for a more expansive and open assessment (Drasgow & Mattern, 2006). According to Haryanto (2012), in program development, three things must be considered: data types, variables, and operators. Data types determine the type and range of values; variables are symbols that function as representatives of a value that can be changed, while operators function for calculations.
Ensuring the safety of tests and testing processes is a significant challenge faced in measuring and assessing exams. Test security must be prioritized by ensuring test takers only have access to exam questions after the test is carried out and providing the score report reflects the actual score. The security of CBT questions and scores has become a significant primary concern since the use of computers in tests for high-risk testing situations (Ockey, 2009). There are various variations in the CBT that instrument developers can use; according to Zenisky and Luecht (2016), common Computer Based Test variations on the theme of the selected answer item type or short answer include; (a) many lists of answer items that require examinees to choose two, three, or more solutions; (b) fill in the blanks with quick answer items; (c) "hot spot" item, selecting a graphic area on a digital image or image using the mouse; (d) computerized essays; (e) text checking and item insertion; (f) selecting information from a chart, reference chart, exhibit, and others; (g) an ordered answer format, in which the examinee chooses the appropriate order or priority for several steps; (h) the inclusion of digital animation, audio, video, or a combination of the three; and (i) graphical modeling format.
Motivational factors such as student orientation and self-efficacy are essential things that need to be considered in using computer-based tests besides cognitive ones (Sins et al., 2008). Motivation is a triggering process for an activity that leads to a specific goal that is strongly influenced by the person that leads to results and is a part of the social cognitive theory (Schunk & DiBenedetto, 2020). Motivation is an essential part of student interest in education (Suparman et al., 2022). There are various motivations for students, such as achievement motivation, intrinsic motivation, extrinsic motivation, and learning motivation. Achievement motivation refers to behavior related to self-evaluation of one's competence about standards of excellence (Grund et al., 2022). The natural motivation closely related to the search for new possibilities and challenges related to cognitive development and social development that has something to do with one's destiny is intrinsic motivation (Bailey et al., 2021). Extrinsic motivation is positively related to enthusiasm, dedication, self-absorption, and insufficiency (Kotera et al., 2021). Learning motivation predicts academic achievement (Gholami et al., 2021).
Various studies are showing the relationship between CBT and motivation, such as CBT significantly increasing intrinsic motivation (Chua, 2012), CBT increasing the intrinsic motivation of test takers (Chua & Don, 2013), CBT effectively increasing intrinsic motivation and extrinsic motivation (Piaw, 2011), computer equipment used for high school students increases intrinsic motivation and extrinsic motivation in science learning (Nikou & Economides, 2016), the use of CBT has an impact on improving students' chemistry learning motivation (Ahmad et al., 2021), and the use of computers is more effective in increasing learning motivation (Nasri et al., 2021). The mixed results of this study indicate a link between CBT and motivation, so further analysis is needed with quantitative analysis using meta-analysis. Meta-analysis is necessary to make more robust conclusions from existing research results by collecting data from several studies and providing more accurate estimates.
The meta-analysis in this study was not only to see the effect of CBT on motivation but also to determine the effectiveness of various moderator variables such as the type of motivation, participants, and region. The moderator variable is included in the analysis because it can provide a deeper understanding of various types of motivation, provide more robust scientific evidence for decision-making, and identify factors that influence the effect of a CBT. Several moderator variables analyzed include (a) Types of motivation in the form of achievement motivation, intrinsic motivation, extrinsic motivation, and learning motivation; (b) Participants come from students, teachers, junior high, high school, and elementary school; (c) The region comprises Iran, Malaysia, Indonesia, Greece, Germany, the United States of America, Kenya, Nigeria, and Saudi Arabia.
The research questions in this study include the following: 1. Does the CBT have a significant effect on motivation?
2. Is using a CBT most effective when applied to achievement, intrinsic, extrinsic, or learning motivation? 3. Is a CBT most effective for university students, teachers, junior high school, senior high school, or elementary school participants?
4. Which countries are the most effective in implementing CBT?

Methodology
This research makes a real contribution to using CBT on motivation. In particular, this meta-analysis research aims to determine the effect of computer-based tests on educational motivation. In addition, this research can also provide knowledge about the types of motivation that dominate the use of CBT.

Research Design
This research is part of quantitative research with a meta-analysis approach. Meta-analysis is a statistical technique combining several similar research results to obtain quantitative data integration. This study analyzes previous empirical research studies related to CBT on motivation. Motivation is based on the education field, so the participants' sources in this study varied, such as university students, teachers, senior high school, junior high school, and elementary school. The results of previous comparative studies were based on the control group and the experimental group, which consisted of the number of samples, the mean, and the standard deviation. Research meta-analysis synthesizes quantitative information collected from different studies and compares the constructs and relationships (Saepuzaman et al., 2021;Taranilla et al., 2022).

Sample
The research sample used in this study used publications related to the use of CBT on motivation, which were published from 2010 to 2022. The publications used came from the Scopus and Google Scholar databases in the form of journals, proceedings, theses, and dissertations. Search for samples using the help of the publish or perish application using the title words "motivation" and the keywords "computer-based test." The sample selection procedure was carried out using the PRISMA method, which consisted of four parts: identification, screening, eligibility, and included methods (Wu et al., 2023). According to Figure 1, the PRISMA method is modified to suit the needs and objectives of the research. The validity of the data and the suitability of the data information with the research objectives are the main requirements for data collection. From the results of the initial analysis based on the suitability and completeness of the data in the form of the number of samples, mean, and standard deviation, a total of 30 articles were obtained. Data from the initial analysis in the form of the number of samples, the mean, and the standard deviation of the experimental and control classes were analyzed using the R studio and OpenMEE programs. OpenMEE aims to combine the power of easy-to-use GUI-based programs such as MetaWin with R's statistical sophistication and flexibility (Wallace et al., 2017). The process of coding and extracting experimental study data is the first step in data analysis. Coding and extraction adjust to the type of motivation, type of participant, country of origin, and publication type. Reliability in coding and extracting study data in the meta-analysis was carried out by involving three data collectors, and then the data collection results were compared and verified to reduce errors in data collection. The second step is to calculate the effect size using the hedges'd effect size because the studies have different measurement scales, so the statistical analysis results must be equated from each study (Hamman et al., 2018). The third step calculates the summary effect using random effects (Beisemann et al., 2020;Jackson & Turner, 2017). The final step is to test the hypothesis, evaluate publication bias, and analyze the moderating factors.

Results
The search results for publications that met the criteria and had data on sample size, the mean, and the standard deviation for the experimental and control groups resulted in 30 publications. From 30 publication data, effect size and variance are obtained according to Table 1 using R studio and OpenMEE. After obtaining the effect size and variance values, a summary effect calculation is carried out as a results model and heterogeneity according to Tables 2 and 3. The summary effect provides an overview of the observed effect size and is a consideration for continuing the moderator analysis (Borenstein & Higgins, 2013). The average effect size value in Table 2 shows that of the 30 studies analyzed, the p-value was < .05, so it was concluded that using a CBT significantly affected motivation. In Table 3, the value of τ 2 > 0 means that the effect size of each study is heterogeneous, and the Q value is statistically significant heterogeneously (Card, 2012). The I 2 value obtained was 96.883, indicating that the variance of the 30 studies was very high. The summary effect results are corroborated by the forest plot results shown in Figure 2.

Figure 2. Forest Plot
In Figure 2, a summary effect of 1,330 > 0 is obtained, meaning that the use of a CBT is more effective than without using a CBT about motivation. Four studies in Figure 2 have a negative effect size; this shows that the results of the control group are better than the experimental group, and most have a positive effect size; this indicates that the results of the experimental group are better than the control group.
In Table 3, the Het. p-Value < .05 means that the variability is statistically significant (Ramdani et al., 2022). Significant variability indicates that the variance of the 30 studies analyzed is diverse, so it has the potential for moderator analysis. The value of the variable motivation analysis is according to Table 4. All subgroups on motivation have a p-value <.05, meaning that all four variables in the motivation category are significant. A CBT is most effective when applied to intrinsic motivation because the estimate is the highest. The position of the mean effect size is shown in yellow in Figure 3, forest plot motivation.

Figure 3. Forest Plot Variable Motivation
If the mean effect size of each category touches the red line, it means there is an indication that there is no difference in the effectiveness of each type of motivation on the use of a CBT. Figure 3 shows no difference in using a CBT for achievement, intrinsic, and learning motivation. A CBT can measure achievement, intrinsic and learning motivation. As for extrinsic motivation, there is a difference in effectiveness, meaning that there are other influences besides the computer-based test. A CBT is most effective when applied to intrinsic motivation because the mean effect size is more to the right. Table 5 shows the variable analysis values of participants and is reinforced by the forest plot participants in Figure 4. All subgroups in the participants have a p-value < .05, meaning that all five variables in the participant's category are significant. A CBT is most effective when applied to university students because it has the highest estimated value. The position of the mean effect size is shown in yellow in Figure 4 forest plot participants.   Figure 4 shows no difference in the effectiveness of using a computer-based test for university students, teachers, elementary schools, and senior high schools. This means CBT can be used for university students, teachers, elementary schools, and senior high schools. Whereas for junior high school, there is a difference in effectiveness, meaning that there are other influences besides the computer-based test. CBT are most effective when applied to university students because the mean effect size is more to the right. Table 6 shows the value of the regional variable analysis and is reinforced by the forest plot region in Figure 5. Four countries in the subgroup region have a p-value <.05, namely Iran, Malaysia, Indonesia, and Greece, meaning that these four countries are significant. In the countries of Saudi Arabia, Nigeria, and Germany with NA information because there was only 1 study obtained in those countries, whereas, in the United States of America and Kenya, the p-Value > .05 was received, meaning it was not significant, this was due to the acquisition of a control class that was higher bigger than the experimental class. The use of CBT is the most effective in Iran because the estimated value is the highest. The position of the mean effect size is in yellow in Figure 5 forest plot region.   Figure 5 shows that there is no difference in the effectiveness of using a CBT when applied to Iran, Malaysia, Indonesia, and Greece. A CBT is most effective when applied to Iran because the mean effect size is more to the right. Nigeria's mean effect size is indeed on the far right in Figure 5, but it is not significant, the same as Saudi Arabia and Kenya. Table 7 shows the value of the variable publication type analysis and is reinforced by the forest plot type in Figure 6. There are two subgroups with a p-Value <.05, namely journal and proceedings, meaning that these two subgroups are significant, while in the thesis, a p-Value >.05 means not significant. Figure 6 shows the position of the mean effect size on the variable publication type.   Figure 6 shows no difference in the effectiveness of CBT published in journals and proceedings. As for the thesis, there is a difference in effectiveness, meaning that other influences influence it. Publications about CBT are most effective in proceedings because the mean effect size is more to the right.
To ensure that the 30 studies analyzed did not have biased data and to test how robust the meta-analysis results were, an evaluation of publication bias was carried out using the funnel plot according to Figure 7 and reinforced by the proposed regression method Egger et al. (1997). Figure 7 shows that there is no publication bias; it can be seen from all sample studies in a symmetrical form. This is also reinforced by the linear regression test of funnel plot asymmetry in Table 8, which obtained a p-value of .7711 > .05, meaning that the funnel plot is symmetrical and has no publication bias.

Discussion
Our meta-analysis used empirical data based on previous studies examining the effect of CBT on motivation. To generalize 30 earlier studies on the impact of CBT on motivation, a Random effect model was used to calculate the summary effect because the data obtained came from different measurement scales (Baragilly & Willis, 2022;Dettori et al., 2022). The results showed that the use of computer-based tests affected motivation; this is consistent with previous research, which stated that the use of CBT could have a positive impact on motivation and increase motivation (Akhtar et al., 2022;Çakir, 2019;Dresel & Haugwitz, 2008;Tüzün et al., 2009). The difference between this study and previous research lies in the data analysis used, Tüzün et al. used the t-test with only 13 primary school students, Dresel and Haugwitz used factorial analysis with a sample of 151 6th-grade students, Çakir used an independent sample t-test with a sample of 80 academic students', while Akhtar et al. the focus of his research is also on computers but more on computerized adaptive testing.
Moderator analysis was used in this study because meta-analysis not only calculates the average effect size but can also be a tool for investigating variation in evidence (Harrer et al., 2021). In addition, the moderator analysis was continued because it has various study variances with significant variability. The first moderator in the research is motivation. Motivation in this study is divided into four parts: achievement motivation, intrinsic motivation, extrinsic motivation, and learning motivation. From the results of the study, it was found that of the three sub-motivations, namely achievement motivation, intrinsic motivation, and learning motivation, there was no difference in effectiveness, meaning that these three sub-motivations showed relevant results on the CBT, while extrinsic motivation had different effectiveness, meaning that there were other factors that influenced besides computer use based test on extrinsic motivation. A CBT is most effective when applied to intrinsic motivation because intrinsic motivation has a more decisive influence than extrinsic motivation (Good et al., 2022). Extrinsic motivation directs the attainment of goals and emphasizes the effect of goal expectations on the action; however, this can be influenced by the environment, internal circumstances, or experience (Morris et al., 2022). The results obtained are in accordance with the research of Hariri-Akbari et al. (2018), which states that students are intrinsically motivated through computer-based tests. The difference between this study and previous studies is that Akbari et al. used the factor analysis method involving 246 Iranian paramedical students.
Another moderator analysis is participants. Based on the analysis results, it was found that if the CBT were applied to university students, teachers, elementary schools, and senior high schools, they would show relevant results, or there would be no difference in effectiveness among the four sub-participants. Different results if applied to junior high school or there is a difference in effectiveness, meaning that there is another influence on the results obtained besides the use of CBT by junior high school students. The use of CBT is most effective when applied to university students. The results of this study are the same as those of Romero et al. (2009), which state that university students are highly motivated and enjoy using CBT for testing. In addition, the research results by Maqableh et al. (2015) said that CBT are more fun for university students. The difference between this study and previous research lies in the data analysis used, Romero et al. (2009)  Another moderator analysis is the region. Based on the analysis results, it was found that the effectiveness of CBT was the same if the use of CBT was implemented in Iran, Malaysia, Indonesia, and Greece. The use of CBT is most effective when implemented in Iran. The results of this study follow the results of Khoshsima et al. (2017), which state that the use of computerized tests can be considered a profitable alternative for university students in Iran. In addition, Çelen (2019) research results say that Iran is one of the most popular countries with motivational research. The difference between this study and previous research lies in the data analysis used, Khoshsima et al. (2017) used One-Way ANOVA involving 30 Iranian undergraduate students.
The results of publication bias show that the 30 studies analyzed in this study do not offer any publication bias. Publication bias needs to be analyzed to prevent severe threats to the validity of the meta-analysis research (Cheng et al., 2019). According to Card (2012), it is essential to test for publication bias to ensure that the 30 studies analyzed represent existing studies on the topic of computer-based tests. This means that this research has presented all previous studies on the effect of CBT on motivation, both those with significant and insignificant results.

Conclusion
The 30 articles analyzed in this meta-analysis study have represented previous research; the absence of publication bias in the results of this study evidences this. This meta-analysis provides clear evidence that there is an effect of using CBT on motivation. Four types of motivation form the basis of this research: achievement motivation, intrinsic motivation, extrinsic motivation, and learning motivation. Of the four types of motivation analyzed, it was found that the CBT was used effectively to measure achievement motivation, intrinsic motivation, and learning motivation, while extrinsic motivation was less effective. Based on the participants, the computer-based test was practical for university students, teachers, elementary school, and senior high school, while it was ineffective for junior high school. Meanwhile, based on region, computer-based tests were effectively used in Iran, Malaysia, Indonesia, and Greece.

Recommendations
The results obtained from this meta-analysis can be used as input and suggestions for further research. Three essential parts are received in this study: (a) A CBT is used effectively for intrinsic motivation. (b) A CBT is used effectively for university students. (c) A CBT is used effectively for Iran. Future researchers can examine the effect of using a CBT by specializing in intrinsic motivation or specifically on university student participants; they can also compare the use of a CBT on motivation between Iran and other countries. One of the research recommendations that can be carried out is the effect of using a CBT on the intrinsic motivation of Iranian students.

Limitations
This study provides transparent information about the effect of using a CBT on motivation. The limitation of this study is that the sources obtained can only divide the moderator types into four parts: motivation, participants, region, and publication type. This research can be expanded by adding several moderator variables so that more studies can be obtained.