P-ISSN: 2808-5957

E-ISSN: 2808-6724

ijoms.internationaljournallabs.com

1541

ANALYSIS OF THE QUALITY OF WEB-BASED ONLINE EXAMS AT

THE R.I. PROSECUTOR'S OFFICE EDUCATION AND TRAINING

AGENCY USING THE ISO 25010 STANDARD

Ardian Ramadita Muhram

, Metty Mustikasari

Department of Business Information Systems, Gunadarma University, Indonesia

1, 2

Email : ardian.ramadita.muhram@gmail.com, metty@staff.gunadarma.ac.id

ARTICLE INFO

ABSTRACT

Date received : 04 September 2022

Revision date : 15 September 2022

Date Approved : 25 September 2022

The Education and Training Agency for the Attorney

General's Office of the Republic of Indonesia is one of the

institutions that requires flexible and distributed learning.

In the exam assessment process, the Education and

Training Agency of the Attorney General's Office of the

Republic of Indonesia collects data on questions given to

participants in the form of files. The file is then used by

participants to answer questions, then the answer file is

sent to the organizers for inspection and assessment.

With the implementation of such an exam, various

problems arise which result in frequent delays in receiving

the exam results report because the organizers

experience time and energy constraints in manually

correcting participants' answers one by one. Other

problems that occur arise from the condition of the

answer file which is often lost or deleted due to the error

of the participants. Based on the above problems, the

Education and Training Agency of the Indonesian

Attorney General's Office feels the need to apply

information technology. The creation of this web-based

online exam system has been implemented since the

beginning of 2022 until April 2022. The online exam

system has been used in May and has started to be used.

To find out whether the quality of the web-based online

exam system has met the initial goal to be achieved,

namely to facilitate the organizers in carrying out the

exam process, assessing training participants' answers

and making reports on the results of the assessment, a

system evaluation is needed. This study aims to analyze

the online exam system used in the Education and

Training Agency of the Indonesian Attorney General's

Office which is currently being used with the ISO 25010

method of product quality dimensions. The objectives to

be achieved with this thesis research are to analyze the

Keywords: ISO 25010, product

quality model, online exam system.

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1542

quality of the online exam system at the Indonesian

Attorney's Education and Training Agency using the ISO

25010 standard, and to provide input and suggestions for

improving the online exam system at the R.I Attorney's

Education and Training Agency. Based on the results of

research and discussions that have been carried out using

the ISO 25010 standard method, the quality of the online

examination system of the Indonesian Attorney's

Education and Training Agency is quite feasible and meets

the criteria.

This work is licensed under CC BY-SA 4.0

INTRODUCTION

The R.I Prosecutor's Training Agency is one of the institutions that requires flexible

and distributed learning. In the examination assessment process, badiklat of the R.I

Prosecutor's Office created an online examination system with the aim of making it easier

for organizers to make questions and check participants' answers so that they can be

automatically carried out by the system, so as to speed up the preparation of participant

score reports.

In mid-2022, this exam system has been completed and began to be used for

online exam activities. In order to find out the quality and results of the web-based online

examination system that has been developed, a method is needed to assess whether the

system has met the initial goals to be achieved, namely making it easier for training

providers to carry out the exam process, assessing the answers of training participants

and making reports on assessment results.

The ISO 25010 - System and Software Quality Requirement and Evaluation method

is a standard method in measuring the quality of software that is online. ISO 25010 is a

method that will be used to analyze the online examination system used at the R.I.

Prosecutor's Training Agency R.I. Testing characteristics consist of product quality model

dimensions consisting of 8 characteristics ISO/IEC, (2011).

Past Research

In the first study entitled "Quality Analysis of E-Exam Applications Using ISO 25010

Standard." From the analysis carried out, the E-Exam application has not met the

reliability character, so the conclusion was drawn, that the E-Exam in Bekasi schools has

not met the ISO 25010 standard Tyas et al., (2021).

The second study was titled "Evaluation of UNG Websites Using ISO/IEC 25010."

From the analysis carried out, the UNG website has not met the characteristics of

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1543

Performance Efficiency, which means that the UNG Website cannot maintain and

attention users Tangkudung et al., (2019).

The third research with the title "Evaluation Analysis of Augmented Reality

Application for Balinese Cultural Information Based on ISO / IEC 25010 Standards." So

based on the results of all the research that has been carried out, it can be concluded

that the quality of the Augmented Reality Museum Bali application has met the ISO 25010

standard Andika & Yanti, (2018).

The fourth study entitled "Development of an Islamic Higher Education Institution

Tracer Study Information System and It's Performance Analysis using ISO/IEC 25010."

The results of this study show that all characteristics of ISO 25010 have been met. With

the successful development of the tracer study information system, it will make it easier

for UIN Syarif Hidayatullah to monitor and collect information about UIN Syarif

Hidayatullah alumni Anggraini et al., (2019).

The fifth research is "Quality Measurement of Transportation Service Application

Go-Jek Using ISO 25010 Quality Model." The results showed that the GO-JEK application

has good quality in the dimensions of product quality and quality of the dimensions of

use, or user perspective Izzatillah, (2019).

The sixth study was titled "Building a Web-Based Student Daily Examination

System with Reference to ISO 25010 Quality Standards." From the analysis carried out,

the results obtained that all test cases have results that are in accordance with the

expected or valid value Larasati et al., (2018).

METHOD

The object of research is the scientific goal of obtaining data with a certain purpose

and usefulness about an objective, valid, and realiable thing about a thing Sugiyono,

(2020).

The object of this study is a web-based online examination system owned by the

Education and Training Agency of the Prosecutor's Office of the Republic of Indonesia.

The type of research "Quality analysis of online examination systems using iso

25010 standards" is quantitative research. Research methods are basically a scientific

way to obtain data with a specific purpose and usefulness. Based on this, there are four

keywords that need to be considered, namely the scientific way, data, purpose and

usefulness Sugiyono, 2020).

Reliability

Characteristics Testing Uses

the WAPT 10.1 tool

to analyze the online

exam system. Using

a stress testing

scenario with a total of 20

virtual users

within 10

minutes. Data analysis of reliability test results is carried out by calculating the

total

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1544

(Formula

success rate

and

failure rate

from the WAPT test results, then reliability calculations are

carried out using the Nelson model: Tian et al., (2004).

Information:

R = Reability

f = Total failure

n = Total test case (work load unit)

r = Error rate

From the results of these calculations, it is then matched with the Telcordia GR

282 standard on

"Software Reliability and Quality Acceptance Criteria"

to determine the

level of reliability of the system. According to Telcordia standards, software

reliability

success is 95% or 0.95. Asthana & Angela, (2009) The image display of

the WAPT 10.1

software

can be seen in figure 1.

Figure 1.

WAPT

software display

Source:

WAPT 10.1 Application,

https://www.loadtestingtool.com/

Usability

Testing the characteristics of

Usability

was carried out using a questionnaire given

to participants of the Prosecutor's Administration Technical Training (TAK) Class II Wave

4 of 2022 with a total of 285 participants. To find out the exact number of samples in a

study, it was calculated using the

Slovin formula with a margin of error of 9%.

Slovin

formula:

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1545

(Formula

where:

n = number of samples

N = total population

e = margin of error / error tolerance (batas toleransi kesalahan)

Then the number of population samples will be given 8 questions to test

usability

The table of list of questions can be seen in table 1.

Table 1. List of Questions To Test

Characterisitic

Usability

Source: (Yusnizar, 2022)

After the respondents' results are obtained, calculations can be made for

the

usability

characteristic test.

Information:

The number of respondents answering Strongly Agree is scored 5

The number of respondents Agreed was scored 4

The number of respondents answering Enough Agree was given a

score of 3

The number of Disapproving respondents was given a score of 2

STS

The number of strongly disapproving respondents was scored at 1

The score Retrieved = (JSS x 5) + (JS x 4) + (JCS x 3) + (JTS x 2) + (JSTS x 1)

After the score is obtained, then a percentage calculation is carried out to obtain

the interpretation of the results of the

usability

test using the formula:

No Pernyataan

Tampilan dari aplikasi ujian online menarik

Menu-menu ujian online mudah dipahami

Soal-soal yang diberikan sudah sesuai dengan yang diajarkan

Ujian online lebih memudahkan peserta diklat untuk mengikuti ujian

Aplikasi ujian online ini membuat peserta diklat menjadi lebih

bersemangat di dalam belajar

Aplikasi ujian online ini sangat bermanfaat untuk mendukung kegiatan

belajar mengajar

Aplikasi ujian online dapat digunakan dengan lancar

Aplikasi ujian online ini sudah cukup baik

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1546

Information:

Total

score

Total score of respondents'

answer results

Number of questions

Number of Respondents

The calculation results are then converted into score interpretation with a slight

modification of the ranking terms used. The division of modified eligibility categories can

be seen in table 2. Riduwan, (2010).

Table 2. Score Interpretation Criteria

Source: Riduwan, (2010)

Performance Efficiency

character testing is intended to test the level of performance

efficiency of an application.

Performance Efficiency

testing is carried out using

GTMetrix

software

. GTMetrix report results

must meet

the load

in less than 10 seconds. The view

of GTMetrix can be seen in figure 2.

Figure 2. GTMetrix

report

view

Source: https://gtmetrix.com/

If the criteria tested have obtained the value, it will be compared with the time

response Stated By. Hoxmeier & Dicesare, (2000) Customer satisfaction according to

Hoxmeier & DiCesare can be seen in table 3.

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1547

Table 3. Customer Satisfaction Table

Response Time

(Seconds)

Predicate

Very Satisfied

3-9

Satisfied

9-12

Quite Satisfied

>12

Not Satisfied

Sources: Hoxmeier & DiCesare, (2000).

Security

characteristics testing is performed to test for system vulnerabilities. Does

the online examination system in the Badiklat of the R.I Prosecutor's Office have any

security loopholes. This test was performed using

the Sucuri SiteCheck web

. The test is

carried out by entering

the link of the online exam system into the

Sucuri SiteCheck web

then the application is run. The view of

the web sucuri SiteCheck

can be seen in figure

Figure 3.

Webview Sucuri SiteCheck

Source: https://sitecheck.sucuri.net/

Maintainability

characteristics testing is carried out using measures tested by direct

researchers operationally against the Badiklat online examination system of the R.I.

Prosecutor's Office In accordance with the testing instruments mentioned by Land, this

test includes 3 aspects, namely

instrumentation, consistency and simplicity.

The analysis for

maintainingability

characteristic testing according to the test

instrumentLand, (2002) is contained in table 4.

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1548

Table 4. Portability Characterization Testing

Source:Land, (2002).

Portability

characteristics testing uses

cross browsing compatibility testing

desktop

and

mobile

to test the quality of software running well on different browsers. To

maximize data collection and efficiency using limited resources and the time required,

this test uses the lambdatest tool. Figure 4 is a view of the lambdatest web.

Figure 4. Lambdatest Webview

Source: https://www.lambdatest.com/

Compatibility

Testing

the characteristics of compatibility

is carried out using

the PowerMapper

tool

by running the system in the

browser

, where the system in this

browser

can perform

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1549

(Formula

its respective functions properly without experiencing interference. The view of the

PowerMapper web

can be seen in figure 5.

Figure 5. PowerMapper webviews

Source: https://www.powermapper.com/

RESULT AND DISCUSSION

Functional Suitability

The results of this test are evidenced by the presentation of data in the form of a

table. The total tested functions run correctly and there are no constrained functions or

errors

. Such as the recapitulation of the test results shown in table 5.

Table 5. Recap of

Functional Sutability

Test Results

Source: Ancharya & Sinha, 2013.

The results of this test can be calculated with the

Matrix Formula Feature

Completeness

X = =

Then: X = = 1

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1550

(Formula

Based on these calculations, the result X = 1 is obtained. This means that the

quality of the R.I Prosecutor's Office's online examination system is declared good,

acceptable and meets

the functional suitability

aspect because the value of X is equal to

Reliability

testing the reliability

characteristics the results are as follows: success session

16635, session failed 0, successful page 16635, page failed 0, successful hit 16635, failed

hit 0. The results of testing using

WAPT 10.1

can be seen in figure 6.

Figure 6.

Reliability

Test Results using WAPT 10.1

Source: WAPT 10.1 Application, https://www.loadtestingtool.com/

The recap of the test results using

WAPT 10.1

is as follows:

Table 6. Total

test case

Test case

Sum

Successful sessions

16.635

Successful pages

16.635

Successful hits

16.635

Total

49.905

Source :

Wapt 10.1

recap results

Table 7. Total failure

(Failure)

Test case

Sum

Session failed

Page failed

Hit failed

Total

Source :

Wapt 10.1

recap results

Then Calculated use formula:

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1551

(Formula

The result, if expressed in the form of a percentage, is 100%. According to

Telcordia standards, the reliability success

the software is 95% or 0.95 . ThusAsthana

& Angela, (2009) , the R.I. Prosecutor's Office's Badiklat online examination system has

met the characteristics of

Reliability.

Usability

Testing

usability

characteristics through questionnaires, the results of respondents'

answers are obtained, then calculations can be made for the

Usability

characteristics test. A

recap of the results of the 86 respondents on the questionnaire on the online exam can

be seen in table 8.

Table 8. Recap of

Usability

Characterisitic Test Results

Source: Likert, (1932)

The data obtained from the respondents' answers were then calculated. Calculates

the percentage of the score by the following formula:

P =

then P =

so that P = 79.24%

The percentage result of testing

the characteristics of usability

is 79.24%, when

referring to the criteria of interpretation of the score Riduwan, (2010) . percentage of

61% - 80% declared Viable. So the R.I. Prosecutor's Office's Badiklat online examination

system was declared feasible and met the

Usability test.

Performance Efficiency

The results of testing

the characteristics of Performance Efficiency

can be seen

from figure 7.

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1552

Figure 7. Performance

efficiency

test results using GTMetrix

Source: https://gtmetrix.com/

From the results above, it can be concluded that although the overall test got

grade

D with a

performance

score of 63%, the result of the loading time got 3.9

seconds.

The results are then compared with a table on user satisfaction with the time response

proposed by . The web is said to beHoxmeier & DiCesare, (2000) good if the

load

time is

at least less than 10 seconds. From these results, it can be concluded that the badiklat

online examination system of the R.I Prosecutor's Office has met the characteristics of

Performance Efficiency

with a satisfied predicate.

Security

The display of the test results of

the Security

characteristics can be seen in figure 8.

Figure 8. Security Test Results using

Sucuri Sitehack

Source: https://sitecheck.sucuri.net/

Based on the results of

security

testing using

the Sucuri Sitehack web

, the Badiklat

online examination system of the R.I Prosecutor's Office has an intermediate or moderate

security risk, no malware was found

and was not declared on

the blacklist

, so it can be

concluded that this online exam system is in accordance with

security

characteristics so

that it is safe to use.

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1553

Maintainability

Testing

the characteristics of maintainability

based on operational tests of

instrumentation

aspects that the system has notifications, consistency aspects

of the

results the system has similarity in model design, and

the simplicity

aspect of the results

the system has a structured

source code

so that the badiklat online examination system

of the R.I Prosecutor's Office has met

the aspects of maintainability

Portability

Testing

portability

characteristics using

Lambdatest's web tool

. This test is carried

out in a wide variety of

desktop

and

mobile browsers.

The following results of testing

portability

characteristics can be seen in table 9.

Table 9.

Portability

Characterisitic Test Results

Source: Ristanto et al., (2020).

Based on the results of testing

portability

characteristics using

Lambdatest

, results

were obtained that the Badiklat Kejaksaan R.I online examination system performed well

in each browser and had met

the portability

criteria.

Compatibility

The results

of compatibility

testing using

PowerMapper software

can be seen in figure

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1554

Figure 9. Tool sucuri sitehack

Source: https://www.powermapper.com/products/sortsite/

Testing

compatibility

characteristics using the

PowerMapper

application on the

Badiklat Kejaksaan R.I online exam system with several browsers on both

desktop

and

mobile devices.

Compatibility test results

do have several major and minor issues on

Internet Explorer, Firefox, Safari, and IOS browsers. This problem occurs because the

CSS code used by the online exam system does not support it in that browser. However,

the results in the

critical issue

section show that there are no problems in all browsers,

so it can be concluded that the online exam system meets

the compatibility criteria.

Recommendations

Considering that the implementation of the Badiklat online exam of the R.I

Prosecutor's Office was attended by participants of the Prosecutor's Administrative

Technical Training (TAK) which numbered around 285 people,

the loading

of the exam

page reached 3.9 seconds on the standard performance

efficiency

characteristics could

still experience

a longer loading when accessed by training participants simultaneously.

Then it needs an evaluation for improvement and an increase in the

load speed of

the

exam page, namely:

1. Optimization of the code of the question data retrieval program carried out by the

participants.

2. Setting up servers, databases and internet networks more precisely and efficiently at

the Badiklat R.I Prosecutor's Office so that the implementation of the exam runs more

smoothly.

Perform

problematic CSS updates both from the CSS version used and from the

CSS code to make it more compatible with all types of browsers. And do CSS optimization

by bundled the code to a more minimalist size so that when loading CSS is lighter and

faster.

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1555

CONCLUSION

Based on the results of research and discussions that have been carried out using

the ISO 25010 standard method, the quality of the Badiklat online examination system

of the R.I Prosecutor's Office is quite decent and meets the criteria. With

the

characteristics of functional suitability

meets the standard because the value of X is equal

to 1. The reliability characteristics

have not failed, so it can be said to be very good. The

usability

characteristic gets a score of 79.24% then it is declared a worthy predicate.

Although

the performance efficiency

characteristics get grade D, but with a response time

of 3.9 seconds, it still gets the title of Satisfied. Security

characteristics

have medium or

moderate security risks. The

characteristics of maintainability

meet the three elements of

instrumentation, consistency

and

simplicity

so that it is easy to re-develop. The

portability

characteristics of the results can be run in any

browser

with a good appearance. The

compatibility

characteristics are compatible with some

desktop

and

mobile

devices, only

on IOS and Internet Explorer devices that have the problem of unsupported CSS code.

The problem is because some browsers do not support the CSS code used by the Badiklat

Kejaksaan R.I online exam system. However, the results in the

critical issue

section show

that there are no problems in all browsers, so it can be concluded that the online exam

system meets

the compatibility criteria.

Based on the results of research and discussions that have been carried out using

the ISO 25010 standard method, the quality of the online examination system of the R.I

Prosecutor's Office is quite feasible and meets the criteria.

REFERENCE

Andika, I. G., & Yanti, C. P. (2018). Evaluation Analysis of Augmented Reality Applications

for Balinese Cultural Information Based on ISO 25010 Standards.

Resistor Journal

(Computer Systems Engineering)

(1), 34–40.

Anggraini, N., Putra, M. J. D., & Hakiem, N. (2019). Development of an Islamic Higher

Education Institution Tracer Study Information System and It’s Performance Analysis

using ISO/IEC 25010.

2019 7th International Conference on Cyber and IT Service

Management (CITSM)

, 1–6.

Asthana, A., & Olivieri, J. (2009). Quantifying software reliability and readiness.

2009

IEEE International Workshop Technical Committee on Communications Quality and

Reliability

, 1–6.

Hoxmeier, J. A., & DiCesare, C. (2000).

System response time and user satisfaction: An

experimental study of browser-based applications

Analysis of The Quality of Web-Based Online Exams at The R.I. Prosecutor's Office Education

and Training Agency Using The ISO 25010 Standard

Indonesian Journal of Multidisciplinary Science

, Vol 1 (11), September 2022

1556

ISO/IEC. (2011).

Quality Models for Software Product Quality and System Quality in use

Software Product Quality Requirements and Evaluation (SQuaRE).

Izzatillah, M. (2019). Quality Measurement of Transportation Service Application GO-JEK

using ISO 25010 Quality Model.

Symmetrical: Journal of Mechanical Engineering,

Electrical And Computer Science

(1), 233–242.

Land, R. (2002). Measurements of software maintainability.

Proceedings of the 4th ARTES

Graduate Student Conference

, 1–7.

Larasati, S. S. A., Rusdianto, D. S., & Kurniawan, T. A. (2018). Construction of a Web-

Based Student Daily Examination System With Reference to ISO 25010 Quality

Standards.

Journal of Information Technology Development And Computer Science

E-ISSN

2548

, 964X.

Likert, R. (1932). A technique for the measurement of attitudes.

Archives of Psychology

Riduwan, M. B. A. (2010). Methods and techniques of compiling a thesis.

London:

Alfabeta

Ristanto, R. D., Kurniawati, K., Dwinanto, A., & Nawassyarif, N. (2020). ISO/IEC 25010

Product Quality Software Analysis on Aptitude Test Development Using Computer-

Based Test (CBT) System.

Edu Komputika Journal

(2), 49–60.

Sugiyono. (2020).

Quantitative, Qualitative, and R&D Research Methods

. Alphabet.

Tangkudung, I., Dako, R. D. R., & Dako, A. Y. (2019). Website Evaluation Using ISO/IEC

25010 Method.

SemanTECH (National Seminar on Technology, Science And

Humanities)

(1), 87–107.

Tian, J., Rudraraju, S., & Li, Z. (2004). Evaluating web software reliability based on

workload and failure data extracted from server logs.

IEEE Transactions on Software

Engineering

(11), 754–769.

Tyas, G., Purnamasari, D., & Suroso, A. (2021). E-Exam Application Quality Analysis Using

ISO 25010 Standard.

Journal of Informatics: Journal of IT Development

(2), 126–

132.

Yusnizar. (2022).

Designing an Online Examination System at the Center for Community

Learning Activities (PKBM) Riksa Ilmu Karawang

. Gunadarma University.