Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Evaluating and Comparing the Usability of Mobile Banking Applications in Saudi Arabia

Evaluating and Comparing the Usability of Mobile Banking Applications in Saudi Arabia Article Evaluating and Comparing the Usability of Mobile Banking Applications in Saudi Arabia 1,2 1, 1 1 Sarah Alhejji , Abdulmohsen Albesher *, Heider Wahsheh and Abdulaziz Albarrak The Department of Information System, College of Computer Sciences and Information Technology, King Faisal University, Hofuf 31982, Saudi Arabia The Department of Management Information Systems and Production Management, College of Business and Economics, Qassim University, Buraydah 52571, Saudi Arabia * Correspondence: [email protected] Abstract: In many countries, the rapid growth of the Internet and mobile technologies has led to the expansion of Internet banking, especially mobile banking. Many banks seek to provide integrated banking services through mobile applications (apps) to increase customer satisfaction and loyalty. A quick look at the reviews of the mobile banking apps in Saudi Arabia reveals different usability issues among these apps. This research analyzed, evaluated, and compared the usability of all Saudi mobile banking apps available for the iOS and Android systems. Usability (as defined by ISO 9241) was measured using three criteria—effectiveness, efficiency, and satisfaction. This research also identified and discussed the most critical weaknesses of the Saudi banks’ apps in regard to provid- ing satisfactory solutions to developers. The results showed that the most critical issues existed in the user interfaces and functionality of the apps, especially those that frequently received updates. Furthermore, the lack of customer support made the interaction between banks and customers weak, leading to customer dissatisfaction. Keywords: usability; user experience; mobile banking; customer review Citation: Alhejji, S.; Albesher, A.; Wahsheh, H.; Albarrak, A. Evaluating and Comparing the Usability of Mobile Banking 1. Introduction Applications in Saudi Arabia. Recently, the mobile app industry has boomed dramatically worldwide. The number Information 2022, 13, 559. of mobile app downloads increased from 140.68 billion in 2016 to 230 billion in 2022 [1]. https://doi.org/10.3390/info13120559 Rizk stated that mobile apps allow users to better share their feelings and opinions about Academic Editor: Norbert Fuhr the delivered services through writing reviews [2]. Reviews often contain valuable infor- mation for developers, as they can help them enhance and develop their apps to meet Received: 9 October 2022 users’ needs and desires. Specifically, reviews normally include requests for improve- Accepted: 25 November 2022 ment, users’ evaluations, bug reports, queries, or general descriptions of user experiences Published: 29 November 2022 that can be positive or negative. Publisher’s Note: MDPI stays neu- Srisopha et al., showed that app ratings and reviews are essential factors that users tral with regard to jurisdictional consider when choosing apps to download [3]. Moreover, the more positive comments claims in published maps and institu- and reviews an app has, the higher it ranks in the search results, which increases the tional affiliations. chances of appearing to potential users. Reviews can indicate customer satisfaction with the apps and the services provided. Therefore, studying an app’s reviews is an important part of the app life cycle and one of the most critical activities required of app developers in order to maintain and improve their app. Copyright: © 2022 by the authors. Li- A quick review of banking apps in Saudi Arabia showed that customers’ experiences censee MDPI, Basel, Switzerland. This article is an open access article were generally not satisfactory. Customer feedback and comments indicated many prob- distributed under the terms and con- lems with the banking services provided. Because of this, this research aimed to analyze, ditions of the Creative Commons At- evaluate, and compare the usability of mobile banking apps belonging to several Saudi tribution (CC BY) license (https://cre- banks. Usability was measured by the criteria set by ISO 9241, which are effectiveness, ativecommons.org/licenses/by/4.0/). efficiency, and satisfaction. Additionally, this research aimed to identify the most critical Information 2022, 13, 559. https://doi.org/10.3390/info13120559 www.mdpi.com/journal/information Information 2022, 13, 559 2 of 14 problems and weaknesses faced by Saudi banking apps in regard to providing satisfactory solutions to developers. The results obtained were used to determine which apps had the best usability and the most critical problems and weaknesses, as well as the types of im- provements that service providers should make to enhance mobile banking. The rest of the paper is organized as follows. The second section reviews the litera- ture related to this research. The third section includes detailed information about the re- search methodology used. The fourth section provides an analysis of the data and a re- view of the research findings. Finally, the fifth section reviews the main issues encoun- tered with mobile banking in Saudi Arabia, followed by a set of recommendations for developers. 1.1. Mobile Banking Applications Mobile apps are essential components of modern information and communication technology. They allow users to have quick and easy access to the products, services, in- formation, and processes they need in real time. The global use of modern technologies has changed interactions between business owners and customers, contributing to in- creasing customer loyalty and satisfaction. The development of apps requires an under- standing of the target users in terms of their requirements, goals, and ideas. Ali et al. stated that the app industry requires more developer attention to develop apps for ease of use and reliability [4]. Ghandi et al., noted that mobile app development requires continuous improvements to meet new technological needs, such as the design of user interfaces in different sizes to fit the screens of mobile devices [5]. Many businesses have taken ad- vantage of mobile apps to meet the needs of users and thus increase customer satisfaction and loyalty, including mobile healthcare apps (m-health), mobile learning apps (m-learn- ing), and mobile banking apps (m-banking). Customers can use m-banking to pay bills, transfer money, manage accounts, inquire about bank information, and search for ATM locations [6]. M-banking improves customer time management through instant communication and access to information, as well as its ability to be used anywhere. Because of this, it helps improve the customers’ quality of life and increase the efficiency of banks [7]. Good-quality m-banking services can also help to retain and attract customers [8]. Furthermore, improvements in the provision of m- banking services contribute to increasing the bank’s market share, reducing the cost of failure, and lowering the costs of business and attracting new customers to the bank. With the rapid advancement of mobile technology, many banking customers in Saudi Arabia find it easy to use m-banking to make many financial transactions. Al-Khalidi noted that m-banking is the fastest-growing channel for financial growth in Saudi Arabia [9]. Al-Khalidi also explained that the spread of m-banking services is expected to con- tinue in Saudi Arabia as the Internet infrastructure is modernized, government projects are implemented, and banking transactions and payment networks are strengthened and developed [9]. 1.2. Customer Reviews A customer review is a comment regarding a product or service written by a con- sumer who has used or experienced it. These reviews and ratings are public, which means developers and other users can read and benefit from them. As customers search online for product information and compare product options, they often have access to dozens or hundreds of product reviews written by other customers. The reviews and ratings can play an essential role in making purchase decisions. Askalidis and Malthouse indicated that 30% of customers under 45 years of age write reviews for every purchase they make, and 86% of customers say that those reviews are essential to making purchase decisions [10]. According to Avant, a company needs to handle user suggestions and complaints to survive, stay in the market, and gain customer loyalty [11]. Customer loyalty is critical, as retaining existing customers is more expensive than acquiring new customers. According to Zhang and Mao, the hotel sector has experienced a decline in customer loyalty because Information 2022, 13, 559 3 of 14 reading online reviews has increased customer understanding and purchasing power [12]. Avant found that when hotels responded personally to customer complaints and re- quests, the guest retention rate was 85% or higher, while hotels that did not respond to customer demands retained around 30% of their customer base [11]. Therefore, imple- menting a customer–company interaction strategy is an effective way to maintain cus- tomer loyalty. 1.3. Usability According to the ISO 9241-11 standard (ISO, 2018), usability is defined as “the extent to which a system, product, or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” As is stated in this definition, three factors (effectiveness, efficiency, and satisfaction) can affect usability [13]. ISO 9241-11 defines these factors as follows: Effectiveness: the accuracy and completeness with which users achieve specified goals. Efficiency: the resources used in relation to the results achieved. User satisfaction: the extent to which the user’s physical, cognitive, and emotional responses that result from the use of a system, product, or service meet the user’s needs and expectations. Usability and its associated factors have a decisive and powerful impact on the suc- cess of any system, website, or mobile app. Alber et al. defined efficiency as a measure of effectiveness that results in the minimum waste of time and effort [14]. Groth and Has- lwanter linked efficiency with time, wherein they clarified the importance of time in meas- uring users’ efficiency during the performance of tasks [15]. Similarly, mobile effectiveness is a significant factor in m-banking apps. Alber et al. defined this factor as the extent to which a goal is reached without regard to the method and resources for optimal use [14]. Groth and Haslwanter also linked effectiveness with accomplishing a task; when users fail to complete a simple task, this may be strong evi- dence that a bug in the application needs to be fixed [15]. Satisfaction is also an important factor that influences mobile app use. Lee et al., de- fined satisfaction as a brief emotional response from a mobile phone user [16]. Ravendran stated that satisfied m-banking users are more likely to buy from their banks than dissat- isfied users [17]. Customers who are satisfied with services come back and buy again, telling others about their experiences. In contrast, customers who are deeply dissatisfied with services leave, while customers who are poorly satisfied with services may not leave but may complain [18]. In general, usability is essential and is considered one of the most important features of apps and software. One of the main reasons for the failure of apps and software is the need for a system to achieve the set goals of the users and measure their satisfaction. For this reason, usability assessment has become an essential part of any app or software de- velopment [19]. There are different methods to assess usability, the most famous of which is the System Usability Scale (SUS) and Sentiment Analysis (SA). On the other hand, the User Experience Questionnaire (UEQ) is one of the most popular measures for compre- hensive user experience measurement. The SUS is one of the most widely used usability testing tools today. The SUS scale was designed as a quick and easy way to assess usability [20]. The SUS scale consists of a set of data, ten verified data, covering five negative and five positive aspects of the system. Participants are asked to record each of the five questions. SUS scores can be grouped into percentage ranges [21], or a grading system can be used. As for the UEQ measurement, it serves as a means to evaluate and measure the entire user experience. The UEQ scale is based on six scales, such as attractiveness, perspicuity, dependability, efficiency, stimulation, dependability, and novelty [20]. Scales are evalu- ated using pairs of opposite adjectives to describe the system, with participants choosing Information 2022, 13, 559 4 of 14 their level of agreement with each. UEQ scores assess how well a system meets users’ expectations. 1.4. Sentiment Analysis (SA) SA is a process that relies on identifying, classifying, and mathematically processing textual data to obtain the opinions and perspectives of users regarding the topics, services, and products offered to them [22]. Devika et al., mentioned another definition of SA, a technique used to extract positive and negative user opinions about products or services offered [23]. These data are represented as customer feedback stored within forums, blogs, and social media. Several methods can be used to gain user feedback, ranging from human analysis to machine learning. SA determines the polarity of data, which is news or product reviews. There are multiple ways of expressing emotions, from the three most common levels of polarity, positive, neutral, and negative, to scales of polarity, which are set, for example, from −10 to +10. Luo et al., reported that opinion words dominate emotion indicators, particularly in adjectives, adverbs, and verbs, for example, “I love this app; it is amazing!” Opinion words are also known as emotion words, polar words, opinion lexicon, or opinion-carry- ing words, which can be categorized into two types: positive words such as cool, elegant, excellent, and negative words such as terrible, disgusting, and poor [24]. The feature extraction step is depended on SA frequencies to learn about the polarity of different reviews and comments. Weight scores for sentiment features are used to de- termine the strength and polarity of each review and comment. All sentiment fea- tures/words used in this study are manually extracted from comments and reviews col- lected from Saudi bank applications. TF (term frequency) refers to the number of times a given term (comment/review) occurs and is repeated. Then, each sentiment feature/word is weighed manually. The tool relies on the frequency of positive and negative terms/fea- tures to determine the polarity of the reviews. A review is considered positive when the frequency of positive terms/features exceeds the frequency of negative terms/features in the same review. A review is considered negative when the frequency of negative terms/features exceeds that of positive terms/traits in the same review. Finally, the review tool is considered neutral if the frequency of positive features/terms in the review equals the frequency of negative features/terms. Scores in polarity dictionaries are used by the tool to determine the strength of each entry [25]. 2. Related Work Omotosho analyzed reviews written by users of m-banking apps in Nigeria to extract valuable insights regarding the sentiments and emotions expressed by the users. The study found that around 66% of the emotions expressed by users were associated with anticipation, joy, and trust, whereby the remaining 34% were related to fear, disgust, sur- prise, and anger [26]. Tabiaa and Madani used online user evaluations to analyze the voice of the customer and to construct a topic modeling approach based on that data. Security, services, quality, and the user interface were the most common topics observed [27]. Permana et al., conducted a sentiment analysis and m-banking app review topic de- tection in Indonesia to determine customer sentiment toward m-banking apps and to learn which aspects of the examined apps needed to be maintained and improved. The most frequent topics observed among the negative reviews were app login problems, OTP code delivery constraints, and network connections. On the other hand, simplicity, help- fulness, and ease-of-use were the most frequent topics among the positive reviews [28]. Oh and Kim proposed a text-mining approach to identify factors that improved customer satisfaction when using m-banking applications. Their study showed that positive re- sponses regarding the security and convenience of m-banking apps improved the rating of apps in stores. In contrast, increasing comments about insecurity, negative customer support experiences, and sophistication correlated with lower user ratings. These results Information 2022, 13, 559 5 of 14 support the idea that security is the most influential factor in customer satisfaction with m-banking services [29]. Metlo et al., conducted an empirical analysis to study the effect of m-banking on cus- tomer satisfaction in the Pakistani banking sector. The results showed that ease of use, credibility, and customer attitude significantly influenced customer satisfaction with the banking services provided [30]. Mkpojiogu et al., studied demographic differences in user satisfaction with the usability of m-banking apps. The results showed significant differ- ences in the satisfaction of m-banking users based on gender, age, educational qualifica- tions, and experience. These results are helpful for banks, as they can help improve the interfaces of m-banking apps to better meet users’ needs [8]. Gomachab and Maseke stud- ied the effects of m-banking on customer satisfaction in commercial banks in Namibia. The results showed that the most-used service provided by the apps was the ability to make airtime purchases, and the least-used service was the allocation of funds [31]. Kaya et al. measured the usability of mobile apps using the SUS. This study aims to reveal the usability difference between four commonly used mobile apps: WhatsApp, Fa- cebook, YouTube, and Mail. The study also looks for the difference in usability between iOS and Android operating systems. In this study, the SUS with an Adjective Rating Scale was applied to 222 young participants using these apps on their mobile phones. The re- sults showed that the usability of all apps is somewhat satisfactory and above the stand- ards. In more detail, the results showed that the usability of WhatsApp is higher compared to other apps, while the Facebook app has the lowest score. In addition, according to the results, there is no difference in the usability of mobile apps between operating systems [32]. Kristanto et al., assessed the usability of Ruang Guru apps using a UEQ. This survey focuses on user satisfaction, measured using a questionnaire with the Don Foundation Standard. In this study, measurements made on 100 active users of the Ruang Guru apps using the questionnaire were used randomly. The results of the measurements showed a degree of effectiveness of 59.38, efficiency of 65.36, and a degree of satisfaction of 62.52 [33]. 3. Materials and Methods The text data examined in this study consisted of reviews submitted by users of m- banking apps provided by 11 Saudi banks (data are available in a publicly accessible re- pository). These banks were Alinma Bank, Riyad Bank, Al-Rajhi Bank, SABB Bank, Al- Ahly Bank, Al-Fransi Bank, Al-Jazira Bank, Al-Bilad Bank, Arab Bank, Samba Bank, and Investment Bank. This research applied sentiment analysis to analyze reviews written in English and Arabic. The main goal of using SA was to identify the polarity of users’ views about different aspects of app usage [34]. Moreover, the SA method is distinguished from others in that it classifies, identifies, and analyzes a large set of data without the need to identify participants or create tools for analysis, such as questionnaires and personal in- terviews. It is also an open method that allows any user to express his/her opinion directly at any time. In contrast, both SUS and UEQ need to create specific evaluation tools, such as questionnaires directed to a specific number of participants and containing specific questions asked by the analysts. The schematic overview of our approach is exhibited in Figure 1. Information 2022, 13, 559 6 of 14 Figure 1. Review analysis schema. The methods used for data collection and processing are explained below. A. The collection of data was performed using Heedzy, an online tool that allows for the downloading of mobile app reviews and ratings for Android and iOS. This study collected 5958 reviews from Android users and 2438 reviews from iOS users. The data were collected between January and March 2022. The next examples are shown from the collected reviews from the m-banking apps in Saudi Arabia. Consider the following samples of the collected English reviews (original English re- view): Example 1: The new update is bad, and the transaction process is slow and sometimes give errors. Example 2: It is very good and quick B. Three polarity lexicons were constructed manually based on our selected usability factors—satisfaction, effectiveness, and efficiency. El-Halees built lexicons that used opinion phrases and words to determine the sentiment orientation of the whole re- view [19]. Our lexicon included 491 words describing satisfaction (278 positive, 213 negative), 265 describing effectiveness (11 positive, 254 negative), and 65 words de- scribing efficiency (12 positive, 53 negative). Table 1 shows that the satisfaction factor contained more positive words than the effectiveness and efficiency factors. At the same time, satisfaction and effectiveness factors had negative words in close propor- tions and were much higher than the efficiency factor. Table 1. Summary of polarity distribution among the usability factors. Usability Factors No. of Positive Polarities No. of Negative Polarities Satisfaction 278 (65.5%) 213 (43.3%) Effectiveness 11 (2.2%) 254 (51.7%) Efficiency 12 (2.4%) 53 (10.8%) Moreover, Tables 2–4 present a sample of positive and negative polarity words of the usability factors extracted from Saudi bank reviews. These features are stored in the lexi- cons for later use to determine the polarity of reviews. Information 2022, 13, 559 7 of 14 Table 2. Satisfaction lexicon (English and Arabic words). Satisfaction Lexicon Positive Negative Arabic English Arabic English زﺎﺘﻤﻣ Excellent ءﻲﺳ Bad ﺪﯿﺟ Good ﻢﯾﺪﻗ Old Table 3. Effectiveness lexicon (English and Arabic words). Effectiveness Lexicon Positive Negative Arabic English Arabic English لﺎﻌﻓ Effective ﻞﻤﻌﯾ ﻻ Not working ﻞﻤﻌﯾ Working perfectly. ﺄﻄﺧ Error Table 4. Efficiency lexicon (English and Arabic words). Efficiency Lexicon Positive Negative Arabic English Arabic English ﺖﻗو ﺮﻓﻮﯾ quick ءﻲﻄﺑ Slow ﻊﯾﺮﺳ Fast ﻖﻠﻌﯾ Hanging C. The reviews of each bank were labeled manually based on usability factors. Groth and Haslwanter found three traits that measured usability in any review of mobile usability models: effectiveness, efficiency, and satisfaction [15]. A polarity score was given by the authors’ judgment for each factor, a quantitative measure of the positive or negative interactions expressed in the reviews. The average polarity score ranged from −1 to +1, with negative values indicating negative opinions, values of zero indi- cating neutral opinions, and positive values indicating positive opinions. D. The total usability score was calculated for each review based on the sum of each review’s satisfaction, effectiveness, and efficiency values [19]. The overall usability score ranged from −3 to +3. The usability of all banks was evaluated and compared based on these scores in order to determine which of the banks had the highest and lowest usability. Based on example 1 (step A), Table 5 exhibits how to give a polarity score for each word in these reviews that match the lexicons built. Then, the total usability score was calculated. Table 5. Manually extracted features with its polarity weight and calculated usability score (exam- ple 1). Usability Review Satisfaction Effectiveness Efficiency Score The new update is bad, and the transaction pro- −1 −1 −1 −3 cess is slow and some- times gives errors. Information 2022, 13, 559 8 of 14 4. Results and Discussion After arranging the reviews into separate excel sheets for each operating system, we categorized and determined the degree of each usability factor (satisfaction, effectiveness, efficiency). The usability score for each Saudi bank was calculated by summing the per- centage of positive and negative reviews for each usability factor. Tables 6 and 7 show the following data for Android and iOS, respectively: the name of the bank, the number of reviews, the percentage of positive and negative reviews for each usability factor, and the overall usability scores. Table 6 shows the usability scores of all Saudi bank apps on the Android system based on the values of our selected usability factors. These are satisfaction, effectiveness, and efficiency (both positive and negative). The Alinma Bank app demonstrated the high- est positive usability ratio, calculated at 0.81, followed by the Riyad Bank app with a score of 0.72, the Al-Rajhi Bank app with a score of 0.71, the SABB Bank app with a score of 0.69, and Al-Ahli Bank app with a score of 0.34. Meanwhile, the Investment Bank app possessed the highest negative usability ratio, calculated at −0.93, followed by the Samba Bank app with a score of −0.49, the ANB Bank app with a score of −0.34, the Al-Bilad Bank app with a score of −0.30, the Al-Jazira Bank app with a score of −0.28, and the Al-Fransi Bank app with a score of −0.25. Table 7 shows the usability scores for all Saudi banks in the iOS system. The Al-Rajhi Bank app demonstrated the highest positive usability ratio, calculated at 0.77, followed by the Alinma Bank app with a score of 0.03. Meanwhile, the Al-Fransi Bank app had the highest negative usability ratio, calculated at −1.38, followed by the Al-Ahli Bank app with a score of −1.32, the Samba Bank app with a score of −1.13, the Al-Jazira Bank app with a score of −1.10, the Investment Bank app with a score of −1.00, the Al-Bilad Bank app with a score of −0.94, the ANB Bank app with a score of −0.78, the SABB Bank app with a score of −0.35, and the Riyad Bank app with a score of −0.23. Tables 6 and 7 show that the satisfaction factor plays an essential role in determining the degree of usability. This was followed by the factors of effectiveness and finally effi- ciency. Among Android users, the Alinma Bank app had the highest satisfaction rate among banking applications, calculated at 0.877, followed by the Al Rajhi Bank app with a score of 0.825, and then the Riyad Bank app with a score of 0.823. Meanwhile, the In- vestment Bank app had the lowest satisfaction rate among all bank applications, with a calculated rate of −0.465, followed by the Samba Bank app with a score of −0.448, and the ANB Bank app with a score of −0.275. Among iOS users, the Al-Rajhi Bank app had the highest satisfaction rate, calculated at 0.808, followed by the Alinma Bank app with a score of 0.503, and then the Riyad Bank app with a score of 0.422. Meanwhile, the Samba Bank app had the lowest satisfaction rate, calculated at −0.771, followed by the Al-Fransi Bank app with a score of −0.676, and then the Al-Ahli Bank app with a score of −0.543. Information 2022, 13, 559 9 of 14 Table 6. Usability score (Android system). Satisfaction Effectiveness Efficiency Bank Total Re- No. of No. of No. of Usability 4 3 2 1 Name views Negative Score % Positive Negative Negative Score % Positive Negative Negative Score % Positive Negative Score Reviews Reviews Reviews Alinma 405 12 343 0.847 0.877 −0.030 21 −20 −0.049 0.002 −0.052 8 6 0.015 0.035 −0.020 0.81 Bank Riyad 962 48 744 0.773 0.823 −0.050 77 −69 −0.072 0.008 −0.080 19 13 0.014 0.033 −0.020 0.72 Bank Al-Rajhi 1240 43 980 0.790 0.825 −0.035 63 −63 −0.051 0 −0.051 50 −40 −0.032 0.008 −0.040 0.71 Bank SABB 964 60 708 0.734 0.797 −0.062 79 −77 −0.080 0.002 −0.082 16 31 0.032 0.049 −0.017 0.69 Bank Al-Ahli 1085 132 556 0.512 0.634 −0.122 177 −176 −0.162 0.001 −0.163 22 −12 −0.011 0.009 −0.020 0.34 Bank Al-Fransi Android 142 29 31 0.218 0.423 −0.204 52 −52 −0.366 0 −0.366 15 −14 −0.099 0.007 −0.106 −0.25 Bank Al-Jazira 326 88 56 0.172 0.442 −0.270 102 −102 −0.313 0 −0.313 49 −45 −0.138 0.012 −0.150 −0.28 Bank Al-Bilad 305 58 63 0.207 0.397 −0.190 123 −122 −0.400 0.003 −0.403 34 −33 −0.108 0.003 −0.111 −0.30 Bank ANB Bank 353 97 47 0.133 0.408 −0.275 155 −155 −0.439 0 −0.439 19 −12 −0.034 0.020 −0.054 −0.34 Samba 105 47 −16 −0.152 0.295 −0.448 26 −26 −0.248 0 −0.248 9 −9 −0.086 0 −0.086 −0.49 Bank Invest- ment 71 33 −21 −0.296 0.169 −0.465 26 −25 −0.352 0.014 −0.366 20 −20 −0.282 0 −0.282 −0.93 Bank 1 2 Negative: The number of negative reviews divided by the total number of reviews. Positive: The number of positive reviews divided by the total number of 3 4 reviews. %: The score is divided by the total number of reviews. Score: The number of positive reviews minus the number of negative reviews. Information 2022, 13, 559 10 of 14 Table 7. Usability score (iOS system). Satisfaction Effectiveness Efficiency Total Re- No. of Neg- No. of Usability Bank Name No. of Nega- views Score % Positive Negative ative Re- Score % Positive Negative Negative Score % Positive Negative Score tive Reviews views Reviews Al-Rajhi 668 26 514 0.769 0.808 −0.039 35 −33 −0.049 0.003 −0.052 9 33 0.049 0.063 −0.013 0.77 Bank Alinma 183 40 52 0.284 0.503 −0.219 38 −37 −0.202 0.005 −0.208 18 −9 −0.049 0.049 −0.098 0.03 Bank Riyad Bank 429 106 75 0.175 0.422 −0.247 122 −122 −0.284 0 −0.284 64 −52 −0.121 0.028 −0.149 −0.23 SABB Bank 129 30 16 0.124 0.357 −0.233 50 −50 −0.388 0 −0.388 15 −11 −0.085 0.031 −0.116 −0.35 ANB Bank 156 53 −23 −0.147 0.192 −0.340 87 −87 −0.558 0 −0.558 15 −12 −0.077 0.019 −0.096 −0.78 Al-Bilad 81 30 −21 −0.259 0.111 −0.370 45 −45 −0.556 0 −0.556 12 −10 −0.123 0.025 −0.148 −0.94 Bank iOS Investment 86 35 −29 −0.337 0.070 −0.407 27 −26 −0.302 0.012 −0.314 31 −31 −0.360 0 −0.360 −1.00 Bank Al-Jazira 155 41 −30 −0.194 0.071 −0.265 101 −101 −0.652 0 −0.652 40 −39 −0.252 0.006 −0.258 −1.10 Bank Samba Bank 48 37 −36 −0.750 0.021 −0.771 16 −16 −0.333 0 −0.333 2 −2 −0.042 0 −0.042 −1.13 Al-Ahli 466 253 −237 −0.509 0.034 −0.543 229 −229 −0.491 0 −0.491 148 −147 −0.315 0.002 −0.318 −1.32 Bank Al-Fransi 37 25 −24 −0.649 0.027 −0.676 21 −21 −0.568 0 −0.568 6 −6 −0.162 0 −0.162 −1.38 Bank Information 2022, 13, 559 11 of 14 From Tables 6 and 7, we can see that opinions regarding effectiveness were weighted toward negative in reviews of all Saudi bank applications. Among Android users, the ANB Bank app had the lowest effectiveness rate among all Saudi bank applications, with an estimated rate of −0.439, followed by the Al-Bilad Bank app with a score of −0.403, and then the Al-Fransi and Investment Bank apps, both with scores of −0.366. Among iOS us- ers, the Al-Jazira Bank app possessed the lowest effectiveness rate, with a calculated value of −0.652, followed by the Al-Fransi Bank app with a score of −0.568 and the ANB Bank app with a score of −0.558. In addition, Tables 6 and 7 revealed that efficiency scores were often close to the neu- tral zero. Among Android users, the SABB Bank app possessed the highest efficiency ratio of the banking applications, with a calculated percentage of 0.049, followed by the Alinma Bank app with a score of 0.035, the Riyad Bank app with a score of 0.033, and the ANB Bank app with a score of 0.020. Meanwhile, the Investment Bank app possessed the lowest efficiency rate of the banking applications, with a calculated percentage of −0.282, fol- lowed by the Al-Jazira Bank app with a score of −0.150, the Al-Bilad Bank app with a score of −0.111, and the Fransi Bank app with a score of −0.106. Among iOS users, only one bank, the Al-Rajhi Bank, possessed a high efficiency rate, with a score of 0.063. The Investment Bank app once again had the lowest efficiency rate, this time calculated at −0.360, followed by the Al-Ahli Bank app with a score of −0.318 and the Al-Jazira Bank app with a score of −0.258. Our research indicated various issues that affected usability from the perspectives of customer satisfaction, effectiveness, and efficiency. Our quantitative analysis led us to find common patterns that the banks have regarding these aspects and thus complement the results obtained from the usability evaluation. We found that these issues mainly stemmed from new updates, disabled functions, and lack of customer support. New Update Problem One of the commonly repeated problems that affected customers’ satisfaction with the Saudi m-banking apps on both the Android and iOS was the issue of new app updates. For example, the comment “problem with the new update” appeared frequently in re- views of the Al-Jazira Bank app, comprising an estimated 28.5% of reviews on Android and 24.5% on iOS. In regard to the Investment Bank app, the problem appeared with a frequency of 11% on Android and 11.7% on iOS. Meanwhile, 9.6% of reviews regarding the ANB Bank app on iOS referred to the new update issue using the repeated comment “this app does not work on a jailbreak device,” despite the evidence that their device was jailbreak-free. Update issues have also been found in apps in other domains, such as health care. For example, Alqahtani and Orji mentioned that users of some mental health apps explained that a new update caused some problems with the app’s functions that led to data loss [35]. In the m-banking domain, Tabiaa and Madani also reported frequent issues with new updates [27]. Functional Problems The main objective of using m-banking apps is to complete banking transactions in a quick and easy manner. Baabdullah et al., mentioned that Saudi customers view m-bank- ing as a method that saves customers time, money, and effort [36]. A strong relationship exists between the actual use of banking services and customer loyalty [36]. They indi- cated that the implementation of m-banking functions is essential not only for customer loyalty but also for increasing customer satisfaction [36]. In regard to the effectiveness factor, the apps reviewed in this research showed that customers complained about the disabling of some app functions by some Saudi banks. Commonly repeated comments regarding this issue included “the application does not work” representing 15% of comments, “cannot log in” 13% of comments, “the application cannot be opened” 11% of comments, and “there is a general error” 8% of comments. The ANB Bank apps for both Android and iOS systems, the Al-Bilad Bank and the Investment Bank apps for the Android system, and the Al-Jazira Bank and the Al-Fransi Bank apps for the iOS system were ranked as the worst apps regarding this issue. Similar issues were Information 2022, 13, 559 12 of 14 found in other research papers. For example, Permana et al., conducted a study on m- banking apps in Indonesia; their study showed that the most common issues involved signing in to an app [28]. Lack of Customer Service Support Saudi banking apps should provide different methods to communicate with custom- ers. One possible method is via an instant chat, which only the SABB Bank app provided. Meanwhile, the apps belonging to the Al-Rajhi, Al-Fransi, and Alahli banks did not pro- vide an instant chat, but they did allow customers to send their queries and complaints through the apps. Many reviews of the Saudi banking apps contained questions and in- quiries from customers directed to the app developers. These reviews, which represent 7% of all comments, include a range of in-app complaints and questions about how to use the app, the contents of new updates, and how to activate certain services. Some reviews that included a specific complaint about the app were answered by app developers with short responses—for example, “Thank you for your note; the problem will be resolved as soon as possible.” Neglecting to respond to customers can negatively affect customer sat- isfaction. Banks should respond to customers swiftly and direct them to proper solutions. Sharma and Sharma emphasized the need for a well-trained staff who can listen to, un- derstand, and handle customers’ problems [37]. Based on our analysis of users’ positive and negative reviews in Saudi banks apps, problems and themes were extracted from our results. We recommend the following to developers in order to improve the usability of Saudi banking apps: • Examine new updates before they are officially released and ensure that they are free from errors and problems. Many reviews indicated that customers were not satisfied with banking apps due to issues with new updates, which caused issues such as ap- plications that stopped working and updates that were not compatible with the user’s mobile device. • Analyze app reviews to improve apps and keep customers satisfied. Reviews are shared spaces that allow customers to express their opinions, requests, and com- plaints regarding downloaded apps. These reviews allow developers to receive cus- tomer feedback about app usability issues. If these reviews are taken into account and the developers fix the problems, it will significantly improve the apps. • Respond quickly to app reviews to increase customer satisfaction. Developers’ re- sponses to customer comments increase customer satisfaction and loyalty. These re- sponses make the customer feel that their feedback is important, and that the devel- oper is keen to solve their problems. Furthermore, responding to reviews has positive results—ratings of the app often increase after developers respond to customer re- views. • Enable a live chat function that can support interaction between banks and their cus- tomers. This would allow for an effective channel of communication that could re- spond to user requests and fix recurring issues in real time. 5. Conclusions Customer feedback and comments on m-banking apps belonging to some Saudi banks indicate many issues in the banking services provided. This study aimed to evalu- ate and compare the usability of all Saudi m-banking apps for Android and iOS based on three usability factors: satisfaction, effectiveness, and efficiency. This research identified several usability issues and recommended some useful solutions. Saudi banks should ex- amine and review new updates before they are issued, ensure that the essential functions of the apps work, and consider adding online chat features for customer service. The dif- ficulties this study encountered are represented in the study and analysis of reviews in the Arabic language, as the Arabic language is considered difficult because it is a highly inflectional and derived language. Moreover, some comments include words in the Saudi dialect, which are somewhat inconsistent with the Arabic language, in addition to the fact Information 2022, 13, 559 13 of 14 that these words do not have a particular lexicon. Additionally, some confusing words were excluded from this study, including positive words in some regions of Saudi Arabia and negative ones in others. Some comments include positive words and negative sym- bols that are difficult to classify; thus, they were excluded. The iOS and Android app stores recently began allowing developers to respond to customer reviews. As part of our future work, we plan to study and analyze the responses of app developers to customer reviews, evaluate the quality of those responses, and de- termine whether the developer’s response affects the customer’s experience. Moreover, a model can be created for this work so that reviews are categorized and analyzed automat- ically instead of by the manual method used. Author Contributions: Data curation, S.A.; Formal analysis, S.A.; Methodology, S.A.; Supervision, A.A. (Abdulmohsen Albesher), H.W. and A.A. (Abdulaziz Albarrak); Writing—original draft, S.A.; Writing–review and editing, A.A. (Abdulmohsen Albesher) and H.W. All authors have read and agreed to the published version of the manuscript. Funding: This work was supported by the Deanship of Scientific Research, Vice Presidency for Graduate Studies and Scientific Research, King Faisal University, Saudi Arabia (grant no. 1074), through its KFU Research Summer initiative. Data Availability Statement: Data available in a publicly accessible repository. These data can be found here: (https://drive.google.com/drive/folders/1ZdG5efrKos0p0-hHT8BsO3uKvC4AOdnu ac- cessed on 9 October 2022). Conflicts of Interest: The authors declare no conflict of interest. References 1. Number of Mobile App Downloads Worldwide from 2016 to 2021 (In Billions). In Statista—The Statistics Portal. Available online: https://www.statista.com/statistics/271644/worldwide-free-and-paid-mobile-app-store-downloads/ (accessed on 5 April 2022). 2. Rizk, M.; Ebada, A.; Nasr, S. Investigating Mobile Applications’ Requirements Evolution through Sentiment Analysis of Users. In Proceedings of the International Computer Engineering Conference, Cairo, Egypt, 29–30 December 2015. 3. Srisopha, K.; Link, D.; Boehm, B. How should developers respond to app reviews? features predicting the success of developer responses. In Proceedings of the Evaluation and Assessment in Software Engineering, Trondheim, Norway, 21–23 June 2021, pp. 119–128. 4. Ali, W.; Riaz, O.; Mumtaz, S.; Khan, R.; Saba, T.; Bahaj, S. Mobile Application Usability Evaluation: A Study Based on Demog- raphy. IEEE Access 2022, 10, 41512–41524. 5. Ghandi, L.; Silva, C.; Martínez, D.; Gualotuña, T. Mobile application development process: A practical experience. In Proceed- ings of the 2017 12th Iberian Conference on Information Systems and Technologies (CISTI), Lisbon, Portugal, 21–24 June 2017; pp. 1–6. 6. Afshan, S.; Sharif, A. Acceptance of mobile banking framework in Pakistan. Telemat. Inform. 2016, 33, 370–387. https://doi.org/10.1016/j.tele.2015.09.005. 7. Malaquias, R.; Hwang, Y. An empirical study on trust in mobile banking: A developing country perspective. Comput. Hum. Behav. 2016, 54, 453–461. 8. Mkpojiogu, E.; Hashim, N.; Adamu, R. Observed demographic differentials in user perceived satisfaction on the usability of mobile banking applications. In Proceedings of the Knowledge Management International Conference, Chiang Mai, Thailand, 29–30 August 2016. 9. Alkhaldi, A. Adoption of mobile banking in Saudi Arabia: An empirical evaluation study. Int. J. Manag. Inf. Technol. 2016, 8, 1– 10. Askalidis, G.; Malthouse, E. The value of online customer reviews. In Proceedings of the 10th ACM Conference on Recom- mender Systems, Boston, MA, USA, 15–19 September 2016; pp. 155–158. 11. Avant, T. Responding to TripAdvisor: How Hotel Responses to Negative Online Reviews Effect Hotel Image, Intent to Stay, and Intent to Return; University of South Carolina: Columbia, SC, USA, 2013. 12. Zhang, J.; Mao, Z. Image of all hotel scales on travel blogs: Its impact on customer loyalty. J. Hosp. Mark. Manag. 2012, 21, 113– 13. ISO. (n.d.). Available online: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed 2:v1:en (accessed on 1 June 2022). 14. Alber, N.; Elmofty, M.; Kishk, I.; Sami, R. Banking efficiency: Concepts, drivers, measures, literature and conceptual model. In Drivers, Measures, Literature and Conceptual Model; SSRN: Rochester, NY, USA, 2019. http://dx.doi.org/10.2139/ssrn.3310982. 15. Groth, A.; Haslwanter, D. Efficiency, effectiveness, and satisfaction of responsive mobile tourism websites: A mobile usability study. Inf. Technol. Tour. 2016, 16, 201–228. Information 2022, 13, 559 14 of 14 16. Lee, D.; Moon, J.; Kim, Y.; Mun, Y. Antecedents and consequences of mobile phone usability: Linking simplicity and interactiv- ity to satisfaction, trust, and brand loyalty. Inf. Manag. 2015, 52, 295–304. 17. Ravendran, R. Improving Usability of e-Commerce Websites via Tag-Based Customization: A Study on Online and Mobile Banking. Doctoral Dissertation, Queensland University of Technology, Brisbane, Queensland, 2013. 18. Bayraktar, E.; Tatoglu, E.; Turkyilmaz, A.; Delen, D.; Zaim, S. Measuring the efficiency of customer satisfaction and loyalty for mobile phone brands with DEA. Expert Syst. Appl. 2012, 39, 99–106. 19. El-Halees, A. Software Usability Evaluation Using Opinion Mining. J. Softw. 2014, 9, 343–349. https://doi.org/10.4304/jsw.9.2.343- 20. Holmes, S.; Moorhead, A.; Bond, R.; Zheng, H.; Coates, V.; McTear, M. Usability testing of a healthcare chatbot: Can we use conventional methods to assess conversational user interfaces? In Proceedings of the 31st European Conference on Cognitive Ergonomics, New York, NY, USA, 10–13 September 2019; pp. 207–214. 21. Lewis, J.R.; Sauro, J. Item benchmarks for the system usability scale. J. Usability Stud. 2018, 13, 158–167. 22. Bangor, A.; Kortum, P.; Miller, J. Determining what individual SUS scores mean: Adding an adjective rating scale. J. Usability Stud. 2009, 4, 114–123. 23. Devika, M.D.; Sunitha, C.; Ganesh, A. Sentiment analysis: A comparative study on different approaches. Procedia Comput. Sci. 2016, 87, 44–49. 24. Luo, T.; Chen, S.; Xu, G.; Zhou, J. Sentiment Analysis. In Trust-Based Collective View Prediction; Springer: New York, NY, USA, 2013. https://doi.org/10.1007/978-1-4614-7202-5_4. 25. Al-Kabi, M.N.; Gigieh, A.H.; Alsmadi, I.M.; Wahsheh, H.A.; Haidar, M.M. Opinion mining and analysis for Arabic language. Int. J. Adv. Comput. Sci. Appl. 2014, 5, 181–195. 26. Omotosho, B. Analyzing user experience of mobile banking applications in Nigeria: A text mining approach. CBN J. Appl. Stat. 2021, 12, 77–108. 27. Tabiaa, M.; Madani, A. Analyzing the Voice of Customer through online user reviews using LDA: Case of Moroccan mobile banking applications. Int. J. Adv. Trends Comput. Sci. Eng. 2021, 10, 32–40. https://doi.org/10.30534/ijatcse/2021/051012021. 28. Permana, M.; Ramadhan, H.; Budi, I.; Santoso, A.; Putra, P. Sentiment Analysis and Topic Detection of Mobile Banking Appli- cation Review. In Proceedings of the 2020 Fifth International Conference on Informatics and Computing (ICIC), Gorontalo, Indonesiam, 3–4 November 2020; pp. 1–6. 29. Oh, Y.; Kim, J. What Improves Customer Satisfaction in Mobile Banking Apps? An Application of Text Mining Analysis. Asia Mark. J. 2022, 23, 28–37. 30. Metlo, M.; Hussain, N.; Saqib, G.; Phulpoto, K.; Abro, S. Impact of mobile banking on customers’ satisfaction. Int. J. Manag. 2021, 12, 1263–1271. https://doi.org/10.34218/IJM.12.1.2021.111. 31. Gomachab, R.; Maseke, B. The impact of mobile banking on customer satisfaction: Commercial banks of Namibia (Keetmans- hoop). J. Internet Bank. Commer. 2018, 23, 1–18. 32. Kaya, A.; Ozturk, R.; Altin Gumussoy, C. Usability measurement of mobile applications with system usability scale (SUS). In Industrial Engineering in the Big Data Era; Springer: Berlin/Heidelberg, Germany, 2019; pp. 389–400. 33. Kristanto, S.P.; Hakim, L.; Hariyati, F. Usability Evaluation in Ruang Guru Applications Using User Experience Questionnaire (UEQ). J. Mantik 2020, 4, 181–186. 34. Al-Kabi, M.; Wahsheh, H.; Alsmadi, I. Polarity classification of Arabic sentiments. Int. J. Inf. Technol. Web Eng. 2016, 11, 32–49. 35. Alqahtani, F.; Orji, R. Usability issues in mental health applications. In Proceedings of the Adjunct Publication of the 27th Con- ference on User Modeling, Adaptation and Personalization, Larnaca, Cyprus, 9–12 June 2019; pp. 343–348. Doi.org/10.1145/3314183.3323676. 36. Baabdullah, A.; Alalwan, A.; Rana, N.; Kizgin, H.; Patil, P. Consumer use of mobile banking (M-Banking) in Saudi Arabia: Towards an integrated model. Int. J. Inf. Manag. 2019, 44, 38–52. 37. Sharma, S.; Sharma, M. Examining the role of trust and quality dimensions in the actual usage of mobile banking services: An empirical investigation. Int. J. Inf. Manag. 2019, 44, 65–75. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Information Multidisciplinary Digital Publishing Institute

Evaluating and Comparing the Usability of Mobile Banking Applications in Saudi Arabia

Download PDF
14 pages

Loading next page...
 
/lp/multidisciplinary-digital-publishing-institute/evaluating-and-comparing-the-usability-of-mobile-banking-applications-2PBhJ6xc1Q

References (39)

Publisher
Multidisciplinary Digital Publishing Institute
Copyright
© 1996-2022 MDPI (Basel, Switzerland) unless otherwise stated Disclaimer Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. Terms and Conditions Privacy Policy
ISSN
2078-2489
DOI
10.3390/info13120559
Publisher site
See Article on Publisher Site

Abstract

Article Evaluating and Comparing the Usability of Mobile Banking Applications in Saudi Arabia 1,2 1, 1 1 Sarah Alhejji , Abdulmohsen Albesher *, Heider Wahsheh and Abdulaziz Albarrak The Department of Information System, College of Computer Sciences and Information Technology, King Faisal University, Hofuf 31982, Saudi Arabia The Department of Management Information Systems and Production Management, College of Business and Economics, Qassim University, Buraydah 52571, Saudi Arabia * Correspondence: [email protected] Abstract: In many countries, the rapid growth of the Internet and mobile technologies has led to the expansion of Internet banking, especially mobile banking. Many banks seek to provide integrated banking services through mobile applications (apps) to increase customer satisfaction and loyalty. A quick look at the reviews of the mobile banking apps in Saudi Arabia reveals different usability issues among these apps. This research analyzed, evaluated, and compared the usability of all Saudi mobile banking apps available for the iOS and Android systems. Usability (as defined by ISO 9241) was measured using three criteria—effectiveness, efficiency, and satisfaction. This research also identified and discussed the most critical weaknesses of the Saudi banks’ apps in regard to provid- ing satisfactory solutions to developers. The results showed that the most critical issues existed in the user interfaces and functionality of the apps, especially those that frequently received updates. Furthermore, the lack of customer support made the interaction between banks and customers weak, leading to customer dissatisfaction. Keywords: usability; user experience; mobile banking; customer review Citation: Alhejji, S.; Albesher, A.; Wahsheh, H.; Albarrak, A. Evaluating and Comparing the Usability of Mobile Banking 1. Introduction Applications in Saudi Arabia. Recently, the mobile app industry has boomed dramatically worldwide. The number Information 2022, 13, 559. of mobile app downloads increased from 140.68 billion in 2016 to 230 billion in 2022 [1]. https://doi.org/10.3390/info13120559 Rizk stated that mobile apps allow users to better share their feelings and opinions about Academic Editor: Norbert Fuhr the delivered services through writing reviews [2]. Reviews often contain valuable infor- mation for developers, as they can help them enhance and develop their apps to meet Received: 9 October 2022 users’ needs and desires. Specifically, reviews normally include requests for improve- Accepted: 25 November 2022 ment, users’ evaluations, bug reports, queries, or general descriptions of user experiences Published: 29 November 2022 that can be positive or negative. Publisher’s Note: MDPI stays neu- Srisopha et al., showed that app ratings and reviews are essential factors that users tral with regard to jurisdictional consider when choosing apps to download [3]. Moreover, the more positive comments claims in published maps and institu- and reviews an app has, the higher it ranks in the search results, which increases the tional affiliations. chances of appearing to potential users. Reviews can indicate customer satisfaction with the apps and the services provided. Therefore, studying an app’s reviews is an important part of the app life cycle and one of the most critical activities required of app developers in order to maintain and improve their app. Copyright: © 2022 by the authors. Li- A quick review of banking apps in Saudi Arabia showed that customers’ experiences censee MDPI, Basel, Switzerland. This article is an open access article were generally not satisfactory. Customer feedback and comments indicated many prob- distributed under the terms and con- lems with the banking services provided. Because of this, this research aimed to analyze, ditions of the Creative Commons At- evaluate, and compare the usability of mobile banking apps belonging to several Saudi tribution (CC BY) license (https://cre- banks. Usability was measured by the criteria set by ISO 9241, which are effectiveness, ativecommons.org/licenses/by/4.0/). efficiency, and satisfaction. Additionally, this research aimed to identify the most critical Information 2022, 13, 559. https://doi.org/10.3390/info13120559 www.mdpi.com/journal/information Information 2022, 13, 559 2 of 14 problems and weaknesses faced by Saudi banking apps in regard to providing satisfactory solutions to developers. The results obtained were used to determine which apps had the best usability and the most critical problems and weaknesses, as well as the types of im- provements that service providers should make to enhance mobile banking. The rest of the paper is organized as follows. The second section reviews the litera- ture related to this research. The third section includes detailed information about the re- search methodology used. The fourth section provides an analysis of the data and a re- view of the research findings. Finally, the fifth section reviews the main issues encoun- tered with mobile banking in Saudi Arabia, followed by a set of recommendations for developers. 1.1. Mobile Banking Applications Mobile apps are essential components of modern information and communication technology. They allow users to have quick and easy access to the products, services, in- formation, and processes they need in real time. The global use of modern technologies has changed interactions between business owners and customers, contributing to in- creasing customer loyalty and satisfaction. The development of apps requires an under- standing of the target users in terms of their requirements, goals, and ideas. Ali et al. stated that the app industry requires more developer attention to develop apps for ease of use and reliability [4]. Ghandi et al., noted that mobile app development requires continuous improvements to meet new technological needs, such as the design of user interfaces in different sizes to fit the screens of mobile devices [5]. Many businesses have taken ad- vantage of mobile apps to meet the needs of users and thus increase customer satisfaction and loyalty, including mobile healthcare apps (m-health), mobile learning apps (m-learn- ing), and mobile banking apps (m-banking). Customers can use m-banking to pay bills, transfer money, manage accounts, inquire about bank information, and search for ATM locations [6]. M-banking improves customer time management through instant communication and access to information, as well as its ability to be used anywhere. Because of this, it helps improve the customers’ quality of life and increase the efficiency of banks [7]. Good-quality m-banking services can also help to retain and attract customers [8]. Furthermore, improvements in the provision of m- banking services contribute to increasing the bank’s market share, reducing the cost of failure, and lowering the costs of business and attracting new customers to the bank. With the rapid advancement of mobile technology, many banking customers in Saudi Arabia find it easy to use m-banking to make many financial transactions. Al-Khalidi noted that m-banking is the fastest-growing channel for financial growth in Saudi Arabia [9]. Al-Khalidi also explained that the spread of m-banking services is expected to con- tinue in Saudi Arabia as the Internet infrastructure is modernized, government projects are implemented, and banking transactions and payment networks are strengthened and developed [9]. 1.2. Customer Reviews A customer review is a comment regarding a product or service written by a con- sumer who has used or experienced it. These reviews and ratings are public, which means developers and other users can read and benefit from them. As customers search online for product information and compare product options, they often have access to dozens or hundreds of product reviews written by other customers. The reviews and ratings can play an essential role in making purchase decisions. Askalidis and Malthouse indicated that 30% of customers under 45 years of age write reviews for every purchase they make, and 86% of customers say that those reviews are essential to making purchase decisions [10]. According to Avant, a company needs to handle user suggestions and complaints to survive, stay in the market, and gain customer loyalty [11]. Customer loyalty is critical, as retaining existing customers is more expensive than acquiring new customers. According to Zhang and Mao, the hotel sector has experienced a decline in customer loyalty because Information 2022, 13, 559 3 of 14 reading online reviews has increased customer understanding and purchasing power [12]. Avant found that when hotels responded personally to customer complaints and re- quests, the guest retention rate was 85% or higher, while hotels that did not respond to customer demands retained around 30% of their customer base [11]. Therefore, imple- menting a customer–company interaction strategy is an effective way to maintain cus- tomer loyalty. 1.3. Usability According to the ISO 9241-11 standard (ISO, 2018), usability is defined as “the extent to which a system, product, or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” As is stated in this definition, three factors (effectiveness, efficiency, and satisfaction) can affect usability [13]. ISO 9241-11 defines these factors as follows: Effectiveness: the accuracy and completeness with which users achieve specified goals. Efficiency: the resources used in relation to the results achieved. User satisfaction: the extent to which the user’s physical, cognitive, and emotional responses that result from the use of a system, product, or service meet the user’s needs and expectations. Usability and its associated factors have a decisive and powerful impact on the suc- cess of any system, website, or mobile app. Alber et al. defined efficiency as a measure of effectiveness that results in the minimum waste of time and effort [14]. Groth and Has- lwanter linked efficiency with time, wherein they clarified the importance of time in meas- uring users’ efficiency during the performance of tasks [15]. Similarly, mobile effectiveness is a significant factor in m-banking apps. Alber et al. defined this factor as the extent to which a goal is reached without regard to the method and resources for optimal use [14]. Groth and Haslwanter also linked effectiveness with accomplishing a task; when users fail to complete a simple task, this may be strong evi- dence that a bug in the application needs to be fixed [15]. Satisfaction is also an important factor that influences mobile app use. Lee et al., de- fined satisfaction as a brief emotional response from a mobile phone user [16]. Ravendran stated that satisfied m-banking users are more likely to buy from their banks than dissat- isfied users [17]. Customers who are satisfied with services come back and buy again, telling others about their experiences. In contrast, customers who are deeply dissatisfied with services leave, while customers who are poorly satisfied with services may not leave but may complain [18]. In general, usability is essential and is considered one of the most important features of apps and software. One of the main reasons for the failure of apps and software is the need for a system to achieve the set goals of the users and measure their satisfaction. For this reason, usability assessment has become an essential part of any app or software de- velopment [19]. There are different methods to assess usability, the most famous of which is the System Usability Scale (SUS) and Sentiment Analysis (SA). On the other hand, the User Experience Questionnaire (UEQ) is one of the most popular measures for compre- hensive user experience measurement. The SUS is one of the most widely used usability testing tools today. The SUS scale was designed as a quick and easy way to assess usability [20]. The SUS scale consists of a set of data, ten verified data, covering five negative and five positive aspects of the system. Participants are asked to record each of the five questions. SUS scores can be grouped into percentage ranges [21], or a grading system can be used. As for the UEQ measurement, it serves as a means to evaluate and measure the entire user experience. The UEQ scale is based on six scales, such as attractiveness, perspicuity, dependability, efficiency, stimulation, dependability, and novelty [20]. Scales are evalu- ated using pairs of opposite adjectives to describe the system, with participants choosing Information 2022, 13, 559 4 of 14 their level of agreement with each. UEQ scores assess how well a system meets users’ expectations. 1.4. Sentiment Analysis (SA) SA is a process that relies on identifying, classifying, and mathematically processing textual data to obtain the opinions and perspectives of users regarding the topics, services, and products offered to them [22]. Devika et al., mentioned another definition of SA, a technique used to extract positive and negative user opinions about products or services offered [23]. These data are represented as customer feedback stored within forums, blogs, and social media. Several methods can be used to gain user feedback, ranging from human analysis to machine learning. SA determines the polarity of data, which is news or product reviews. There are multiple ways of expressing emotions, from the three most common levels of polarity, positive, neutral, and negative, to scales of polarity, which are set, for example, from −10 to +10. Luo et al., reported that opinion words dominate emotion indicators, particularly in adjectives, adverbs, and verbs, for example, “I love this app; it is amazing!” Opinion words are also known as emotion words, polar words, opinion lexicon, or opinion-carry- ing words, which can be categorized into two types: positive words such as cool, elegant, excellent, and negative words such as terrible, disgusting, and poor [24]. The feature extraction step is depended on SA frequencies to learn about the polarity of different reviews and comments. Weight scores for sentiment features are used to de- termine the strength and polarity of each review and comment. All sentiment fea- tures/words used in this study are manually extracted from comments and reviews col- lected from Saudi bank applications. TF (term frequency) refers to the number of times a given term (comment/review) occurs and is repeated. Then, each sentiment feature/word is weighed manually. The tool relies on the frequency of positive and negative terms/fea- tures to determine the polarity of the reviews. A review is considered positive when the frequency of positive terms/features exceeds the frequency of negative terms/features in the same review. A review is considered negative when the frequency of negative terms/features exceeds that of positive terms/traits in the same review. Finally, the review tool is considered neutral if the frequency of positive features/terms in the review equals the frequency of negative features/terms. Scores in polarity dictionaries are used by the tool to determine the strength of each entry [25]. 2. Related Work Omotosho analyzed reviews written by users of m-banking apps in Nigeria to extract valuable insights regarding the sentiments and emotions expressed by the users. The study found that around 66% of the emotions expressed by users were associated with anticipation, joy, and trust, whereby the remaining 34% were related to fear, disgust, sur- prise, and anger [26]. Tabiaa and Madani used online user evaluations to analyze the voice of the customer and to construct a topic modeling approach based on that data. Security, services, quality, and the user interface were the most common topics observed [27]. Permana et al., conducted a sentiment analysis and m-banking app review topic de- tection in Indonesia to determine customer sentiment toward m-banking apps and to learn which aspects of the examined apps needed to be maintained and improved. The most frequent topics observed among the negative reviews were app login problems, OTP code delivery constraints, and network connections. On the other hand, simplicity, help- fulness, and ease-of-use were the most frequent topics among the positive reviews [28]. Oh and Kim proposed a text-mining approach to identify factors that improved customer satisfaction when using m-banking applications. Their study showed that positive re- sponses regarding the security and convenience of m-banking apps improved the rating of apps in stores. In contrast, increasing comments about insecurity, negative customer support experiences, and sophistication correlated with lower user ratings. These results Information 2022, 13, 559 5 of 14 support the idea that security is the most influential factor in customer satisfaction with m-banking services [29]. Metlo et al., conducted an empirical analysis to study the effect of m-banking on cus- tomer satisfaction in the Pakistani banking sector. The results showed that ease of use, credibility, and customer attitude significantly influenced customer satisfaction with the banking services provided [30]. Mkpojiogu et al., studied demographic differences in user satisfaction with the usability of m-banking apps. The results showed significant differ- ences in the satisfaction of m-banking users based on gender, age, educational qualifica- tions, and experience. These results are helpful for banks, as they can help improve the interfaces of m-banking apps to better meet users’ needs [8]. Gomachab and Maseke stud- ied the effects of m-banking on customer satisfaction in commercial banks in Namibia. The results showed that the most-used service provided by the apps was the ability to make airtime purchases, and the least-used service was the allocation of funds [31]. Kaya et al. measured the usability of mobile apps using the SUS. This study aims to reveal the usability difference between four commonly used mobile apps: WhatsApp, Fa- cebook, YouTube, and Mail. The study also looks for the difference in usability between iOS and Android operating systems. In this study, the SUS with an Adjective Rating Scale was applied to 222 young participants using these apps on their mobile phones. The re- sults showed that the usability of all apps is somewhat satisfactory and above the stand- ards. In more detail, the results showed that the usability of WhatsApp is higher compared to other apps, while the Facebook app has the lowest score. In addition, according to the results, there is no difference in the usability of mobile apps between operating systems [32]. Kristanto et al., assessed the usability of Ruang Guru apps using a UEQ. This survey focuses on user satisfaction, measured using a questionnaire with the Don Foundation Standard. In this study, measurements made on 100 active users of the Ruang Guru apps using the questionnaire were used randomly. The results of the measurements showed a degree of effectiveness of 59.38, efficiency of 65.36, and a degree of satisfaction of 62.52 [33]. 3. Materials and Methods The text data examined in this study consisted of reviews submitted by users of m- banking apps provided by 11 Saudi banks (data are available in a publicly accessible re- pository). These banks were Alinma Bank, Riyad Bank, Al-Rajhi Bank, SABB Bank, Al- Ahly Bank, Al-Fransi Bank, Al-Jazira Bank, Al-Bilad Bank, Arab Bank, Samba Bank, and Investment Bank. This research applied sentiment analysis to analyze reviews written in English and Arabic. The main goal of using SA was to identify the polarity of users’ views about different aspects of app usage [34]. Moreover, the SA method is distinguished from others in that it classifies, identifies, and analyzes a large set of data without the need to identify participants or create tools for analysis, such as questionnaires and personal in- terviews. It is also an open method that allows any user to express his/her opinion directly at any time. In contrast, both SUS and UEQ need to create specific evaluation tools, such as questionnaires directed to a specific number of participants and containing specific questions asked by the analysts. The schematic overview of our approach is exhibited in Figure 1. Information 2022, 13, 559 6 of 14 Figure 1. Review analysis schema. The methods used for data collection and processing are explained below. A. The collection of data was performed using Heedzy, an online tool that allows for the downloading of mobile app reviews and ratings for Android and iOS. This study collected 5958 reviews from Android users and 2438 reviews from iOS users. The data were collected between January and March 2022. The next examples are shown from the collected reviews from the m-banking apps in Saudi Arabia. Consider the following samples of the collected English reviews (original English re- view): Example 1: The new update is bad, and the transaction process is slow and sometimes give errors. Example 2: It is very good and quick B. Three polarity lexicons were constructed manually based on our selected usability factors—satisfaction, effectiveness, and efficiency. El-Halees built lexicons that used opinion phrases and words to determine the sentiment orientation of the whole re- view [19]. Our lexicon included 491 words describing satisfaction (278 positive, 213 negative), 265 describing effectiveness (11 positive, 254 negative), and 65 words de- scribing efficiency (12 positive, 53 negative). Table 1 shows that the satisfaction factor contained more positive words than the effectiveness and efficiency factors. At the same time, satisfaction and effectiveness factors had negative words in close propor- tions and were much higher than the efficiency factor. Table 1. Summary of polarity distribution among the usability factors. Usability Factors No. of Positive Polarities No. of Negative Polarities Satisfaction 278 (65.5%) 213 (43.3%) Effectiveness 11 (2.2%) 254 (51.7%) Efficiency 12 (2.4%) 53 (10.8%) Moreover, Tables 2–4 present a sample of positive and negative polarity words of the usability factors extracted from Saudi bank reviews. These features are stored in the lexi- cons for later use to determine the polarity of reviews. Information 2022, 13, 559 7 of 14 Table 2. Satisfaction lexicon (English and Arabic words). Satisfaction Lexicon Positive Negative Arabic English Arabic English زﺎﺘﻤﻣ Excellent ءﻲﺳ Bad ﺪﯿﺟ Good ﻢﯾﺪﻗ Old Table 3. Effectiveness lexicon (English and Arabic words). Effectiveness Lexicon Positive Negative Arabic English Arabic English لﺎﻌﻓ Effective ﻞﻤﻌﯾ ﻻ Not working ﻞﻤﻌﯾ Working perfectly. ﺄﻄﺧ Error Table 4. Efficiency lexicon (English and Arabic words). Efficiency Lexicon Positive Negative Arabic English Arabic English ﺖﻗو ﺮﻓﻮﯾ quick ءﻲﻄﺑ Slow ﻊﯾﺮﺳ Fast ﻖﻠﻌﯾ Hanging C. The reviews of each bank were labeled manually based on usability factors. Groth and Haslwanter found three traits that measured usability in any review of mobile usability models: effectiveness, efficiency, and satisfaction [15]. A polarity score was given by the authors’ judgment for each factor, a quantitative measure of the positive or negative interactions expressed in the reviews. The average polarity score ranged from −1 to +1, with negative values indicating negative opinions, values of zero indi- cating neutral opinions, and positive values indicating positive opinions. D. The total usability score was calculated for each review based on the sum of each review’s satisfaction, effectiveness, and efficiency values [19]. The overall usability score ranged from −3 to +3. The usability of all banks was evaluated and compared based on these scores in order to determine which of the banks had the highest and lowest usability. Based on example 1 (step A), Table 5 exhibits how to give a polarity score for each word in these reviews that match the lexicons built. Then, the total usability score was calculated. Table 5. Manually extracted features with its polarity weight and calculated usability score (exam- ple 1). Usability Review Satisfaction Effectiveness Efficiency Score The new update is bad, and the transaction pro- −1 −1 −1 −3 cess is slow and some- times gives errors. Information 2022, 13, 559 8 of 14 4. Results and Discussion After arranging the reviews into separate excel sheets for each operating system, we categorized and determined the degree of each usability factor (satisfaction, effectiveness, efficiency). The usability score for each Saudi bank was calculated by summing the per- centage of positive and negative reviews for each usability factor. Tables 6 and 7 show the following data for Android and iOS, respectively: the name of the bank, the number of reviews, the percentage of positive and negative reviews for each usability factor, and the overall usability scores. Table 6 shows the usability scores of all Saudi bank apps on the Android system based on the values of our selected usability factors. These are satisfaction, effectiveness, and efficiency (both positive and negative). The Alinma Bank app demonstrated the high- est positive usability ratio, calculated at 0.81, followed by the Riyad Bank app with a score of 0.72, the Al-Rajhi Bank app with a score of 0.71, the SABB Bank app with a score of 0.69, and Al-Ahli Bank app with a score of 0.34. Meanwhile, the Investment Bank app possessed the highest negative usability ratio, calculated at −0.93, followed by the Samba Bank app with a score of −0.49, the ANB Bank app with a score of −0.34, the Al-Bilad Bank app with a score of −0.30, the Al-Jazira Bank app with a score of −0.28, and the Al-Fransi Bank app with a score of −0.25. Table 7 shows the usability scores for all Saudi banks in the iOS system. The Al-Rajhi Bank app demonstrated the highest positive usability ratio, calculated at 0.77, followed by the Alinma Bank app with a score of 0.03. Meanwhile, the Al-Fransi Bank app had the highest negative usability ratio, calculated at −1.38, followed by the Al-Ahli Bank app with a score of −1.32, the Samba Bank app with a score of −1.13, the Al-Jazira Bank app with a score of −1.10, the Investment Bank app with a score of −1.00, the Al-Bilad Bank app with a score of −0.94, the ANB Bank app with a score of −0.78, the SABB Bank app with a score of −0.35, and the Riyad Bank app with a score of −0.23. Tables 6 and 7 show that the satisfaction factor plays an essential role in determining the degree of usability. This was followed by the factors of effectiveness and finally effi- ciency. Among Android users, the Alinma Bank app had the highest satisfaction rate among banking applications, calculated at 0.877, followed by the Al Rajhi Bank app with a score of 0.825, and then the Riyad Bank app with a score of 0.823. Meanwhile, the In- vestment Bank app had the lowest satisfaction rate among all bank applications, with a calculated rate of −0.465, followed by the Samba Bank app with a score of −0.448, and the ANB Bank app with a score of −0.275. Among iOS users, the Al-Rajhi Bank app had the highest satisfaction rate, calculated at 0.808, followed by the Alinma Bank app with a score of 0.503, and then the Riyad Bank app with a score of 0.422. Meanwhile, the Samba Bank app had the lowest satisfaction rate, calculated at −0.771, followed by the Al-Fransi Bank app with a score of −0.676, and then the Al-Ahli Bank app with a score of −0.543. Information 2022, 13, 559 9 of 14 Table 6. Usability score (Android system). Satisfaction Effectiveness Efficiency Bank Total Re- No. of No. of No. of Usability 4 3 2 1 Name views Negative Score % Positive Negative Negative Score % Positive Negative Negative Score % Positive Negative Score Reviews Reviews Reviews Alinma 405 12 343 0.847 0.877 −0.030 21 −20 −0.049 0.002 −0.052 8 6 0.015 0.035 −0.020 0.81 Bank Riyad 962 48 744 0.773 0.823 −0.050 77 −69 −0.072 0.008 −0.080 19 13 0.014 0.033 −0.020 0.72 Bank Al-Rajhi 1240 43 980 0.790 0.825 −0.035 63 −63 −0.051 0 −0.051 50 −40 −0.032 0.008 −0.040 0.71 Bank SABB 964 60 708 0.734 0.797 −0.062 79 −77 −0.080 0.002 −0.082 16 31 0.032 0.049 −0.017 0.69 Bank Al-Ahli 1085 132 556 0.512 0.634 −0.122 177 −176 −0.162 0.001 −0.163 22 −12 −0.011 0.009 −0.020 0.34 Bank Al-Fransi Android 142 29 31 0.218 0.423 −0.204 52 −52 −0.366 0 −0.366 15 −14 −0.099 0.007 −0.106 −0.25 Bank Al-Jazira 326 88 56 0.172 0.442 −0.270 102 −102 −0.313 0 −0.313 49 −45 −0.138 0.012 −0.150 −0.28 Bank Al-Bilad 305 58 63 0.207 0.397 −0.190 123 −122 −0.400 0.003 −0.403 34 −33 −0.108 0.003 −0.111 −0.30 Bank ANB Bank 353 97 47 0.133 0.408 −0.275 155 −155 −0.439 0 −0.439 19 −12 −0.034 0.020 −0.054 −0.34 Samba 105 47 −16 −0.152 0.295 −0.448 26 −26 −0.248 0 −0.248 9 −9 −0.086 0 −0.086 −0.49 Bank Invest- ment 71 33 −21 −0.296 0.169 −0.465 26 −25 −0.352 0.014 −0.366 20 −20 −0.282 0 −0.282 −0.93 Bank 1 2 Negative: The number of negative reviews divided by the total number of reviews. Positive: The number of positive reviews divided by the total number of 3 4 reviews. %: The score is divided by the total number of reviews. Score: The number of positive reviews minus the number of negative reviews. Information 2022, 13, 559 10 of 14 Table 7. Usability score (iOS system). Satisfaction Effectiveness Efficiency Total Re- No. of Neg- No. of Usability Bank Name No. of Nega- views Score % Positive Negative ative Re- Score % Positive Negative Negative Score % Positive Negative Score tive Reviews views Reviews Al-Rajhi 668 26 514 0.769 0.808 −0.039 35 −33 −0.049 0.003 −0.052 9 33 0.049 0.063 −0.013 0.77 Bank Alinma 183 40 52 0.284 0.503 −0.219 38 −37 −0.202 0.005 −0.208 18 −9 −0.049 0.049 −0.098 0.03 Bank Riyad Bank 429 106 75 0.175 0.422 −0.247 122 −122 −0.284 0 −0.284 64 −52 −0.121 0.028 −0.149 −0.23 SABB Bank 129 30 16 0.124 0.357 −0.233 50 −50 −0.388 0 −0.388 15 −11 −0.085 0.031 −0.116 −0.35 ANB Bank 156 53 −23 −0.147 0.192 −0.340 87 −87 −0.558 0 −0.558 15 −12 −0.077 0.019 −0.096 −0.78 Al-Bilad 81 30 −21 −0.259 0.111 −0.370 45 −45 −0.556 0 −0.556 12 −10 −0.123 0.025 −0.148 −0.94 Bank iOS Investment 86 35 −29 −0.337 0.070 −0.407 27 −26 −0.302 0.012 −0.314 31 −31 −0.360 0 −0.360 −1.00 Bank Al-Jazira 155 41 −30 −0.194 0.071 −0.265 101 −101 −0.652 0 −0.652 40 −39 −0.252 0.006 −0.258 −1.10 Bank Samba Bank 48 37 −36 −0.750 0.021 −0.771 16 −16 −0.333 0 −0.333 2 −2 −0.042 0 −0.042 −1.13 Al-Ahli 466 253 −237 −0.509 0.034 −0.543 229 −229 −0.491 0 −0.491 148 −147 −0.315 0.002 −0.318 −1.32 Bank Al-Fransi 37 25 −24 −0.649 0.027 −0.676 21 −21 −0.568 0 −0.568 6 −6 −0.162 0 −0.162 −1.38 Bank Information 2022, 13, 559 11 of 14 From Tables 6 and 7, we can see that opinions regarding effectiveness were weighted toward negative in reviews of all Saudi bank applications. Among Android users, the ANB Bank app had the lowest effectiveness rate among all Saudi bank applications, with an estimated rate of −0.439, followed by the Al-Bilad Bank app with a score of −0.403, and then the Al-Fransi and Investment Bank apps, both with scores of −0.366. Among iOS us- ers, the Al-Jazira Bank app possessed the lowest effectiveness rate, with a calculated value of −0.652, followed by the Al-Fransi Bank app with a score of −0.568 and the ANB Bank app with a score of −0.558. In addition, Tables 6 and 7 revealed that efficiency scores were often close to the neu- tral zero. Among Android users, the SABB Bank app possessed the highest efficiency ratio of the banking applications, with a calculated percentage of 0.049, followed by the Alinma Bank app with a score of 0.035, the Riyad Bank app with a score of 0.033, and the ANB Bank app with a score of 0.020. Meanwhile, the Investment Bank app possessed the lowest efficiency rate of the banking applications, with a calculated percentage of −0.282, fol- lowed by the Al-Jazira Bank app with a score of −0.150, the Al-Bilad Bank app with a score of −0.111, and the Fransi Bank app with a score of −0.106. Among iOS users, only one bank, the Al-Rajhi Bank, possessed a high efficiency rate, with a score of 0.063. The Investment Bank app once again had the lowest efficiency rate, this time calculated at −0.360, followed by the Al-Ahli Bank app with a score of −0.318 and the Al-Jazira Bank app with a score of −0.258. Our research indicated various issues that affected usability from the perspectives of customer satisfaction, effectiveness, and efficiency. Our quantitative analysis led us to find common patterns that the banks have regarding these aspects and thus complement the results obtained from the usability evaluation. We found that these issues mainly stemmed from new updates, disabled functions, and lack of customer support. New Update Problem One of the commonly repeated problems that affected customers’ satisfaction with the Saudi m-banking apps on both the Android and iOS was the issue of new app updates. For example, the comment “problem with the new update” appeared frequently in re- views of the Al-Jazira Bank app, comprising an estimated 28.5% of reviews on Android and 24.5% on iOS. In regard to the Investment Bank app, the problem appeared with a frequency of 11% on Android and 11.7% on iOS. Meanwhile, 9.6% of reviews regarding the ANB Bank app on iOS referred to the new update issue using the repeated comment “this app does not work on a jailbreak device,” despite the evidence that their device was jailbreak-free. Update issues have also been found in apps in other domains, such as health care. For example, Alqahtani and Orji mentioned that users of some mental health apps explained that a new update caused some problems with the app’s functions that led to data loss [35]. In the m-banking domain, Tabiaa and Madani also reported frequent issues with new updates [27]. Functional Problems The main objective of using m-banking apps is to complete banking transactions in a quick and easy manner. Baabdullah et al., mentioned that Saudi customers view m-bank- ing as a method that saves customers time, money, and effort [36]. A strong relationship exists between the actual use of banking services and customer loyalty [36]. They indi- cated that the implementation of m-banking functions is essential not only for customer loyalty but also for increasing customer satisfaction [36]. In regard to the effectiveness factor, the apps reviewed in this research showed that customers complained about the disabling of some app functions by some Saudi banks. Commonly repeated comments regarding this issue included “the application does not work” representing 15% of comments, “cannot log in” 13% of comments, “the application cannot be opened” 11% of comments, and “there is a general error” 8% of comments. The ANB Bank apps for both Android and iOS systems, the Al-Bilad Bank and the Investment Bank apps for the Android system, and the Al-Jazira Bank and the Al-Fransi Bank apps for the iOS system were ranked as the worst apps regarding this issue. Similar issues were Information 2022, 13, 559 12 of 14 found in other research papers. For example, Permana et al., conducted a study on m- banking apps in Indonesia; their study showed that the most common issues involved signing in to an app [28]. Lack of Customer Service Support Saudi banking apps should provide different methods to communicate with custom- ers. One possible method is via an instant chat, which only the SABB Bank app provided. Meanwhile, the apps belonging to the Al-Rajhi, Al-Fransi, and Alahli banks did not pro- vide an instant chat, but they did allow customers to send their queries and complaints through the apps. Many reviews of the Saudi banking apps contained questions and in- quiries from customers directed to the app developers. These reviews, which represent 7% of all comments, include a range of in-app complaints and questions about how to use the app, the contents of new updates, and how to activate certain services. Some reviews that included a specific complaint about the app were answered by app developers with short responses—for example, “Thank you for your note; the problem will be resolved as soon as possible.” Neglecting to respond to customers can negatively affect customer sat- isfaction. Banks should respond to customers swiftly and direct them to proper solutions. Sharma and Sharma emphasized the need for a well-trained staff who can listen to, un- derstand, and handle customers’ problems [37]. Based on our analysis of users’ positive and negative reviews in Saudi banks apps, problems and themes were extracted from our results. We recommend the following to developers in order to improve the usability of Saudi banking apps: • Examine new updates before they are officially released and ensure that they are free from errors and problems. Many reviews indicated that customers were not satisfied with banking apps due to issues with new updates, which caused issues such as ap- plications that stopped working and updates that were not compatible with the user’s mobile device. • Analyze app reviews to improve apps and keep customers satisfied. Reviews are shared spaces that allow customers to express their opinions, requests, and com- plaints regarding downloaded apps. These reviews allow developers to receive cus- tomer feedback about app usability issues. If these reviews are taken into account and the developers fix the problems, it will significantly improve the apps. • Respond quickly to app reviews to increase customer satisfaction. Developers’ re- sponses to customer comments increase customer satisfaction and loyalty. These re- sponses make the customer feel that their feedback is important, and that the devel- oper is keen to solve their problems. Furthermore, responding to reviews has positive results—ratings of the app often increase after developers respond to customer re- views. • Enable a live chat function that can support interaction between banks and their cus- tomers. This would allow for an effective channel of communication that could re- spond to user requests and fix recurring issues in real time. 5. Conclusions Customer feedback and comments on m-banking apps belonging to some Saudi banks indicate many issues in the banking services provided. This study aimed to evalu- ate and compare the usability of all Saudi m-banking apps for Android and iOS based on three usability factors: satisfaction, effectiveness, and efficiency. This research identified several usability issues and recommended some useful solutions. Saudi banks should ex- amine and review new updates before they are issued, ensure that the essential functions of the apps work, and consider adding online chat features for customer service. The dif- ficulties this study encountered are represented in the study and analysis of reviews in the Arabic language, as the Arabic language is considered difficult because it is a highly inflectional and derived language. Moreover, some comments include words in the Saudi dialect, which are somewhat inconsistent with the Arabic language, in addition to the fact Information 2022, 13, 559 13 of 14 that these words do not have a particular lexicon. Additionally, some confusing words were excluded from this study, including positive words in some regions of Saudi Arabia and negative ones in others. Some comments include positive words and negative sym- bols that are difficult to classify; thus, they were excluded. The iOS and Android app stores recently began allowing developers to respond to customer reviews. As part of our future work, we plan to study and analyze the responses of app developers to customer reviews, evaluate the quality of those responses, and de- termine whether the developer’s response affects the customer’s experience. Moreover, a model can be created for this work so that reviews are categorized and analyzed automat- ically instead of by the manual method used. Author Contributions: Data curation, S.A.; Formal analysis, S.A.; Methodology, S.A.; Supervision, A.A. (Abdulmohsen Albesher), H.W. and A.A. (Abdulaziz Albarrak); Writing—original draft, S.A.; Writing–review and editing, A.A. (Abdulmohsen Albesher) and H.W. All authors have read and agreed to the published version of the manuscript. Funding: This work was supported by the Deanship of Scientific Research, Vice Presidency for Graduate Studies and Scientific Research, King Faisal University, Saudi Arabia (grant no. 1074), through its KFU Research Summer initiative. Data Availability Statement: Data available in a publicly accessible repository. These data can be found here: (https://drive.google.com/drive/folders/1ZdG5efrKos0p0-hHT8BsO3uKvC4AOdnu ac- cessed on 9 October 2022). Conflicts of Interest: The authors declare no conflict of interest. References 1. Number of Mobile App Downloads Worldwide from 2016 to 2021 (In Billions). In Statista—The Statistics Portal. Available online: https://www.statista.com/statistics/271644/worldwide-free-and-paid-mobile-app-store-downloads/ (accessed on 5 April 2022). 2. Rizk, M.; Ebada, A.; Nasr, S. Investigating Mobile Applications’ Requirements Evolution through Sentiment Analysis of Users. In Proceedings of the International Computer Engineering Conference, Cairo, Egypt, 29–30 December 2015. 3. Srisopha, K.; Link, D.; Boehm, B. How should developers respond to app reviews? features predicting the success of developer responses. In Proceedings of the Evaluation and Assessment in Software Engineering, Trondheim, Norway, 21–23 June 2021, pp. 119–128. 4. Ali, W.; Riaz, O.; Mumtaz, S.; Khan, R.; Saba, T.; Bahaj, S. Mobile Application Usability Evaluation: A Study Based on Demog- raphy. IEEE Access 2022, 10, 41512–41524. 5. Ghandi, L.; Silva, C.; Martínez, D.; Gualotuña, T. Mobile application development process: A practical experience. In Proceed- ings of the 2017 12th Iberian Conference on Information Systems and Technologies (CISTI), Lisbon, Portugal, 21–24 June 2017; pp. 1–6. 6. Afshan, S.; Sharif, A. Acceptance of mobile banking framework in Pakistan. Telemat. Inform. 2016, 33, 370–387. https://doi.org/10.1016/j.tele.2015.09.005. 7. Malaquias, R.; Hwang, Y. An empirical study on trust in mobile banking: A developing country perspective. Comput. Hum. Behav. 2016, 54, 453–461. 8. Mkpojiogu, E.; Hashim, N.; Adamu, R. Observed demographic differentials in user perceived satisfaction on the usability of mobile banking applications. In Proceedings of the Knowledge Management International Conference, Chiang Mai, Thailand, 29–30 August 2016. 9. Alkhaldi, A. Adoption of mobile banking in Saudi Arabia: An empirical evaluation study. Int. J. Manag. Inf. Technol. 2016, 8, 1– 10. Askalidis, G.; Malthouse, E. The value of online customer reviews. In Proceedings of the 10th ACM Conference on Recom- mender Systems, Boston, MA, USA, 15–19 September 2016; pp. 155–158. 11. Avant, T. Responding to TripAdvisor: How Hotel Responses to Negative Online Reviews Effect Hotel Image, Intent to Stay, and Intent to Return; University of South Carolina: Columbia, SC, USA, 2013. 12. Zhang, J.; Mao, Z. Image of all hotel scales on travel blogs: Its impact on customer loyalty. J. Hosp. Mark. Manag. 2012, 21, 113– 13. ISO. (n.d.). Available online: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed 2:v1:en (accessed on 1 June 2022). 14. Alber, N.; Elmofty, M.; Kishk, I.; Sami, R. Banking efficiency: Concepts, drivers, measures, literature and conceptual model. In Drivers, Measures, Literature and Conceptual Model; SSRN: Rochester, NY, USA, 2019. http://dx.doi.org/10.2139/ssrn.3310982. 15. Groth, A.; Haslwanter, D. Efficiency, effectiveness, and satisfaction of responsive mobile tourism websites: A mobile usability study. Inf. Technol. Tour. 2016, 16, 201–228. Information 2022, 13, 559 14 of 14 16. Lee, D.; Moon, J.; Kim, Y.; Mun, Y. Antecedents and consequences of mobile phone usability: Linking simplicity and interactiv- ity to satisfaction, trust, and brand loyalty. Inf. Manag. 2015, 52, 295–304. 17. Ravendran, R. Improving Usability of e-Commerce Websites via Tag-Based Customization: A Study on Online and Mobile Banking. Doctoral Dissertation, Queensland University of Technology, Brisbane, Queensland, 2013. 18. Bayraktar, E.; Tatoglu, E.; Turkyilmaz, A.; Delen, D.; Zaim, S. Measuring the efficiency of customer satisfaction and loyalty for mobile phone brands with DEA. Expert Syst. Appl. 2012, 39, 99–106. 19. El-Halees, A. Software Usability Evaluation Using Opinion Mining. J. Softw. 2014, 9, 343–349. https://doi.org/10.4304/jsw.9.2.343- 20. Holmes, S.; Moorhead, A.; Bond, R.; Zheng, H.; Coates, V.; McTear, M. Usability testing of a healthcare chatbot: Can we use conventional methods to assess conversational user interfaces? In Proceedings of the 31st European Conference on Cognitive Ergonomics, New York, NY, USA, 10–13 September 2019; pp. 207–214. 21. Lewis, J.R.; Sauro, J. Item benchmarks for the system usability scale. J. Usability Stud. 2018, 13, 158–167. 22. Bangor, A.; Kortum, P.; Miller, J. Determining what individual SUS scores mean: Adding an adjective rating scale. J. Usability Stud. 2009, 4, 114–123. 23. Devika, M.D.; Sunitha, C.; Ganesh, A. Sentiment analysis: A comparative study on different approaches. Procedia Comput. Sci. 2016, 87, 44–49. 24. Luo, T.; Chen, S.; Xu, G.; Zhou, J. Sentiment Analysis. In Trust-Based Collective View Prediction; Springer: New York, NY, USA, 2013. https://doi.org/10.1007/978-1-4614-7202-5_4. 25. Al-Kabi, M.N.; Gigieh, A.H.; Alsmadi, I.M.; Wahsheh, H.A.; Haidar, M.M. Opinion mining and analysis for Arabic language. Int. J. Adv. Comput. Sci. Appl. 2014, 5, 181–195. 26. Omotosho, B. Analyzing user experience of mobile banking applications in Nigeria: A text mining approach. CBN J. Appl. Stat. 2021, 12, 77–108. 27. Tabiaa, M.; Madani, A. Analyzing the Voice of Customer through online user reviews using LDA: Case of Moroccan mobile banking applications. Int. J. Adv. Trends Comput. Sci. Eng. 2021, 10, 32–40. https://doi.org/10.30534/ijatcse/2021/051012021. 28. Permana, M.; Ramadhan, H.; Budi, I.; Santoso, A.; Putra, P. Sentiment Analysis and Topic Detection of Mobile Banking Appli- cation Review. In Proceedings of the 2020 Fifth International Conference on Informatics and Computing (ICIC), Gorontalo, Indonesiam, 3–4 November 2020; pp. 1–6. 29. Oh, Y.; Kim, J. What Improves Customer Satisfaction in Mobile Banking Apps? An Application of Text Mining Analysis. Asia Mark. J. 2022, 23, 28–37. 30. Metlo, M.; Hussain, N.; Saqib, G.; Phulpoto, K.; Abro, S. Impact of mobile banking on customers’ satisfaction. Int. J. Manag. 2021, 12, 1263–1271. https://doi.org/10.34218/IJM.12.1.2021.111. 31. Gomachab, R.; Maseke, B. The impact of mobile banking on customer satisfaction: Commercial banks of Namibia (Keetmans- hoop). J. Internet Bank. Commer. 2018, 23, 1–18. 32. Kaya, A.; Ozturk, R.; Altin Gumussoy, C. Usability measurement of mobile applications with system usability scale (SUS). In Industrial Engineering in the Big Data Era; Springer: Berlin/Heidelberg, Germany, 2019; pp. 389–400. 33. Kristanto, S.P.; Hakim, L.; Hariyati, F. Usability Evaluation in Ruang Guru Applications Using User Experience Questionnaire (UEQ). J. Mantik 2020, 4, 181–186. 34. Al-Kabi, M.; Wahsheh, H.; Alsmadi, I. Polarity classification of Arabic sentiments. Int. J. Inf. Technol. Web Eng. 2016, 11, 32–49. 35. Alqahtani, F.; Orji, R. Usability issues in mental health applications. In Proceedings of the Adjunct Publication of the 27th Con- ference on User Modeling, Adaptation and Personalization, Larnaca, Cyprus, 9–12 June 2019; pp. 343–348. Doi.org/10.1145/3314183.3323676. 36. Baabdullah, A.; Alalwan, A.; Rana, N.; Kizgin, H.; Patil, P. Consumer use of mobile banking (M-Banking) in Saudi Arabia: Towards an integrated model. Int. J. Inf. Manag. 2019, 44, 38–52. 37. Sharma, S.; Sharma, M. Examining the role of trust and quality dimensions in the actual usage of mobile banking services: An empirical investigation. Int. J. Inf. Manag. 2019, 44, 65–75.

Journal

Information – Multidisciplinary Digital Publishing Institute

Published: Nov 29, 2022

Keywords: usability; user experience; mobile banking; customer review

There are no references for this article.