User Experience Analysis of Neobank Application Using Heart Metrics

—Based on the accumulated reviews, it is known that the symptoms of disrupted user experience are supported by several reviews which show dissatisfaction in operating the Neobank application, which may have an effect on decreasing user satisfaction. This problem is something that must be monitored by PT. Neo Commerce Bank so that the Neobank application remains sustainable. This research has a mission to find out what will be emphasized for the development of the Neobank application based on the HEART Metrics measurement tool with the Importance Performance Analysis method implemented. Determining the number of respondents in this study implemented the formula created by Slovin and obtained a population of 99 respondents who are residents of the island of Java who use the Neobank application. Until the results of the importance and performance elaboration, there are several statement items that need to be evaluated, but there are items that need to be evaluated and prioritized immediately based on the results of the calculation of the suitability level and Cartesian diagram, namely the user experience that it is not uncommon to experience errors when operating a Neobank with a level conformity 91.91%. The items in the aforementioned statement have the potential to divert the user experience in using the Neobank application.

The company strives to develop mobile applications that will assist businesses in the delivery of information and services.PT.Bank Yudha Bakti which in 2020 changed its name to PT.Bank Neo Commerce Tbk is a public company engaged in banking.In order to improve banking services to consumers in Indonesia, PT.Bank Neo Commerce transformed into a digital bank by providing a mobile application called Neobank in 2020, with the aim of meeting the needs of neo commerce bank customers, especially in the millennial market segment [1] and generation Z in the future to be more flexible to use.
The user experience of the Neobank application is quite good based on the review scores obtained on the application on the application distribution platform.On the app store (iOS and iPadOS), the Neobank app gets a score of 3.8 out of 5 stars and on the playstore (Android OS), the Neobank app gets a score of 4.1 out of 5 stars.But from the accumulated score, the second most value is a score of 1 star.Based on the accumulation contained in the appstore and playstore that there are symptoms of a user experience that is experiencing a disorder driven by several reviews that show dissatisfaction in using the Neobank application.Carrine Lallemand, et al. (2015) say that user experience (UX) is generally described as a holistic quality of interaction between the user and interactive systems [2].The three classic pillars that influence UX are user, system, and context.User experience (UX) is a crucial view related to the science of interactive systems (Human Computer Interaction).UX plays a very important role in ensuring the effective and efficient delivery of a product or service to the end user [3].PT.Neo Commerce Bank should evaluate the user experience of the Neobank application because if the user experience does not work properly and correctly, the user will not want to use the product (Garret 2010: 17) [4].Regarding the above issues, the user experience assessment was carried out with the HEART Metrics (Happiness, Engagement, Adoption and Retention, and Task Success) measuring tool which was innovated by Rodden et al. as Google reviewers [5].In the search for some aspects that need to be developed, PT.Bank Neo Commerce processes decisions from the HEART Metric measuring instrument while implementing the Importance Performance Analysis method.

II. M ETHOD
A.

Collecting Data
Quantitative research methods are used in research activities and focus on analyzing the user experience of Neobank applications using the HEART metric.The data analysis technique used in this study uses descriptive analysis and how to accumulate data is carried out by taking essential data taken from online reviews in the form of forms provided by Google companies and then shared via social media and secondary data obtained from previous studies, namely study literature, and information obtained via the internet is published in the media, the sample of this study is a user of the Neobank application with.The measurement scale used is a likert scale with the following levels: 1. Strongly Disagree HEART Metrics HEART Metrics is an additional measurement tool designed with the user in mind.HEART Metrics is a framework created by Kenny Rodden based on Google's user-centric metrics [6].HEART measures 5 variables, namely Happiness, Engagement, Adoption, Retention and Task Success.The purpose of using HEART-Metrics is to support decision making based on the user experience of the application [7].Happiness is a metric that is a subjective part of the user experience, including user satisfaction and interest in using the app.Engagement is a measure of how a user's role in a product includes time spent using or interacting with the product.Adoption is a measure that allows you to investigate how many users have just started operating your app over a specific period of time.Retention is a measure used to track how many users are still using a product from one period to the next.Task success is a metric that evaluates the traditional user experience, including efficiency, effectiveness, and user error. C.

Population and Sample
In general the sample is interpreted as part of the population.A population according to Tarsi and Tuff (2012) is defined as a group of individuals of the same species living in an exclusive region.Members of a population often depend on the same resources, are subject to the same environmental constraints, and their survival over time depends on the availability of other members [8].The research sample must be representative/representative to obtain accurate results.The determination of the number of samples according to the Slovin formula is as follows: (1) Explanation: n = number of samples α = fault tolerance (10%) N = overall population According to Wiyadi (2009) based on Roscoe (1972), Gay and Diehl (1992), the minimum number of samples is 30 samples [9].

Importance Performance Analysis
In 1977 Martilla and James stated in their article entitled "Importance Performance Analysis" as for publication in the Journal of Marketing.In this technique, respondents are asked to assess the level of importance and performance of an enterprise, then the average level of importance and performance is analyzed in a resultimportance matrix, where the x-axis represents the observation and the y-axis represents the observation.represents hope.The results are then obtained in the form of four quadrants as shown in Figure 1.The interpretation according to the quadrant of Fig. 1 becomes the following: • Primary (Focus Here) Following quadrant I there are still factors that are believed to be important and or needed by consumers, but the company's performance is considered unsatisfactory, so the company must focus on allocating resources to the activities involved.in this quadrant.

• Continuation of Achievement (Keep Up the Good Work)
Following quadrant II there are still factors that should be decisive and should be factors supporting customer satisfaction, therefore the company must maintain its past performance.

• Poor Accentuation
Following quadrant III there are still factors that have a low perception or actual performance and are not very important and are not really needed by consumers, so companies do not need to prioritize or pay more attention.those factors before.
• Too Much (Maybe Too Much) Following quadrant IV there are factors that are considered too important and not very much needed by the client.As a result, the company is better off allocating resources related to these factors to other, higher priority factors.

Analytical Variables and Instruments
In table 1, the statement items based on this study are user experiences using variables such as Happiness, Engagement, Adoption and Retention, and Task Success.F.

Validity Test
The application of instruments in research is able to measure and reveal information about the variables studied.This can be seen by using a validity test to find out whether an instrument is valid or not.To test the validity of a measuring instrument, a correlation score is first calculated using Pearson's product moment formula using IBM SPSS Statistics version 22.0, and the total instrument is considered valid if the correlation coefficient > 0.3.

Reliability Test
Reliability refers to the problem of accuracy (accuracy) of measuring instruments.This accuracy can be evaluated by using statistical analysis to determine measurement errors or measurement errors.A device is considered reliable if it can be reliable enough as a data measuring device.Research with reliability test implements Cronbach's Alpha formula using IBM SPSS Statistics version 22.0 based on Sugiono (2017) that statements greater than 0.6 statement instrument values are considered reliable [10].

A.
Data and Analysis Of the existing population of 19,000,000, the calculation of the required number of samples using the Slovin formula: 19  10 6 1 + 19  10 6 (0,1) 2  = 19  10 6 1 + 19  10 6 (0,1) 2  = 99  After the analysis instrument is carried out and stated to have passed the validity and reliability test, the next step is that the next data is processed using Importance Performance Analysis (IPA) which first determines the degree of conformity between perception (performance) and expectations (interest).Then calculate the average number of each question item used for the limit when visualizing the cartesian diagram [11].The final step is to calculate the total of the variable importance level with the following calculation: From the results of the level of application of variables in Table 3, it can be seen that the level of application of the Happiness variable between observation (performance) and user expectations (interest) is 97.6%.The configuration engagement between observation (efficiency) and user expectations (interests) was 96.3%.The Adoption match between perception (efficiency) and user expectations (interests) is 96.90%.Retention got a 98.33% match between observations (efficiency) and user expectations (interests).Task Success compatibility between observations (results) and user expectations (interests) on the engagement variable was 95.99%.All variables belong to the group of high applications (80% to 100%).
The retention variable was the variable with the highest compliance rate of 98.33%.If it provides information that the user experience of the Neobank application can be said to be successful in terms of retention (user activity).The variable with the lowest level of compliance is the liability variable with a value of 95.99%.If it provides information that the Engagement variable is the variable that the Neobank team or application development manager needs to improve the Neobank team or manager the most to improve the Neobank user experience.For the compliance rate shown in Table 3, 97.0% of user expectations were met, while 2.96% were not met.In addition, more comprehensive processing is carried out by mapping cartesian diagrams into quadrant sections.Based on the processing provisions seen in Figure 2, it can be concluded that: • Quadrant I From the user experience, items 9 and 20 become several items with quality that must be developed.Improvements regarding user responses to the desire to use the Neobank application in making transactions need to be explored by PT.Neo Commerce Bank so that the Neobank application becomes a flexible and best banking application.The impression of the error problem in the user's Neobank application is not very good.A detailed assessment should be made to look for the problem that caused it.

•
Quadrant II  IV.C ONCLUSION Based on calculations that have been made for the UX satisfaction score, including the HEART variables (happiness, engagement, adoption, retention, and task success).The results of data processing and analysis that have been carried out in the results and discussion section, then several conclusions can be drawn.
It can be known the three best items namely that: 1.The Neobank app is easy to use.
2. Neobank application makes it easy to make transactions, 3. Users will recommend others to use the Neobank app.
For things or items that need to be repaired, cartesian diagrams can be used in important performance analysis calculations to monitor where there are items that need to be repaired as soon as possible.The item is in task success variable number 20 about error problems when using the application so that it provides a less experience for users.So the manager is expected to pay more attention to the statement item in order to improve the user experience on Neobank.

TABLE I .
Q UESTION V ARIABLES AND I NSTRUMENTS Engageme nt 7 I can use the Neobank app at any time.8 I use the Neobank app to find out the amount of my balance.9 I always use the Neobank app to make transactions.10 I have hoped to use the Neobank application for a long time.

TABLE II .
Calculation Results of Perceptions and Expectations

TABLE III .
Conformity Level Calculation Results

TABLE IV .
Quadrant I

TABLE V
Based on quadrant II, PT.Bank Neo Commerce is advised to protect several items, namely items number 1, 2, 3, 4, 5, 6, 8, 12, 19 because based on the experience gained by users, it has been felt well and the services and information that have been provided by PT.Neo Commerce Bank has also been good.Therefore, PT.Bank Neo Commerce must maintain this for the future to be better.

TABLE VI
The next quadrant is III, importance and performance levels on items 7, 10, 11, 13, 14, 15, 16, 17, 18, users and PT.Neo Commerce Bank does not consider it important.The company also does not prioritize it in the neobank application development focus.But PT.Neo Commerce Bank needs to focus a little on this part of the item, as this does not prevent the item from becoming a priority for the user.