Older adults aged 65 and above are expected to comprise 10% of the global population in 2022, with a projected increase to 16% by 2050 (United Nations, 2022). This demographic shift is accompanied by a demand for healthy aging and a rise in healthcare costs. Older adults in low-income brackets often face difficulties in accessing and sustaining medical services (Doetsch et al., 2017; Murata et al., 2010). To alleviate the rise in healthcare costs, alternatives such as health management, prevention, telemedicine, and mobile health (mHealth) are emerging (Ainsworth et al., 2013; Bali, 2018; Bricker et al., 2014; Ignatowicz et al., 2019; Ramsey et al., 2020; Torous and Powell, 2015; WHO, 2021).

mHealth is primarily utilized on smartphones and is defined as "medical and public health practice supported by mobile devices" (WHO, 2011). The global mHealth application market is rapidly expanding, offering services such as fitness, disease management, telemedicine, medication management, and personal health record management (Grand View Research, 2023). Among these, the monitoring services sector holds the highest revenue share in the 2022 mHealth market at 61.5% (Grand View Research, 2023).

The smartphone adoption rate among individuals aged 65 years and above, who constitute a significant portion of patients with chronic diseases, is substantial, with rates of 67% in the United Kingdom and 61% in the United States (Statista, 2021; Statista, 2022). Although the number of digital healthcare applications has increased to 350,000, barriers to entry persist for mHealth (IQVIA, 2021). While younger users find application usage more intuitive, older adults encounter difficulties in intuitive comprehension during initial use. Older people tend to hesitate to use mobile devices due to a fear of lack of knowledge (Cajita et al., 2018). They face challenges in utilizing smartphones due to physical functional decline such as visual, auditory, and memory issues (Davis et al., 2007; Farage et al., 2012). (Scheibe et al., 2015) revealed that diabetes patients aged 50 or older found discomfort in data input, menu navigation, and small font sizes when using the diabetes management applications Glucose Monitor and OnTrack Diabetes.

In a study by Ruzic (Ruzic et al., 2016), guidelines for fonts, layout, and images for older adults were proposed, but their effectiveness remains unverified as no actual evaluations using them were conducted. Lindberg (Lindberg and De Troyer, 2021) created a web application and conducted expert Likert scale evaluations for each item in the guidelines, identifying essential elements to be considered in the application. However, quantitative comparative analyses of a specific figure, such as font size and spacing width, were not conducted. While research is being conducted on application design guidelines for older adults, there is a significant lack of research experimenting with their effectiveness, despite consideration for various problems (Chung and Kang, 2020; Gomez-Hernandez et al., 2023). Furthermore, the evaluation methods for mHealth applications for older adults mainly rely on subjective surveys or interviews, with objective data collection methods being less commonly utilized (Wang et al., 2022).

This study aimed to improve mHealth applications for older adults based on existing guidelines and identify effective design elements by analyzing the effects of improvement using a usability evaluation. We conducted a quantitative comparative evaluation of the existing rehabilitation service application based on gait assessment (RSAGA) and its improved version with the applied guidelines for content, graphics, layout, and interaction using the System Usability Scale (SUS) and performance metrics (Albert and Tullis, 2022). The SUS is easy to use even for older adults, and reliable can be obtained even with small sample sizes. If the guidelines for designing customized applications for older adults are applied to the development of mHealth applications, performance time, the click count, and the error count will decrease and the SUS score will improve compared to that of the early version of the application.

The remainder of this paper is organized as follows. Section 2 identifies problems with existing mHealth applications for older adults and suggests improvements according to the guidelines. Moreover, this section presents an experimental process and data analysis method to quantitatively analyze the improvement effect. Section 3 presents the experimental results. Finally, the discussion and conclusions are presented in Sections 4 and 5, respectively.

2.1 Participants and setting

The experiment took place at the Living Lab of the Senior Industry Innovation Center in Seongnam, South Korea in December 2022. To evaluate the usability of the improved application (IA), a usability assessment was conducted with participants aged 60 and above.

Participants were recruited through the Korea Institute of Industrial Technology and drawn from the pool of participants of the Seongnam Senior Industrial Innovation Center, a research institute specializing in usability evaluation for older adults. The appropriate number of participants was determined following Albert (Albert and Tullis, 2022). Thirty-six participants were recruited in accordance; therefore, data from 10 or more individuals were necessary to ensure statistical significance for tests utilizing performance metrics. Similarly, tests employing the formative usability test revealed that significant performance indicators could be obtained through the use of personal data from 8 to 12 individuals.

To enhance the reliability of the usability evaluation, 20 men and 16 women were recruited from a healthy elderly population without knee impairment. The inclusion criteria were as follows: not lacking in physical ability and knee health, able to walk independently, and no history of mental illness such as dementia. Participants were from 60 to 70 years of age, with a mean age of 64.91 (SD=2.74). Participants were randomly divided into groups of 18 people, and each group used a different application. To ensure balanced participant characteristics, smartphone proficiency was self-assessed and categorized into three levels—high, medium, and low—based on subjective evaluation. Group allocation was adjusted to maintain comparable proficiency levels between the EA and IA groups. Although no significant differences related to smartphone proficiency were found in the results, this factor was considered during randomization. Additionally, each group consisted of 10 males and 8 females to ensure gender balance.

This study obtained approval for the experimental procedures from Korea Institute of Industrial Technology the Institutional Review Board. Participants were provided with a description of the study and completed a consent form prior to participating in the study, and were assured that their information would remain confidential.

2.2 Design improvement

RSAGA mainly functions to provide post-surgery monitoring services. Users can check their recovery status after TKR surgery through knee angle measurement and walking pattern analysis before and after surgery, step counting, pain level assessment, and self- medical examination. The application also provides videos of rehabilitation exercises necessary after discharge.

Usability issues related to content, graphics, layout, and interaction were identified within the service after confirming the guidelines presented in previous studies (Barros et al., 2014; de Almeida et al., 2015; Harte et al., 2017; Jo and Lee, 2018; Kascak et al., 2014; Morey et al., 2019; Silva et al., 2014). Guidelines were formulated by integrating user interface evaluation guidelines and heuristic evaluation lists, considering the characteristics of older adults described in previous studies. Table 1 shows the problems of the existing application (EA) that do not follow improvement factors in design presented in previous studies. Table 2 summarizes the seven items applied to the IA. Among the design improvement elements listed in Table 2, alphabetic indicators were added to denote which elements were applied to the figure.

The design improvement factors between the EA and IA are as follows. The main screen was rearranged in terms of layout and interaction, considering the importance of elements and the sequence of application use (Jo and Lee, 2018). It was divided into two parts: (1) functions requiring measurement and action (rehabilitation exercises, measuring the knee range of motion (ROM), and gait assessment) and (2) functions requiring simple checks and enabling automatic measurements (assessing pain levels, self-medical examination, and step counting). In the main menu, the less important item was moved down. The changes from 1, 2, 3, 4, 5, 6 to 2, 3, 4, 5, 6, 1 can be seen in Figure 1. As the overall layout remained unchanged, Figure 1 selectively illustrates the portion where the menu order was modified, highlighting the movement of item 1 to the last position (Figure 1).

Figure 1. Improvement of the application's main page design (Table 2. (e) Categorized by priority into key and additional information, (g) Place in logical order), (Only the modified portion of the layout is displayed to emphasize how item 1 was repositioned to the last sequence (item 6), rather than presenting the entire redesigned layout)

Modify Delete

In rehabilitation exercises, the main heading was changed from "essential exercises" to "rehabilitation exercises," the button size was increased, and the space between images and text was expanded (de Almeida et al., 2015; Harte et al., 2017). The screens for rehabilitation exercises before and after improvement can be observed in Figure 2.

Figure 2. Improvement of the application's rehabilitation exercise page design (sitting in a chair and stretching your legs), (Table 2. (c) Zoom in on control button size)

Modify Delete

In gait assessment, the title was changed from "gait check" to "gait assessment," and in measuring the knee ROM, the term was changed from "daily joint angle measurement" to "measuring the knee ROM." Additionally, in both gait assessment and measuring the knee ROM, assistance was added to indicate today's measurement status and help with measurement preparation (Barros et al., 2014; Morey et al., 2019). Furthermore, a sound notification was introduced during the 3-second waiting period before starting to walk (Kascak et al., 2014; Silva et al., 2014). The improvements of before and after can be seen in Figure 3.

Figure 3. Improvement of the application's gait assessment page design (Table 2. (a) Unification of terminology, (b) Provides help for understanding)

Modify Delete

2.3 Measurement

The experiment began with an introduction to the simple procedural steps and guidance on precautions. Participants were asked to perform tasks based on three of the application's functions (Task 1: rehabilitation exercise, Task 2: gait assessment, Task 3: measuring the knee ROM) without assistance. Each task was then performed once more after a detailed explanation. Thereafter, SUS was evaluated.

After completing Task 1 and before starting Task 2, participants received assistance in wearing the tracker. One step in each task is defined as the period between the initiation or transition of screens and pressing the next button. The flow of each task can be verified in Figures 4-6. In Figure 6, the term "Gait check" was revised to "Gait assessment," and this textual modification is visually emphasized with an underline to help readers recognize the difference between the EA and IA process flows.

Figure 4. Task process before and after improvement of rehabilitation exercise (Task 1)

Modify Delete

Figure 5. Task process before and after improvement of gait assessment (Task 2)

Modify Delete

Figure 6. Task process before and after improvement of measuring the knee range of motion (Task 3). (Textual modifications: "Gait check" revised to "Gait assessment")

Modify Delete

The wearable tracker and application visually display recovery status and progress. The wearable tracker's accelerometers and gyro sensors were used to measure knee angles and walking patterns. These data were then transmitted to a mobile application installed on a smart phone via Bluetooth. Participants' use of the application was recorded using the observation camera (Figure 7).

Figure 7. Data measurement configuration for comparing usability before and after improvement (collected data: performance time, click count, and error count)

Modify Delete

2.4 Analysis

To assess the usability of the service in comparison to the improved version, performance metrics were analyzed by watching experiment footage captured on video. These performance metrics were used to assess efficiency through the following parameters: time taken to perform tasks, click count made to complete tasks, and occurrence of errors during the application usage process. To provide a relative comparison, performance time (Equation (1)) and click count (Equation (2)) were measured against the minimum time and clicks required by skilled experts to perform the task. The same task was performed twice, and usability was measured using the SUS questionnaire.

Relative performance time = {(Participan's time - Expert time)/ Expert time}*100 (1)

Relative click count = {(Participant's clicks - Expert clicks)/ Expert clicks}*100 (2)

Efficiency was analyzed using an independent t-test. With one independent variable, improvement status (ImSt), influencing five dependent variables (absolute performance time, relative performance time, absolute click count, relative click count, error count), an independent t-test was conducted to determine if there were significant differences in the averages of Task 1, 2, and 3 (p < 0.05).

Figures 8-10 summarize the results for each task and user satisfaction based on SUS scores. There was a 34% reduction in absolute performance time after improvement in Task 1 (Figure 8(a)). Compared to expert performance times, a decrease of 59%, 37%, and 4% in relative performance time was observed in Tasks 1, 2 and 3, respectively, after improvement (Figure 8(b)).

Figure 8. Result of performance time (a: absolute performance time, b: relative performance time; (*) Independent t-test, p < 0.05)

Modify Delete

A 21% and 35% reduction in the absolute click count was observed in Tasks 1 and 2, respectively, after improvement (Figure 9(a)). Regarding relative click count compared to expert click counts, there was an 88%, 92%, and 36% decrease in Tasks 1, 2 and 3, respectively, post-improvement (Figure 9(b)).

Figure 9. Result of click count (a: absolute click count, b: relative click count; (*) Independent t-test, p < 0.05)

Modify Delete

The error count decreased by 88%, 89%, and 50% for Tasks 1, 2, and 3, respectively, after improvement (Figure 10(a)). Regarding SUS scores, a 9%, 13%, and 10% increase was observed for Tasks 1, 2, and 3, respectively, post-improvement (Figure 10(b)).

Figure 10. Result of error count (a), Result of SUS score (usability satisfaction) (b) ((*) Independent t-test, p < 0.05)

Modify Delete

According to the independent t-test results, a significant mean difference between pre- and post-improvement was observed in the absolute and relative performance times for Task 1, the relative click and error counts for Task 2 and the absolute click count for Task 3. There was no significant mean difference in SUS score between pre- and post-improvement for all tasks.

Older adults lack an intuitive understanding of applications they encounter for the first time, making it difficult for them to use an application without a detailed explanation of unified terminology and preparations. This study identified which design improvement factors positively influence application usability. We found that terminology unification and help provision have a positive effect on the reduction of the error count and the SUS.

While previous studies have developed guidelines for the heuristic evaluation of existing applications, it is difficult to determine whether these guidelines are valid. We thus conducted an experiment divided into the pre- and post-improvement stages to confirm the influence of the factors to which the guidelines were applied. For Task 1, rehabilitation exercises, standardizing terminology, adjusting layout spacing for enhanced readability, and enlarging button sizes for users (de Almeida et al., 2015; Harte et al., 2017) led to a reduction in performance time, click count, and error count. The usability of an mHealth application can help older adults more easily master rehabilitation exercises. Mastering these exercises may motivate them to adhere to rehabilitation routines, ultimately contributing to improved overall health and well-being, as well as increased confidence and independence.

For Task 2, gait assessment, application improvements included standardizing terminology, providing sufficient explanations through help, and adding sound alerts (Barros et al., 2014; Morey et al., 2019). Processes were added; however, performance time, click count, and error count decreased. Specifically, relative item results significantly decreased due to cognitive support provided to users during execution.

For Task 3, measuring the knee ROM, improvements were made by standardizing terminology and providing additional help (Kascak et al., 2014; Silva et al., 2014). Relative results and error count decreased. However, absolute performance time and click count increased due to additional processes with help. The experimental results indicate that providing additional help enhances user understanding of the tasks, leading to a reduction in relative performance time and click count. This is interpreted as a means to minimize mistakes, consequently increasing SUS scores.

The positive results of Tasks 2 and 3 related to measurement not only benefit individual users, but also have broader implications for the quality of life of older adults and the professionals and caregivers who support them in their health journey. For instance, medical professionals can monitor patient progress more efficiently and coordinate medical services based on accurate data through the health application.

The study's limitation is that the data analysis did not consider the smartphone proficiency of participants. Participants were divided into EA and IA groups, and could not be evenly distributed based on individual smartphone proficiency. Through observations and surveys during the experiment, it was found that the EA group scored 2.7 points while the IA group scored 2.9 points on a 5-point scale, indicating an approximately 7% difference in smartphone proficiency.

This study conducted a comprehensive usability evaluation of a rehabilitation service application for older adults and applied guideline-based improvements across four dimensions—content, graphics, layout, and interaction. Through comparative experiments between the existing application (EA) and the improved application (IA), we verified that design modifications tailored to the needs of older adults can significantly enhance application usability. Quantitative analysis using performance metrics and SUS demonstrated substantial reductions in relative performance time, click count, and error occurrence, thereby validating the practical effectiveness of the proposed guidelines.

The findings underscore the importance of user-centered design in mHealth applications for older adults, particularly in reducing barriers to technology adoption. Improvements such as unified terminology, enlarged buttons, optimized information layout, and supportive explanations not only reduced operational errors but also increased user satisfaction and confidence. These outcomes highlight the role of design guidelines as essential tools for bridging the gap between application functionality and accessibility for older populations.

Beyond the immediate usability improvements, this study contributes broader implications for healthcare practice. Enhanced usability of rehabilitation service applications can promote adherence to exercise routines, support self-monitoring of recovery, and reduce dependence on clinical resources, thereby contributing to cost-effective healthcare delivery. Furthermore, the empirical validation of guidelines provides a foundation for developing standardized frameworks in mHealth application design for older adults.

Although this study was conducted in a controlled laboratory environment, it did not fully account for environmental and physical constraints that older adults may face during real-world mHealth use. Factors such as lighting, background noise, and device-handling difficulties may influence usability outcomes. Future studies should consider these contextual variables to enhance ecological validity and ensure practical applicability.

Future research should extend these findings by conducting longitudinal studies to assess how usability needs evolve as individuals age, as well as evaluating applications in real-world clinical and community contexts. Integration with wearable devices, telemedicine services, and AI-driven personalization could further enhance the effectiveness of mHealth solutions. Ultimately, this research offers a validated pathway for designing accessible and reliable mobile healthcare applications that can support healthy aging and improve quality of life for older adults.