Measuring end-users' opinions for establishing a user-centred Electronic Health Record (EHR) system from the perspective of nurses

Establishing an acceptable user-centred electronic health record (EHR) system is a challenging task for healthcare providers due to the need for such systems to meet the requirements of its user population. Concerned nurses are the main end-users of EHR systems. Based on knowledge of evidence-based management (EBM) and the issues (goals and methods) of Health Information Systems (HIS) evaluation, this research was performed in four regional teaching hospitals by adopting a quantitative approach research design to perform "goal-based evaluation" research. The results of Path Analysis indicated that 17 of 21 hypotheses were accepted in this study. In addition, the results of One-way ANOVA with Scheff test comparisons indicated that Age is the most important variable in measuring System Quality, Service Quality, Safety Quality, User Use and User Satisfaction; Education is important in measuring Service Quality; and Seniority is important in measuring System Quality, Service Quality and User Use. Furthermore, the results of Linear regression indicated that only one hypothesis is affected by the demographic variable Education. In summary, this empirical investigation provided evidence-based knowledge to explain nurses' opinions of EHR systems success, and to distinguish which demographic variables influence their viewpoints of using such systems in southern Taiwan.


Introduction
In hospitals, a patient medical record involves all information and procedures that healthcare professionals perform in treating a patient, and an EHR system is adopted to record and offer sufficient data for decision-making in both the clinical service of patient care and the administration of health care. Indeed, the development of electronic health records (EHR) systems helps health professionals to enhance the quality of patient care and the efficiency of clinical services [1], since healthcare professionals are not only key stakeholders in patient care, but also end-users of such systems. The aim of establishing a successful EHR system is to improve efficiency, effectiveness, and to reduce medical errors, however an EHR system contains sensitive health data of individual patients [2].
Investing in EHR systems is a costly process. More specifically, establishing an acceptable user-centred EHR system is a challenging task for healthcare providers, due to such systems needing to meet the requirements of its user population. The importance of acceptable EHRs has been previously identified [3]. Moreover, developers who ignore stakeholder needs and work processes in designing an EMR system often fail in implementing EHRs [4]. Accordingly, it is necessary to consider and meet the requirements of stakeholders (end-users) to effectively apply EHRs in a real-world medical environment.
Health Information System (HIS) evaluation reflects on the relationship between the system user, the technology, and the medical environment [5], so it necessarily incorporates both human and organizational aspects [6]. HIS evaluation is not only a key consideration in determining whether a HIS is accepted by health professionals, but also to confirm whether the use of such a emphasized HIS helps/hinders the realization of such goals in a real-world medical environment [2]. Knowledge of evidence-based management (EBM) is used to generate and estimate the required supervision information for healthcare administration in hospitals [7]. The importance of stakeholders has been mentioned previously; therefore, HIS evaluation could be used to realize stakeholders' (end-users) opinions of using EHR systems. Accordingly, the application of HIS evaluation for EBM helps top-level executives to generate sufficient evidence-based information to support such important decision-making in adopting EHR systems.
Evaluation studies should be grounded on scientific theory and rigorous approaches [5]. Socio-technical theory is used to discuss the interaction between people (users), instruments (information system), and conversations (communications) within a social context [8]. Such knowledge has been applied in HIS evaluation, namely to sociotechnical systems (STS). For STS, it provides a complete structure for analysing the interaction between social and technical aspects, and offers a solution to balance both aspects [9]. In addition, the aim of user-centred design is to establish systems which satisfy user's features and tasks; previous studies have identified lack of a user-centred approach as leading to increased costs [10]. Regarding nurses not only play an important role in bedside care, but also are end-users' of EHR systems; therefore, based on knowledge of EBM and goals (i.e. what to evaluate), and methods (i.e. how to evaluate) [11], this research adopted "goal-based evaluation" [12], which focuses on measuring nurses' opinions for providing evidence-based knowledge to establish appropriate user-centred EHR systems in Taiwan.
In term of the S-P-O model: (1) Structure covers organizational and technology aspects; (2) Process equates to human aspects; (3) Outcomes are related to net benefits; and (4) there is feedback from Outcomes to Structure. Organizational aspects include Healthcare Environment (HE) and Organizational Behaviours (OB); Technology (Structure) aspects include System Quality (Sys_Q), Medical Data Quality (MDQ), Service Quality (Ser_Q), and Safety Quality (Safe_Q); Human (Process) aspects cover User Use (UU) and User Satisfaction (US); and the Benefits (Outcome) aspect is Organizational Net Benefits (ONB). It supposed that UU and US of implementing EHRs will be positively affected by Sys_Q, MDQ, Ser_Q, and Safe_Q; there is also a feedback loop from Human Aspects to Organization Aspects. In short, this evaluation framework contains nine evaluation elements ( Figure 1) and suggested that: (1) Organization Aspects will have an affect on Technological Aspects; (2) Technological Aspects will have an affect on Human Aspects; (3) there is an interaction between Human Aspects and Benefit Aspects; and (4) there is an interaction between UU and US. Furthermore, UU and US will have a positive influence on ONB by implementing EHRs in clinical service. Detailed definitions of the nine evaluation elements are given in Table 1.

HE
Realizing end-user impressions of national EHR health policies.

OB
Recognizing end-user impressions of the reasons and motivation for implementing EHR in a hospital.

Sys_Q
Identifying end-user opinions of the performance distinctiveness of the EHR processing it provides.

MDQ
Identifying end-user opinions of the output information produced by the EHR.

Ser_Q
Considering how to provide accessible help to the EHR stakeholders by the technological vendor based on identifying end-users' judgment.

Safe_Q
Identifying end-user opinions of risk management within the EHR.

UU
Measuring the use of the EHR based on end-users' judgment.

US
Measuring the user responses by using the EHR output information.

ONB
Realizing the impact and goodness of implementing EHR in patient care performance based on identifying end-users' judgment.
The function of a research framework is to structure and describe a specific research based on theories/assumptions [19]. Therefore, this study extended the structure of the evaluation framework and proposed 21 hypotheses to achieve the study aim (Table 2). H3: OB will have an affect on MDQ. H4: OB will have an affect on Ser_Q.
H5: OB will have an affect on Safe_Q. H6: OB will have an affect on ONB.
H7: Sys_Q will have affects on UU.
H8: Sys_Q will have affects on US.
H9: MDQ will have an affect on UU. H10: MDQ will have an affect on US.
H11: Ser_Q will have an affect on UU. H12: Ser_Q will have an affect on US.
H13: Safe_Q will have an affect on UU. H14: Safe_Q will have an affect on US.
H15: UU will have an affect on ONB. H16: UU will have an affect on US.
H17: US will have an affect on ONB. H18: US will have an affect on ONB.
H19: ONB will have an affect on UU. H20: ONB will have an affect on US.
H21: ONB will have an affect on OB.

Research design and Data Collection
The triangulation of data (data being collected in different hospitals), theory (examination of 21 hypotheses by analysing data from various participants) and method (related statistical skills) were applied to complete the task of research design. For HIS evaluation research, a user-centred design must perform for real users in real environments [20]. Hence, this study adopted an appropriate questionnaire with real end-users (health professionals) to collect data for testing the aforementioned 21 research hypotheses in the sample hospitals. There were seventyfour structured questions and one free-text question in this questionnaire; answers were assigned a value of 1 to 5 from "strongly disagree" to "strongly agree" by using the Likert-scale format. The reliability and validity of such an evaluation questionnaire has been identified [21]. Participants were requested to fill out this questionnaire anonymously during the period 19th February to 18th May 2007.

Sampling
This empirical investigation was performed in four regional teaching hospitals (ownership being religious, university, private and public) that have implemented EHR systems in patient care in southern Taiwan. In order to reduce bias, this study only focused on inpatient and outpatient EHR systems. In addition, we only invited nurses who need to use EHR systems in their daily work as participants in this survey.

Data Analysis
In

Results
Seven hundred and ninety one (791) valid questionnaires were collected for data analysis; the detailed information of participants is displayed in Table 3. Confirmatory Factor Analysis (CFA) was performed to determinate which questions were suitable for PA [22]. Previous researchers have indicated that both CFA (measurement model) and PA (structural model) of SEM should be conducted by measuring the model fit [23], [24]. Generally, model fit of SEM could be estimated by using the following indices: (1) (5) one badness (parsimonious) fix index [28]. The detailed results of CFA are presented in Table 4. In Table 4, 15 questions were deleted based on the results of CFA in the revised measurement mode, 59 questions retained for nurses. In addition, all values of Likelihood ratio were less than 3 and greater than 1 (and range from 1.362 to 2.762) in the revised measurement mode, too. All values of GFI, AGFI and CFI were greater than 0.95; the results of RMR were less than 0.05. In short, all these results confirm that there is a good fit between the observed data and the hypothesized model in each evaluation element. The results of PA by using Maximum Likelihood Estimates are presented in Table 5 and Figure 2.
As shown in Table 5, the revised model fit is considerably better compared with the initial one.    Table 6, the results of One-way ANOVA with comparisons of Scheff test indicated that participants' opinions of Sys_Q, Ser_Q, Safe_Q, UU and US are affected by Age; Ser_Q are affected by Education; and Sys_Q, Ser_Q and UU are affected by Seniority. The results of the Scheff test indicate that the subgroup 20-30 year-olds is the most significant one among the elements of Sys_Q, Ser_Q, Safe_Q, UU and US within the variable Age. In addition, junior college is the only significant subgroup within the variable Education. Moreover, working experience for 1~12 months is the only significant subgroup in the variable Seniority. It is harder to attain significance in post-hoc testing due to such tests protecting against type I error; the Scheff test is especially tough on rejecting the null hypothesis [33]. In short, within the nurses group, Age is the most important variable in measuring Sys_Q, Ser_Q, Safe_Q, UU and US; Education is important in measuring Ser_Q; and Seniority is important in measuring Sys_Q, Ser_Q and UU.
• * path coefficient is significant at the 0.05 level •** path coefficient is significant at the 0.01 level •*** path coefficient is significant at the 0.001 level • N.S. means a path coefficient is not significant at the 0.05 level The results of PA indicate that 17 of 21 hypotheses were accepted, therefore, there were eight dependent variables (Organizational Behaviours, System Quality, Medical Data Quality, Service Quality, Safety Quality, User Use, User Satisfaction and Organizational Net Benefits) in the regression models (Table 7). In order to recognize the association among demographic variables within the 17 accepted hypotheses, each dependent variable for each demographic was compared with the reference categories by calculating antilogarithms of the regression coefficient (B). Briefly, adjusting for gender, age, education and seniority in each accepted hypothesis, the regression coefficients of evaluation elements increase to a large extent. According to the results of Linear regression within participants, only H13 is affected by a demographic variable (Education).

Discussion
The purpose of adopting CFA is to determinate which questions are suitable for the PA. Accordingly, compared with the original questionnaire, 59 of 74 questions were retained based on the results of CFA. This result identified that CFA is an appropriate tool for refining questionnaires and to help explain the meaning of those retained questions in this study.
Based on the results of PA in Table 5, this study identified that organizational and technology aspects (Structure) will affect human aspects (Process), and human will affect an organization's net benefits (outcome). In addition, the results of PA also indicated several significant findings: (1) for Organizational aspects affecting Technology aspects (H1 to H5), these results confirm that end-users consider that the Healthcare Environment (external organizational aspect) affects Organizational Behaviour (internal organizational aspect) in the adoption of EHR systems. This is consistent with previous research which shows that health policy plays an important role and affecting Taiwanese hospitals in the adoption of EHR systems in patient care [34], [35]. (2) For Technology aspects affect on Human aspects (H7 to H14), only H9 is not accepted in this study. Such a result confirms that end-users consider User Satisfaction to be positively affected by System Quality, Service Quality and Safety Quality. H8, H10, H11, H12, and H13 are consistent with previous HIS evaluation research in Taiwan [2], [36], [37]. (3) In term of human aspects, the results indicate that User Satisfaction is not affected by User Use, but User Use is affected by User Satisfaction. Regarding the User Use questions confirmed by CFA, the retained questions focus on comparing the functions of EHR with paper-based records, the number of PCs using EHR systems, and the patient record data entry interface. Consequently, nurses will use such systems when they need to, and User Use will not affect User Satisfaction. However, their satisfaction will be affected by their willingness to use such systems. As a result, User Use is affected by User Satisfaction, and this is consistent with previous HIS research in Taiwan [37]. In other words, no interaction between User Use and User Satisfaction of human aspects was found in this investigation. (4) A feedback loop is generated from Benefits Aspects to Organizational aspects (H21) which signifies that Benefits Aspects negatively affect Organizational Behaviours. This result shows that if the current EHR systems are accepted by end-users, such systems will increase a hospital's net benefits (eg. increase working efficiency and reduce cost), and that Organizational Net Benefits will reduce a hospital's Organizational Behaviours (eg. the functions of EHR systems will The results of PA also indicate that four research hypotheses (H9, H15, H17 and H20) were not supported (direct affect) in this study. For H9, a positive indirect effect exists (Medical Data Quality → User Satisfaction → User Use, β=0.042). This means that nurses' judgement between Medical Data Quality and User Use will depend on their opinion of User Satisfaction (whether or not using EHR systems are better than paper-based records and easy to operate). In addition, there are neither direct nor indirect effects in H17 and H20. A possible reason for this is that nurses consider they are forced to use EHR systems, so Organizational Net Benefits is not affected by User Use. Also, irregardless of whether the adoption of EHR systems increases or decreases a hospital's net benefits, their opinion of User Satisfaction is not affected by Organizational Net Benefits. Moreover, as previously mentioned, the failure of implementing EHR could be attributed to developers ignoring users' needs and work processes in designing an EHR system [4]. This empirical investigation indicated the importance of user-centred in implementing EHR systems. Briefly, each EHR systems stakeholder has their own opinion. Accordingly, the results of PA provided a guideline to discover potential problems among nurses: (1) recognizing what should be improved (not-supported hypotheses) to meet nurses' needs; (2) concerning how to increase nurses' opinions of using EHR system; (3) understanding what will effect nurses' opinions of using such systems.
Based on the results of One-way ANOVA with post-hoc test (Scheff test), Age is the most important variable in measuring System Quality, Service Quality, Safety Quality, User Use and User Satisfaction; Education is important in measuring Service Quality; and Seniority is important in measuring System Quality, Service Quality and User Use. These results are consistent with previous HIS evaluation studies -namely that Age, Education, and Seniority will affect nurses opinions about using EHR systems [37], [39]. However, the results of Linear regression indicated that only one hypothesis (H13) is affected by Education. In other words, the other 16 accepted hypotheses of PA were not affected by demographic variables. In H13, when measuring the relationship between Safety Quality and User Use, nurses' opinions of User Use will be deceased by Bachelor of Education; however, it will be increased by Safety Quality. Executive managers and Developers of EHR systems need to consider this finding.

Conclusion
This empirical investigation provides clear evidence to explain nurses' opinions of EHR systems success, and distinguishes which demographic variables influence their viewpoints about using such systems in southern Taiwan. To sum up, this study offers evidence-based information for developers/researchers of EHR systems to establish an appropriate user-centred EHR system from a nursing perspective.