RAPID ASSESSMENT OF INSTITUTIONAL RESEARCH ETHICS CAPACITY : A CASE STUDY FROM ZAMBIA

Recently, there has been a remarkable increase in biomedical research being conducted in low and middle-income countries. This increase has brought attention to the need for high quality research ethics systems within these countries and a greater focus on research ethics training. Though most programs tend to concentrate on training individuals, less attention has focused on institutions as the target of such training. In this paper we demonstrate a rapid approach to evaluating institutional research capacity. The method adapts the Octagon Model, which evaluates institutional research ethics using eight domains. The framework was applied to the University of Zambia in order to conduct a baseline assessment of university research ethics capacity. Internal and external assessments were conducted. The domains of working environment and proper skills scored highest, while relevance, target groups and identity scored lower. Consistent with previous work, a systems approach to evaluating institutional research development capacity can provide a rapid assessment of an institutional bioethics program. This case study reveals the strengths and weaknesses of the university’s research ethics program and provides a framework for future capacity growth.


Introduction
For the past few decades, biomedical research has undergone a process of globalization, with a remarkable increase in the amount of research conducted in low and middle-income countries (LMICs) (1).To further this progress, there has been a push to build research capacity, especially across countries in Africa.In recent years, several international organizations such as the African Institute of Biomedical Science and Technology, The Wellcome Trust, and the World Health Organization began initiatives aimed at increasing research within African institutions and improving training for African scientists (2)(3)(4).
This growth also generated a parallel expansion of efforts to address the ethical aspects of international research.That is, as research capacity increases, questions such as those relating to priority setting, informed consent, and the ability of individuals and institutions to monitor research also arise (1).To help address this growing need, several programs have emerged to help researchers and research institutions in LMICs increase their capacity in research ethics.These programs include those from the Wellcome Trust, Fogarty International Center of the U.S. National Institutes of Health, and the European Developing Countries Clinical Trial Partnership (5)(6)(7).The goals of these programs are often to support the development of research ethics capacity in order to enhance research oversight and research ethics systems, scholarship on locally pertinent topics in research ethics, and locally developed training opportunities (8).
As capacity in research ethics grows, however, it is also prudent to examine where and how programs are targeting their work to ensure that they are tailored to local needs, are properly implemented and significant progress is being made.This allows LMIC researchers and institutions to identify specific targets for improvement.Evaluation is a key part of capacity development in any field; the growth of such evaluations has allowed for more targeted and successful research interventions (9).However, very few publications have focused specifically on evaluating research ethics and bioethics programs (10,11).
Importantly, many efforts to increase research ethics capacity have focused on training individuals and research ethics committees; however, there is also a need to focus on building institutional research ethics capacity to further enable and sustain a culture of ethics (12).As will be discussed further, the "systems" approach that we used in this case study is one example of this type of framework.It is important to note that research ethics systems can be influenced by outside factors such as wider community values, regional or national regulations, and international norms (13).

The Johns Hopkins-Fogarty African Bioethics Training Program
In 2012, The Johns Hopkins-Fogarty African Bioethics Training Program (FABTP) began a one-year institutional partnership with the University of Zambia (UNZA), and specifically with the School of Medicine through the Department of Public Health.The history of FABTP and its partnership model have been described previously in several publications (10,14).The primary goal of the partnership was to help further research ethics capacity within the university.In order to proceed with meaningful engagement and support future benchmarking, a systematic approach to assess baseline institutional research ethics capacity and needs was initiated.
In this paper, we use the UNZA case study to demonstrate a rapid approach to assessing institutional research ethics capacity within the context of an LMIC.Since 2010, FABTP has completed two such evaluations at universities in Uganda and Botswana (10,14).The UNZA case study employed many of the same methods, though it builds on the previously conducted assessments by attempting to produce a rapid version of an institutional assessment.
Our aims are to present the baseline evaluation of research ethics capacity at UNZA, as well as demonstrating the utility of the framework for assessing institutional bioethics development capacity.We begin with background information on the university as a whole and the School of Medi-cine, then discuss the specific model used for the assessment, followed by a discussion of the application of the model in this case study.We aim to demonstrate how the model used in this case study can be applied in other contexts as well.
The University of Zambia UNZA, founded in 1966, is one of six public universities in Zambia and matriculates just under 10,000 students, the great majority of whom are undergraduates (15 ,4).The university is comprised of ten different schools including the School of Medicine, which was founded independently in 1965 and joined UNZA in 1970 (15).In addition to training health professionals, the school also conducts biomedical and public health research.The School of Medicine began offering graduate Master in Medicine degrees for clinical disciplines in 1985 and a Master of Public Health degree in 1995 (15).The commencement of these graduate programs increased research volume in the university in general; and this resulted in calls for capacity strengthening in both health research and research ethics.
As a research university, UNZA incorporates ethics into both its teaching and research practice.As will be discussed further, at the time of the baseline evaluation, these responsibilities fell into three main university domains: the research ethics curriculum, the Directorate of Research and Graduate Studies (commonly referred to as, "The Directorate"), and the Biomedical Research Ethics Committee (hereafter referred to as, "the ethics committee").Each of these addressed a different facet of research ethics and was the focus of this evaluation.The research ethics courses at the university were coordinated through the School of Humanities and Social Sciences and were available for both undergraduate and graduate students.The REC receives sufficient financial support (rate level of agreement on a scale of 1-5) Table 1: Sample questions from the institutional and individual REC member surveys At the time of data collection, the university also operated two other research ethics committees in addition to the above mentioned committee; one focused on natural and applied sciences and another for social sciences(16).These two committees were not assessed in this evaluation.

Methods -Approach to Institutional Capacity Assessment
We took a "systems" approach to our evaluation of research capacity.This meant focusing on the institution as a whole -evaluating the research ethics programs, as well as the organization and infrastructure of the institution.As we have detailed in previous publications, institutional research ethics systems should include the following: 1) a justifiable research agenda, 2) protection of research participants, 3) training for institutional members, 4) the creation of institutional priorities and structures that promote ethical conduct, and 5) strengthening communications with regional, national and international stakeholders, especially institutional leadership (13).
We conducted a rapid assessment utilizing several methods to elucidate the breadth and scope of the university's research ethics capacity.First, an institutional survey was administered to the local partnership leaders (faculty within the School of Medicine) who then gathered information from other university faculty as needed.The survey contained 168 questions on a range of topics including educational opportunities, training for committee members and finances.Individual questionnaires were also administered to members of the ethics committee with questions regarding their specific training and role within the committee (Table 1).The evaluation also included a site visit by the FABTP partnership team to the School of Medicine, which took place in January 2012, as well as a document review to better understand bioethics programs, institutional policies and future plans.
We used the Octagon Model for our evaluation, which was developed by the Swedish International Development Cooperation Agency (17).As designed, this model is intended to evaluate nongovernmental organizations in order determine their strengths and weaknesses (17).The model details eight domains (hence "octagon") that are used to help provide an overall analysis of the organization: basic values and identity, structure and organization of activities, implementation of activities, relevance, right skills in relation to activities, systems for financing and administration, target groups, and working environment (Table 2).Each of these eight domains is rated on a scale of 1 (the lowest) to 7 (the highest) (Table 2).
We have previously detailed how we have adapted the Octagon Model for the institutional research ethics context, using modified criteria (10,14).
Table 2: Ranking scale and description of the Octagon Model domains as applied to institutional research ethics evaluation Our evaluation and octagon score were based specifically on assessing the School of Medicine, The Directorate, and the ethics committee.Two FABTP faculty members ranked each of the eight categories after reviewing all of the data.UNZA faculty members also prepared an octagon score based on their own perception of the institution and the two scores were compared.

Basic Values and Identity:
The School of Medicine Strategic Plan for Operations was finalized in 2012 to help establish the school's goals and vision.While the document did discuss the school's research priorities, the plan only briefly mentioned research ethics, in the context of the school's objective to further develop ethics capacity and in stating the existence of the ethics committee (18).There were no further details regarding how the school planed to meet its ethics objectives.Further incorporating research ethics into the school's strategic plan could help establish ethics as a core element of the school's research program.
School of Medicine leaders involved in bioethics program planning and implementation stated during conversations with FABTP staff that their vision was to expand the scope of bioethics within the school in the near future, and improving research ethics training for faculty was a top priority.They spoke of plans to create a dedicated Staff actively supports the institution management.

Systems for financing and administration
Evaluates the source of the institution's financial resources Are there sufficient funds for the institution to operate and achieve its goals Guaranteed funding Efficient administrative system for documentation of all financial documents and transactions.

Target groups
Target groups are actively participating in the work of the organization

Legitimacy of the institution -working with its networks
Organization has clearly defined its target groups and these groups are actively involved in organizational activities.

Groups involved in the planning and evaluation of these activities
Working environment Determines the larger context in which the institution operates and the role the institution plays within this environment Focuses on laws or frameworks established by the government Organization's activities are aligned with the mission and policies of the government as well as international organizations.
Institution acts as a leader within its environment.
bioethics center that will reach across the school, and become a formal home for bioethics teaching, research and service within the entire university.These goals demonstrated that integrating bioethics within university teaching and research practice were key priorities for UNZA.

Structure and organization of research ethics activities:
We learned from speaking with university faculty and reviewing school documents that research ethics within the university fell under the responsibility of several different individuals and departments.There was an Assistant Dean of Research who was responsible for overseeing research, as well as the Director of Research and Graduate Studies who oversaw all research ethics activities Additionally, The Directorate, an office of the graduate school, was responsible for coordinating research, developing research practice and supporting student research work.The office reported to the Vice-Chancellor of the university and was comprised of sixteen members, two academic staff and fourteen support staff.However, at the time of data collection, The Directorate did not provide a written document outlining its mission or a formal framework for the functions of the office.There was an organizational chart depicting the chain of management for research; however, there was no depiction of how the office related to other units and departments within the university.Such a chart may help clarify the role and responsibilities of The Directorate.
The structure and management of the ethics committee was detailed within the committee's standard operating procedures.There were clear instructions for membership requirements and leadership positions.The committee was made up of a diverse group of members, with representatives from the School of Medicine, University Teaching Hospital, and the Ministry of Health, amongst others.These members represented a range of training backgrounds, and two of the members were external and not employed by the university.However, though the standard operating procedures outlined the roles of the committee members, it was not clear where the committee sat administratively within the rest of university or the School of Medicine.
Responses within the evaluation questionnaire indicated that formal academic courses in research ethics were offered through the School of Humanities and Social Sciences only.These classes were offered on campus for registered students.As of 2012, research ethics classes were not offered within the School of Medicine or School of Natural Sciences.Faculty members were responsible for securing their own ethics training at workshops or courses that took place off campus, including multi-day trainings at universities in other countries.There was some funding available for faculty to attend such trainings.

Implementation of research ethics activities:
As mentioned above, responses from the questionnaire indicated that UNZA offered several research ethics educational opportunities through the School of Humanities and Social Sciences.These included tutorials, academic lectures, and a sub-concentration in bioethics for Masters' students.Students conducting research must have completed coursework in research ethics as well, though there was no general requirement for all students to complete these courses.However, it should be noted that, according to the questionnaire, several critical ethics-related topics were absent from the courses offered to students.These included: informed consent, human rights, and research with vulnerable populations.Additionally, there was no requirement for UNZA researchers to complete any form of training in research ethics.
UNZA faculty stated that while the role of overseeing research within the university was officially the responsibility of The Directorate, in practice this was mainly handled at a departmental level.It was the departmental chairs and academic faculty who guided students and coordinated research within each department.Faculty suggested that the formation of an independent bioethics department or center may help raise awareness for the importance of ethics more broadly and serve as a resource for students, department heads and others.
The ethics committee, in order to accomplish its goal of providing ethics review of proposed human and vertebrate animal research, met once a month to review research protocols.As indicated in the answers to the questionnaire, in total, the committee evaluated 127 protocols in 2011, approximately 100 of which were local studies (i.e., no international collaborators or funding involved).An expedited review process was used for approximately 30% of protocols.In these cases, the chairperson was principally responsible for reviewing the proposals though it was unclear what criteria were used to determine whether a protocol qualified for expedited review.Additionally, the ethics committee was registered with the U.S. Office of Human Research Protection and had a Federal-wide Assurance.Though the committee had documentation for its administrative structure and protocols, there was a need for improved implementation of these policies and improved efficiency.UNZA staff stated during interviews that the work of the committee was hampered by perceived inadequacies in communication between the committee members and researchers, and insufficient administrative staffing within the office.Since the committee was not housed within a specific institutional department or unit, it was unclear which entity was responsible for facilitating staff hiring.It was also reported that the ethics committee did not have a dedicated meeting room to conduct protocol reviews.These elements reduced efficiency within the work environment and made it difficult, at times, for the committee to execute all of its functions.

Right Skills in Relation to Activities:
As stated, the university's leadership expressed that training faculty in bioethics and research ethics was an institutional priority.In conversations with FABTP members, UNZA faculty stated that they felt that, as of 2012, many university researchers lacked adequate knowledge of scientific study design and research theory.
Strengthening training programs for researchers will not only improve the quality of the research within the university, but will also help expedite the work of the ethics committee.The committee was hampered by inadequate research proposals, largely due to the fact that researchers were not adequately trained in conducting studies and preparing such proposals.
The ethics committee provided training for its members in the form of a three-day workshop; continuing committee members also completed this workshop in order to ensure that their training was up to date.

Relevance:
As of 2012, UNZA had developed a vision for the future, though it had not yet begun to implement programs necessary to achieve its new bioethics goals.As stated, the university's main goals were to strengthen its bioethics teaching and training programs, as well as create an independent department or center for bioethics.The university did have several elements already in place to facilitate pursuit of these goals, such as The Directorate and various deans keenly interested in expanding bioethics teaching and training.This existing framework can be utilized to ensure that UNZA carries out its plans for the future; however, as of 2012, this work had not yet started.
One roadblock to fulfilling the school's needs was the high volume of external research proposals submitted to the ethics committee.University faculty stated, and survey data confirmed, that the majority of the proposals reviewed by the committee (approx.54%) were projects that were not affiliated with the university.Though the committee is meant to serve the greater Zambian research community, this large volume of work limited the committee's ability to serve researchers based at the university.However, in 2011 the committee received a grant from the European and Developing Countries Clinical Trials Partnership, which, at the time of this evaluation, it was hoped would further research ethics development.

Systems for financing and administration:
University leaders highlighted that the lack of funding for ethics programs as a major challenge for building research ethics capacity.The university did not have any budget set aside for research ethics related activities.Thus, The Directorate operated without any dedicated funding for its ethics programs.The ethics committee did have a small operating budget and charged fees for reviews in order to raise more.The committee had a full time secretary seconded to it by the Tropical Gastroenterology and Nutrition Research Group, which was funded by external donors.

Target Groups:
UNZA ethics leadership recognized students and academic faculty as their main target audiences for bioethics programs.However, since individual departments were largely responsible for overseeing research and managing bioethics training, they too were stakeholders.Additionally, the work of the university also affected the larger research community both within Zambia and beyond.These groups must be taken into account when considering the bioethics programs within UNZA.As of 2012, students were not represented within the planning or implementation of ethics programs though they represented the primary target group for many of the university programs.Incorporating student input will help tailor programs to student needs and will increase program impact.Student input may also be helpful for crafting new ethics courses and improving existing courses.Strengthening the capacity of The Directorate will improve its ability to coordinate with stakeholders such as departments and faculty.Codifying protocols may also help standardize and clarify the role of The Directorate to the research community within UNZA.
During meetings with FABTP staff, university members highlighted the need to improve communication between the ethics committee and faculty researchers.The committee did have clear instructions regarding what types of proposals must be submitted for review so researchers were aware of the committee standards.However, the distinction between the three research ethics committees within UNZA was unclear.Faculty members wrote in an internal document overviewing research at the university, there was "no clear delineation as to which of the three committees specific proposals must be sent to"(16).This lack of clarity not only made it more difficult for researchers to submit proposals but it also confused the role of each committee within the university.
UNZA partnered with several other universities and organizations to expand the scope its ethics research and training programs.For example, the university maintained a relationship with Tan-ZamBo, a research collaboration between three universities within Tanzania, Zambia and Botswana.UNZA also worked with several other sub-Saharan universities as well as universities in the United Kingdom and the United States through the Southern African Consortium for Research Excellence, an initiative aimed at increasing research and funded by the Wellcome Trust(19).At the time of this evaluation, UNZA did not have a formal working relationship with the World Health Organization (WHO), or specifically with their regional office for Africa, with regards to bioethics programs.However, the School of Medicine did have several collaborations with the WHO on other health related projects.

Working Environment:
UNZA operates within the larger research environment of Zambia.As of 2012, there were national guidelines regarding research with human subjects, though there were no national laws regarding research ethics.There has been progress in the last several years, including the formation of the National Health Research Ethics Committee (NHREC), a government operated ethics committee(20).However, within Zambia, UNZA acted as one of the leaders for research ethics and the ethics committee at the university was one of only a handful within the country and therefore was responsible for reviewing materials from all over Zambia.In addition to the national committee, there were ethics committees housed at the Tropical Diseases Research Centre, Macha Mission Hospital as well as the Excellence in Research Ethics and Science Converge Committee, a privately operated research ethics committee (21,22).
In 2013, the parliament passed the National Health Research Act, which included the formation of the National Health Research Ethics Board (23).The board was tasked with generally overseeing research and ethics programs within the country as well as the abovementioned national committee (23).This Act, and the board specifically, helped establish a national framework for research ethics.Though new, it will hopefully provide support for UNZA and create a broader cooperative community for research ethics development within the country.

Overall Assessment:
To supplement the above, primarily qualitative, descriptions, we also used the Octagon framework to generate overall octagon scores tabulated after completion of the baseline evaluation.Figure 1a is the score prepared by the faculty and staff at UNZA; Figure 1b is the score prepared by members of FABTP.Two members of the FABTP team independently reviewed data and created scores in all eight categories; then differences between scores were discussed and reconciled to generate a final FABTP score.The UNZA team also separately convened to consider data and generate a consensus score.
The FABTP octagon score recognizes strengths in the working environment and right skills domains.Systems for finance and structure score highly as well.The area of most need is relevance, followed by target groups and identity.The UNZA self-assessment demonstrates that the university is confident in its work in several areas: basic values and identity, systems for financing, right skills and working environment.These scores demonstrate that UNZA self-identifies areas for improvement in all categories, though no single domain is weakest and each ranks at least at a "good" level.The discrepancies between the internal and external assessments are important to note as these areas warrant further discussion and investigation.

Discussion
In this paper, we describe a rapid assessment to evaluate the strengths and weaknesses of an institutional research ethics system using a novel approach.This case study builds on previous work by demonstrating a faster, less resource-intensive approach to program evaluation.Our assessment focused on The University of Zambia, and specifically on the School of Medicine.We evaluated the programs individually as well as within the greater context of the Zambian research environment in 2012.We believe that this approach provides a useful initial overall evaluation of UNZA as a research institution that is pursuing enhanced capacities in research ethics.
When entering this partnership with FABTP, UNZA identified four main goals for the year.These were: 1) to evaluate the research ethics capacity of the university; 2) to create a bioethics unit that works across all departments within the School of Medicine; 3) to strengthen ethics teaching, research and service capacity by training public health faculty in research ethics; and 4) to create ongoing research ethics programs for faculty and students focused on training, scholarship and mentorship.These goals demonstrated that building both individual and institutional bioethics capacity was a priority for UNZA and these tangible goals focused on the needs of the university.
Our assessment highlighted several strengths of the UNZA bioethics program, as well as particular areas where additional focused effort will likely help the university reach its goals.The university had several faculty members who were well trained in research ethics and committed to improving the research ethics capacity.The university also acted as a leader within the broader Zambian research community and expanding research capacity within the school will strengthen this role.In order to achieve the goals that it has set for itself, and meet the needs of various internal and external stakeholders, UNZA needs to ensure that the programs it implements are relevant.
Clarifying the roles of the various offices that are responsible for research ethics will help standardize and strengthen each of these departments.A responsibility of one such office could be to con-duct additional and ongoing stakeholder engagement and needs assessments to ensure connectivity between activities and audiences.Including research ethics within the university's strategic plan for research may also help solidify the role of ethics within UNZAs infrastructure.
A comparison of the external and internal assessments was most useful for our joint discussions between FABTP and UNZA (Fig. 1).The general similarity of the octagons overall demonstrated some consistency of outcomes, while the differences reflected either variations in perceptions or understanding of the approach -though all involved in scoring the categories used the same rubric and guidelines in order to reduce scoring errors.It is also notable that external and internal assessments for research ethics capacity were most divergent around what were more generally the weaker aspects of the institutional research ethics system.That is to say, both UNZA and FABTP identified similar areas for improvement, though scores attributed by FABTP were somewhat lower in those areas.We believe that the Octagon Model is a useful tool for evaluating baseline program capacity, even though it has limitations.As employed in this assessment, the approach was conducive to understanding general (macro-level) information on various topics; however, more detailed information was often left out.For example, though the questionnaires gathered information on courses offered in research ethics, there was no information as to the quality of these courses or how many students enrolled in them.Additionally, in this case, the assessment did not reflect the opinions of all stakeholders within UNZA.
Most notably, we were unable to speak with university students.In this rapid assessment, we did not conduct specific focus groups or in-depth interviews with students and faculty as we have in previous case studies (10,14).These tools allow us to gain more information and a wider range of opinions regarding ethics capacity (10,14).
Despite these limitations, we believe the approach used provides a reasonable means to initiate empirically-informed institutional capacity development in bioethics, and perhaps more broadly.A limited number of tools are available for evaluating research ethics capacity, especially within LMICs; with even fewer incorporating institutional and other contextual elements into the assessment (24).The Octagon Model provides a multi-level framework and allows institutions to use data specifically to target efforts in domains that are typically administratively and organizationally meaningful.This case study in particular demonstrates the utility of an assessment that can be performed with fewer resources and in less time as compared with other evaluations.We believe that this type of rapid systematic approach will be helpful for further evaluations and should be applied to other institution in order to refine the approach and make it more user friendly in the future.

Figure 1a .
Figure 1a.Internal octagon evaluation score prepared by UNZA

Table 2 :
Ranking scale and description of the Octagon Model domains as applied to institutional research ethics evaluation