Proposal for a Method for Business Model Performance Assessment : Toward an Experimentation Tool for Business Model Innovation * *

The representation of business models has been recently widespread, especially in the pursuit of innovation. However, defining a company’s business model is sometimes limited to discussion and debates. This study observes the need for performance measurement so that business models can be data-driven. To meet this goal, the work proposed as a hypothesis the creation of a method that combines the practices of the Balanced Scorecard with a method of business models representation – the Business Model Canvas. Such a combination was based on study of conceptual adaptation, resulting in an application roadmap. A case study application was performed to check the functionality of the proposition, focusing on startup organizations. It was concluded that based on the performance assessment of the business model it is possible to propose the search for change through experimentation, a path that can lead to business model innovation.


Introduction
Developing innovative business models cannot be limited to brainstorming sessions.The study of business m odels plays an i mportant role in the first stages of development of startups; mainly because these models are responsible for connecting the created value with its commercialization in the market (Chesbrough 2010;Teece 2010).Given its important role, the conception of a strong business model cannot be narrowed to discussions about its elements, and the adoption of useful tools only for representing it (Euchner and Ganguly 2014).Some startups fail by several possible reasons, even with: the presence of market opportunities, adequate resources, and innovative ideas.An explanation for that could be related with the business model driving the venture (Chesbrough 2010;Teece 2010;Morris et al. 2005).Business models play a fundamental role in the success of innovation, which requires the formulation and articulation of a relevant business concept, as demonstrated in Aranha et al. (2015) to service innovation and in Zilber and Araujo (2012) to e-business adoption.
The literature presents some tools that enable creating representations of these kinds of models, which may help the entrepreneur on making hypothetical representations of a given business model.Nevertheless, despite some recent studies dealing with the assessment and development of business models (Euchner and Ganguly 2014), there is still a lack of studies in that direction.According to Kijl e Boersma (2010) "[...] most of current literature is focused on business model design only, whereas there is almost no attention for business model validation and actual implementation of and experimentation with business models".The research question within the context presented above is how to assess the performance of a business model.
In order to answer the research question proposed, even if hypothetically, the studies of Osterwalder (2004) are taken as a starting point: the possibility of combining the practices of Balanced Scorecard (Kaplan and Norton, 1992) and his business model ontology (which would later evolve to become the Business Model Canvas) (Osterwalder and Pigneur, 2009).
In other words, the hypothesis established for this project is the possibility that the combination of both these methods (Balanced Scorecard and Business Model Canvas) may result in a method for assessing a venture's business model performance.In Osterwalder (2004) there is no delimitation suggested for testing and applying this hypothetical method, inferring its global application.However, this article, as a first approach of the problem situation, will limit the hypothesis verification in startup ventures.
Although there is certain complexity in defining the concept of a startup, the option is made regarding the intrinsic relation between business model innovation and the life cycle of startups.This kind of organization reaches its potential only after testing its technology and/or business model in the market (Hyytinen and Maliranta, 2011).From such perspective, and for allowing us to reach the aim of adapting a method for the assessment of business model performance, this work will seek to:

•
Study the main concepts of Business Model Canvas and Balanced Scorecard, via bibliographical research, enabling verification of the theoretical adherence between both.
• Provide a roadmap approach to the method developed, targeting the replicability of this study.
• Carry out a case study in a startup venture with the purpose of presenting a first application instance for this experimental method.

Theoretical adherence and conceptual proposals
The discussions about the conceptualization of business models, their role and elements have become major points for debates.Notwithstanding, the literature expresses certain concern around the lack of consensus on determining a universal concept for business model (Morris et al. 2005).
In order to meet the purpose of this work, the adoption of elements on the subject was necessary.Thus, the Business Model Canvas, a tool developed by Osterwalder and Pigneur (2009), was adopted as the meta-business model (business modeling method).The Business Model Canvas divides a business model into nine blocks, providing an integrated visual representation that facilitates the discussion and the debate about the business (BERTELS et al, 2015).
Such option resulting from the following reasons: (1) It is the most widely used tool for developing and analyzing business models, as expressed in Bertels et al. (2015); (2) For its availability for representing a business model (Euchner and Ganguly 2014); (3) For its theoretical adherence to a performance assessment method, the Balanced Scorecard, given its origin in the thesis of Osterwalder (2004); and (4) For its capacity to support the creation of a startup (Zaina and Álvaro 2015).
The performance measurement system, Balanced Scorecard, is presented in Kaplan and Norton (1992).It was initially designed as a performance measurement tool, and later evolved as a way to implement the strategy by creating alignment and focus.This system allows the inclusion of financial and non-financial measurements, through four perspectives: financial, customer, internal business process and learning & growth.The success of the four perspectives relies on the fact that the perspectives themselves and the measures chosen are consistent with the corporate strategy (Fernandes, Raja and Whalley 2006).Osterwalder (2004) considered that these perspectives were suitable as a starting point for the creation of his business model ontology.His studies would later evolve and take the current format, introduced as the Business Model Canvas.Considering the logic presented, Figure 1 seeks to show the relation among the nine blocks of Business Model Canvas, overlapped by the four perspectives of the Balanced Scorecard.Other notable examples of performance management tools could be considered for a possible integration with a business model.As examples may be referenced the performance measurement matrix (Keegan, Eiler and Jones, 1989), the performance pyramid (Lynch and Cross, 1991) and the performance prism (Neely et al., 2002).However, the Balanced Scorecard was the performance management method selected, especially for this conceptual compatibility with the Canvas, presented in Osterwalder (2004).Besides the compatibility, the scalability feature of the Balanced Scorecard also show up as factors contributing to the proposed work.In the words of Savioz and Blum (2002): "The scalability of a balanced scorecard makes it interesting and applicable for any company." As the Balanced Scorecard has the ability to transform into action the business strategy through a well-defined process (Fernandes, Raja and Whalley 2006), is purpose of this work suggest steps that business models can also be translated into action.

Conceptual roadmap steps proposal
Based on the theoretical study of the presented tools, a method comprising five steps was established, providing a roadmap application in compliance with the guidelines determined for this study.It has also shown to be appropriate the use of the Balanced Scorecard concept for the determination of iterative processes.Figure 2 shows the roadmap of the method proposed in this article.
Step 1 -Representation of the choices of the startup's business model: For Step 1, one should avail of the study of Osterwalder and Pigneur (2009).However, there is no script with predetermined steps guiding the development of the Business Model Canvas.According to Osterwalder (2004), the order in which one fills out the business model enables a wide range of possibilities.The Business Model Canvas depends, above all, on the understanding of the meaning of each block, and on how the combination of these blocks develops the logic representation of the company's business model.
Step 2 -Selection of performance indicators: As noted in the studies of Kaplan and Norton (1996) 2009).Thus, from the moment an organization defines its business model, it also has its particular set of relevant indicators to select.One may find similar indicators among different companies; however, that does not imply the existence of generic indicators.It is also worth add that we share the concept of Molina-Castillo and Munuera-Aleman (2009) that managers do not attach the same level of importance for different performance indicators.It would be relevant in this context and, along with other factors of the practice of choice of performance indicators, a more accurate study on how to determine the selection of indicators for the proposed method.
Step 3 -Current measurement of selected performance indicators: Once the indicators have been defined for each of the choices of the business model, it is up to the organization's executives to measure them.For the application of the Balanced Scorecard it is important to consider that not all the data for evaluation will be derived from the same source: important data can be spread throughout the organization.To reduce errors of collection of such data, it may be highlighted the relevance of it being centralized at the highest possible level of the company.It is important not to underestimate the effort and energy required to maintain the evaluation indicators.As much as the company's information system is computerized and integrated, there will always be necessary compilation work and analysis (Kaplan and Norton, 1996).
Step 4 -Creation of goals for the selected performance indicators: The Balanced Scorecard requires performance targets to demonstrate the evolution of its strategy.The same should be done in this method to demonstrate the business model performance improvement.The need to set goals is to demonstrate if the business model follows the direction in which the company and its stakeholders want.According to Dewagan and Godse (2014), meet the needs of multiple stakeholders is especially important given the complex ecosystem in which companies work today.Thus, performance indicators should be aligned to organizational goals.It would be incongruent to determine, therefore, incompatible goals with reality and with the environment in which the business model is inserted.
Step 5 -Establishment of initiatives to reach the targeted goals: In the Balanced Scorecard, once defined the goals to be met, i.e. required to achieve the strategic objectives, the planning of initiatives must be executed.The initiatives should be planned similarly to the business model performance assessment method; i.e., the establishment of the desired goals should seek to find ways, actions, and initiatives that enable the achievement of such goals.
Iterative process: After the initiatives have been applied, new measurements shall be performed in order to assess which results the initiatives are providing and especially to investigate whether the stipulated targets are being met.Thus, the method proposed in this research is established in an iterative manner.If after the new measurements it has achieved its targets the following questions can be raised: • Is there a need for new initiatives?
• Is there a need for new goals?
• Is there a need for new indicators?
• Is the business model performance validated?
• Were the appropriate initiatives selected?
• Were the selected targets inappropriate?

•
Were the appropriate indicators selected?
• Are the choices of our business model inappropriate?
Based on the conceptual study it was possible to draw up a proposal for aggregating the Business Model Canvas and the performance evaluation practices provided by the Balanced Scorecard.It is feasible to argue that the simple conceptual study would allow a myriad of combinations to work out a method as proposed.However, it is emphasized that this study sought rapprochement with the hypothesis of combination of the two methods exposed in Osterwalder (2004).

Methodological procedures
In this research, a set of steps and characteristics were determined aiming the coherence with scientific methodology.The main research basis found in the literature review were books, thesis, and portals (for instance: CAPES journals, Web of Knowledge and Scielo), with the review being fundamental to propound the theoretical adherence for the method developed in this work.And, hence, the theory outlined enabled the development of a case study.
Based in Yin (2013), the case study methodology is not necessarily limited to exploratory research strategies.Hence, its utilization in the description or test propositions is allowed.Such condition of methodological procedure seemed appropriate for the objectives established for this article.The case study research increase understanding of how things work, and allows us to identify important events.
The adoption of this procedure allowed for the first test of the roadmap method, following the objectives presented in the results section of this article.However, for research purposes, and for the first test, the profile of the analysis unit was narrowed.According Hayton (2002), a startup can be understood as a newly started organization, with lifetime varying from eight to ten years.Furthermore, according to Hayton (2002), at that age they are, therefore, organizations in their initial cycle of existence, and have not achieved full operations in a sustainable manner.
The analysis unit selected for this single case study is a company that deals with videos on the Internet.Specifically, the initial value proposition of its business model is the offer of software for managing videos for companies that need to show their videos on the Internet.
The development of the case study proceeded with providing the method and instructing the startup managers on how to use it, not interfering in their decision making.The purpose of such approach was to allow the observation of the performance measurement of the company's own business model so as to evaluate the initial adaptation to the method and the need of adjustment.It is also worth mentioning that the case study was limited to a two-month period of the operation of the analysis unit.This period was considered the minimum time for basic application of the proposed method, i.e., a month for the assessment of the company's current situation, and another month for the evaluation of company behavior and results.
It is emphasized that the choice for a single case study results from this work's main objective: to present a first application for the proposed method.This work seeks to provide the conceptual roadmap framework to enable the performance evaluation of company business models.Thus, we part from the presentation of the framework and of this first case study aiming to propose a variety of future tests enabling the verification of the effectiveness of the proposal in various scenarios.In order to delineate the research, the case study focused on showing the method for only two blocks: "Channels" and "Customer Relationship".The need for this segmentation was due to the broad scope of possible results.The method applied to the nine blocks of the Canvas would entail a broad and extensive discussion, and would hinder a detailed analysis.The choice was based on the observed influence of the block "Channels" over the block "Customer Relationship", actually verified after the measurement of both, as will be demonstrated in the following sections.

Structure for data collection and applying the method
Aiming at an orderly data collection and a structured application of the method, a performance evaluation scoreboard of the business model choices was developed, as shown in Figure 3.Each element of the scoreboard will be explained in this section of the research.
The element "business model block" refers to one of the nine blocks established by Osterwalder and Pigneur (2009): value proposition, channels, customer segments, customer relationships, revenue streams, key resources, key activities, key partners, and cost structure.
The element "choice", in turn, represents decisions made for each of these blocks.For each of these choices, performance indicators are drawn up for the "indicator" element so that these decisions can be measured.The element "current" represents the results of an admeasurement of these indicators at the present time.
With the performance measurement data, performance goals for each of the choices are established in the "goal" element.The goals will provide indexes to be reached for each indicator.The "initiative" element refers to the attitude required for successful goal setting.And, finally, the "results" element represents the values measured after the initiatives have been set.

Business Model Block
Choice Indicator Current Goal Initiative Results It is added that each choice may have more than one indicator, which would provide more than one current level and goal.There is no way to set a maximum number of indicators.However, as it is recommended in the Balanced Scorecard studies must be careful with stipulating a large number of indicators, the same goes for the selection of indicators for the business model choices.
As noted, it is clear the relationship of synergy between the roadmap framework and its scoreboard.With this structure, it was possible to develop a first pilot test in the form of a case study.

Results
Complying with the study design, it was possible to develop a first pilot test in the form of a case study.Following the proposed roadmap, in Step 1 the Business Model Canvas is established for the selected analysis unit, as shown in Figure 4. Steps 2-5 are performed using the Scoreboard structure developed.With the business model of the analysis unit defined in Figure 4, only the choices of the selected blocks will be detailed in this study, i.e., Channels and Customer Relationships, as explained previously.

Applying the Business Model Canvas
Figure 4. First version of the business model of analysis unit, using the Business Model Canvas (Osterwalder and Pigneur, 2009).
Channels: The analysis unit uses the "sales force" as a channel to reach the customer segment "companies that have video on the Internet." The company's commercial department searches and contacts customers that could potentially acquire the resources available in its software.Through the "website" channel, the company can reach a large number of customers because information on the software resources is available there.Note the difference between the two channels in Figure 5. Customer Relationship: In the customer relationship block, the unit of analysis declares that its business model possesses two ways of relating.The first, "personal assistance", is a personal relationship that the company offers its customers.That is, all requests, questions, complaints are dealt with in a personal manner by an assistance team.
The other form of relationship that the analysis unit proposes is "personal dedicated assistance".This type relationship is offered, however, only to a single customer.This client is a large organization that requires a different form of relationship, defined by the analysis unit managers as a "partner".As banks offer dedicated assistance to certain types of customers, the startup focus of the case study does the same for that big client.

Applying the business models Scoreboard structure
The statement of Steps 2, 3, 4 and 5 of the proposed roadmap framework can be performed using the business model scoreboard.Evidently, each step takes place separately and has specific characteristics.However, for presentation and discussion of results, the scoreboard structure will be used to summarize the application of the method.
Channel Scoreboard: In the scoreboard of Table 1, the application is presented for the channel block.
The indicators chosen for the Channel block were: • Number of companies contacted by the commercial department: Refers to the number of companies contacted through the actions of the commercial sector; i.e, offer the software solutions to companies that might be potential customers.

Number of closed deals with companies by the commercial department:
This indicator shows the quantity of companies contacted that decided to become clients of the analysis unit.
• Number of companies that requested budget through the website: Reflects the number of companies that spontaneously requested budget through the website to verify the possibility of acquiring the software features of the analysis unit. •

Number of closed deals with companies through the website:
This indicator shows the number of companies that were converted into customers through budget request via website.As observed, the commercial department was able to contact a larger number of companies compared to the current period via website.At the same time, it also increased the number of deals and had great influence on other panels of the analysis unit as well as the performance of the company's business model as a whole.
Through the website the analysis unit also showed some advances since it has already started the initiative for greater dissemination and advertising.Performance is practically identical in the "current" column comparing with the "results" column, as for deals closed.The "current" column had no closed deals through this channel; the closure of a single deal can be seen in the "results" column.
However, even though the investments in advertising and promotion had been modest, there was an increase in the number of companies that asked for budget analysis unit through that channel.It might mean it is valid to have that channel as long as it is developed as a new key activity, i.e., disseminating the company's solutions.This would alter the way business model analysis unit is designed.
Customer Relationship Scoreboard: Table 2 presents the application of the Customer Relationship block.
The indicators chosen for the Customer Relationship block were: • Average response time for clients: Clients often ask questions about the system's functionality.This indicator seeks to measure how long it takes for clients get answers to their questions.

Number of defects per month reported in software:
This indicator shows the number of defects that were identified in software by clients: important for assessing the quality of software.
• Average response time to company with exclusivity: Refers to the average response time of any request from the company exclusively (partner).Requests like answering questions or technical support, might have its response time measured by this indicator.An event that happened in Customer Relationship Scoreboard: the worsening of the indicators in the comparison between the "current" and "results" columns.That is possibly due to the increase of new customers, as a consequence of the growing results in the Channel Scoreboard, which may have brought a higher demand for the staff of the analysis unit exceeding the company's management capacity.That can be observed in the average response time having increased from 60 hours in the "current" column to 72 hours in the "results" column.For that reason, and given the method's iterative feature, a new indicator can be suggested for future measurements, i.e.: the number of completed requests.Hence, it would be possible to know whether this worsening of the indicator "average response time" is due to an increase in demand or to the team's operational inefficiency.Further, the initiative to create a page with answers to FAQ was not finalized during the period under review of the case study.
The number of defects reported by customers also draws attention.
There was an increase of defects, from 10 reported in the "current" column to 12 in the "results" column, wherein the goal set by the founders of the unit of analysis was zero defect.Though personal dedicated assistance offered exclusively to the partner company managed to maintain its standard, it did not show improvement.Such status of Customer Relationship Scoreboard may reveal the need to create a new initiative for this panel.For instance, hiring new employees is an initiative hypothesis that could be tested by the unit of analysis in order to enhance its performance of Customer Relationship.

Discussion
The application has shown different characteristics of the implementation of the Balanced Scorecard.Although the application of the Balanced Scorecard do not point restriction on changing objectives of an organization, these are usually stable and guide the goals and initiatives of a company.Assess a business model, however, does not seek to achieve objectives, but mostly validate the choices that form the model.Or, in other words, evaluate whether the model choices reveal themselves as valid for the moment of the company.This implies the need for flexibility, so that eventual changes in the choices of a business model may occur.
Business models must have this characteristic for adaptation to be responsive to the competitive environment.As an example, Tongur and Engwall (2014) show that technological changes are lethal to many manufacturing companies.However, also second Tongur and Engwall (2014), "previous research indicates that this is not purely a problem of technological innovation, but is closely related to the inertia of business models and business model innovation.".The proposed method, in this sense, can contribute as a systematic way that provides a condition to manage this need to adapt.
It is easier to measure things that are established, stable and wellunderstood.However, measuring something new, on evolving and dynamic creates challenges, especially on knowing "what" and "how" to measure (Kirchhoff et al., 2013).In the presented case study, this potential instability given the dynamics of a startup business model, it was probably mitigated by the guidance provided by a roadmap and its characteristic to iterate the hypothetical choices of the model.Thus, the work could be aligned with the established initial hypothesis.It was possible to contribute to the literature presenting a method suggestion for assessing the business models of performance, by the integration of the Balanced Scorecard and the Business Model Canvas.However, although it was not that the initial proposal of the work, it was observed in the method the ability to experience solutions as a managerial implication.This potential application will be better discussed in the following section.

Toward an experimentation tool for business model innovation
It is relevant to report the behavior of the method while applying the roadmap framework.It was observed in the Customer Relationship Scoreboard, for instance, that the exercise of adopting such method suggested the founders of the analysis unit that a new indicator and a new initiative be created.Another important result was that on the Channel block the increase in performance of the website revealed this channel as an important choice for new clients to contact the analysis unit.If these satisfactory results are still being confirmed in the coming periods, it may propose a change in the company's business model.
The dissemination of the analysis unit solutions can be considered a new key activity.That would also generate a new choice to consider in the cost structure: dissemination costs.Thus, changes would occur in the business model of the analysis unit, as shown in Figure 6.
The change can demonstrate that the method has the potential to verify the performance of a business model as a way to modify the choices of business models.The method proposed in this article can be targeted in future studies providing a systematic process in the search for innovation in business models, adding to proposals based on design, such as in Zott and Amit (2015).Some authors like Chesbrough (2010) report the experimentation as a factor conducive to innovation in business models.According to Chesbrough (2010), to overcome the barriers of innovation in the business model, experimentation processes, effectiveness and organizational leadership for change must be exercised.
Recent studies have demonstrated the importance of business models for the competitiveness of enterprises.According to Velu et al. (2015) "[…] new firms with a high or low degree of business model innovation are more likely to survive for longer than new firms with a moderate degree of business model innovation.".
On the other hand, empirical analysis of management applied to innovative business models are scarce (Burmeister, Lutgens and Piller 2015).The study presented in this article could be a starting point toward an experimentation tool for innovation in business models based on performance.

Limitations and proposals for future work
The conceptual investigation of the tools studied demonstrates complementarity on one hand, but on the other hand there are other questions unanswered in a more clear sense.Regarding a discussion of the concept, one might question what relationship exists between business models and strategy in an organization.This is likely to mean discussing the use of tools of both, their elimination and synergy.Osterwalder and Pigneur ( 2002) have worked similar situations, but it is not known studies that address comparison and complementarity between such tools.The study presented in this article, however, was limited to providing the theoretical framework for the development of the proposed method.
In this approach there was no in-depth study in defining the elements of a performance management system.The option of this study was to seek an adaptation of methods to assess the performance of a business model, and does not provide all the features and capabilities of Balanced Scorecard applied to a given context.Is worth mentioning that even the Balanced Scorecard sought to evolve their elements over time, and from initial applications.According to Fernandes, Raja and Whalley (2006) "The balanced scorecard, originally seen by Kaplan and Norton as a measurement tool, is now presented as a means for implementing strategy by creating alignment and focus." Although all the elements deserve emphasis seems to be a need for better understanding of indicators for hypothesis choices.An important question seems to arise in this context: What criteria to use to determine if an indicator is suitable for validating a choice in a business model?In the study presented, this situation was absolutely arbitrary and in charge of decision-makers of the analysis unit.Evidently issues such as: definition of short and long-term goals, questions of cause and effect and data collection are other key points that deserve more future attention.
From a methodological point of view, it is necessary to perform new case studies in different contexts to make it possible observe the behavior of the proposed method, applied to a variety of business models.Even though new case studies are needed, methodologically will be important broader quantitative studies.
The possibility raised in the previous subsection, of providing a framework for innovation in business models will depend on the adaptation with innovative concepts.The method currently suggests the possibility of change in different parts of a business model, but not all change necessarily mean the generation of innovation.An important study would consider the business model innovation concepts based on the framework of Linton (2009) about the dimensions of innovation.

Conclusion
The conceptual conditions were created for the application of the adapted method in a startup company.It was also important to provide a framework to assist in organizing the implementation of the roadmap, as well as in data collection.The case study examined the usefulness of the method and found out that by such adaptation it is possible to evaluate the business model's performance of a startup.Confirming, therefore, the initial hypothesis proposed in this work and broadening our understanding of how to solve the research problem posed by this investigation.
The process of creating the business model of companies still seems to be fairly crude and subjective.In other words, they restrict the creation of hypotheses for the design of a business model.The submission of a roadmap application offered procedural characteristics for the design and development of the business model.The iterative capability that was attributed to the method during its development might mean the search for continuous improvement.However, this same characteristic explores the hypothesis of operating as a mechanism of learning by trial and error, triggering the discussion among fields concerned with business models innovation.
Finally, the contribution left by this study is enabling a starting point in the search for methods and tools that allow an objective manner of developing a new business model, measuring the assumptions made in the sessions that discuss and devise a business model.

Figure 1 .
Figure 1.The nine blocks of the Business Model Canvas, overlapped by the four perspectives of the Balanced Scorecard.

Figure 2 .
Figure 2. Flow for the development of the evaluation method of business model performance.

Figure 3 .
Figure 3. Generic Scoreboard of business models performance analysis.

Figure 5 .
Figure 5. Relationship between channels and customer segments of company's business model.

Figure 6 .
Figure 6.Second version of the business model analysis unit, after applying the Canvas Scoreboard

Table 2 .
Customer Relationship Scoreboard