Factors Affecting the Practices of External Problem Solvers in Broadcast Search

In Broadcast Search (BS) an organization discloses the details of a problem to a community of potential solvers. An online intermediary often manages the process. This study investigates the problem solving practices of solvers. In particular it focuses on the intermediation services and knowledge sources thy use during BS. A questionnaire was distributed to successful solvers. Ninety-three complete questionnaires were returned. The results show that intermediation services can be divided in 3 categories: managing interfaces, supporting collaboration and supporting problem solving. Similarly the sources of knowledge can be divided in: institutional, social and output-oriented sources. The results also show that differences in solver’s use of intermediation services and knowledge sources can be explained taking into account solvers’ experience, organizational context as well as the nature of the proposed solution.

diary organizations need to develop a deeper knowledge of the role and practices of the potential solvers in order to better plan BS (Boimabeau 2009).For example it can be useful to develop more precise specifications for the design of web-based tools and services supporting BS (Yang and Qian, 2012).A better understanding of the role of potential solvers can also activate learning processes in communities of solvers, improving their capabilities.
Previous research has investigated the motivations, individual characteristics and practices of participants in IOI and crowdsourcing (Franke and Shah, 2003;Fichter, 2009;Ebner et al. 2009;Brabham, 2010;Muhdi et al. 2011).However, these results are not always directly applicable to BS.The specific features of BS are expected to change the motivation, ways of working and individual characteristics of the participants.In particular in other crowdsourcing processes solvers collaborate, often for free, and each solver only provides a piece of the final solution.BS, instead, is a competitive challenge in which a reward is awarded to an individual (or group of individuals), which provides a complete solution to the problem at hand (Rouse, 2010;Saxton et al. 2013).The characteristics, organizational context and motivations of participants in BS, as well as their interaction practices, could be different from those of participants in other IOI or crowdsourcing processes.Similarly, the causal relations between individual characteristics, context, motivations and practices, need to be further investigated in BS because they could be different from those found in other, apparently similar phenomena.
The number of studies specifically focusing on BS is limited.Motivation to participate in BS has been addressed in a study by Frey and colleagues (2011).They found that extrinsic motivation is positively related to the making of nonsubstantial contributions while intrinsic enjoyment tends to breed more substantial contributions.They also considered the role of knowledge diversity in explaining the performance of solvers in BS and found that it is positively related to good performance.Jeppesen and Lakhani (2010) investigated the individual characteristics of solvers.They focused on marginality (i.e. the distance between the solver's field of technical expertise and the focal field of the problem) and found that it is positively associated with the provision of winning solutions.Battistella and Nonino (2013) found that participants with different motivations play different roles in challenge based open innovation processes like BS. Shao and colleagues (2012) have found that there is a relation between the characteristics of the challenge and those of the solvers.In particular, higher awards, longer duration

Introduction
Inbound Open Innovation (IOI) consists in outsourcing an organization's innovation processes to external actors (Chesbrough, 2003;Chesbrough et al., 2006).Ideas and technologies can be sourced from other organizations through licences and patents or outsourcing R&D services (Chesbrough, 2006;Kim and Park, 2008).They can also be obtained by submitting a problem to a population of anonymous individuals: a practice known as crowdsourcing (Surowiecki, 2005).In particular, a special case of IOI or crowdsourcing is the so-called broadcast search (BS).In BS an organization (the seeker) broadcasts predefined innovation challenges to unknown external actors, called "solvers" (Jeppesen and Lakhani, 2010).Any individual can participate in the challenge and try to solve the proposed innovation problem.IOI, crowdsourcing and BS are expected to accelerate innovation processes and reduce the cost for generating new products, technologies and processes (Verona et al. 2006;Terwiesch and Xu, 2008;Boimabeau, 2009;Giannopoulou, 2010;Volpentesta and Ammirato, 2013) and also to reduce risk involved in developing new technologies (Iazzolino et al., 2013;Pantano et al., 2013).Solvers in BS are often expert and talented individuals that work outside the boundaries of the organization that formulated the challenge.The seeker expects solvers to have knowledge or capabilities that are not available within the organization (Mahr et al. 2010).This knowledge and capabilities make them be able to solve technical and scientific problems which internal resources are not able to solve, with a relatively small investment.Some companies implemented proprietary systems to manage BS (Huston and Sakkab, 2006).Besides several intermediary organizations like Innocentive or Atizo (Lichtenthaler and Ernst, 2008;Jeppesen and Lakhani, 2010;Giannopoulou, 2010;Frey et al. 2011) provide services to support BSs through their websites.
BS has raised the interest of both practitioners and researchers during the last few years (Huston and Sakkab, 2006;Jeppesen and Lakhani, 2010;Corvello, 2013;Frey et al. 2011;Mahr et al. 2010).However, the motivations of solvers in BS, their individual characteristics and working practices remain largely under-investigated (Corvello et al. 2013;Feller et al. 2012).
Knowing the motivations, practices and characteristics of solvers, and in particular successful solvers, is relevant for the successful implementation of BS (Shao et al. 2012).Open Innovation processes, indeed, are based on the division of innovative work between independent actors (Chesbrough, 2003).A successful implementation of IOI processes requires the participating actors to understand the complementary roles of their partners.Both seeker and interme- (Tran et al. 2012), prestige/reputation and enjoyment in solving a problem have been found to be motivating factors (De Carolis and Corvello 2006;Tran et al. 2012).In several studies on motivation has been found that past experience with a task or phenomenon has an impact on motivation (Bayus, 2010).Experience changes the way individuals perform a task (Banerjee and Campbell, 2009).In problem solving processes an important phase is the acquisition of the knowledge needed to develop solutions and evaluate the most effective one (Mortara et al. in press).Experience as an inventor is expected to change the knowledge acquisition practices of solvers.
H1: There is a statistically significant difference in the perceived usefulness of different sources of knowledge between experienced and not experienced inventors.
The experience of solvers as inventors is expected to change the way solvers interact with the intermediary organization (Calderini et al. 2007;Leimeister, 2009;Breazeal, 2013).Innocentive, for example, provides support in different phases of BS.Experienced inventors are more familiar with the problem solving process and, as a consequence, are expected to demand a different type of support from the intermediary.Some of these services will be perceived as more useful by not experienced solvers.H2: There is a statistically significant difference between experienced and not experienced inventors in the perceived usefulness of the services provided by the intermediary in BSs.
Depending on their organizational context, solvers are exposed to different sources of knowledge (Lewrick et al. 2007).Besides they learn different work practices (Martins Leal andCoelho, 2006, Volpentesta et al. 2012).We expect the practices they experience in their daily work are transported (with the necessary modifications) in BS.As a consequence both the knowledge sources and the use of Innocentive's services are different for solvers working in different organizational contexts.In particular, workers of research organizations are expected to reproduce the problem solving practices of scientific research.Compared to solvers working in non-research organizations, they are expected to use sources of knowledge like scientific journals and academic conferences to a larger extent.
H3: There is a statistically significant difference in the perceived usefulness of different sources of knowledge between solvers working in research and non-research organizations.Similarly, solvers working in research organizations develop peculiar problem solving practices compared to employees of non-research organizations.When they interact with intermediaries like Innocentive, they are expected to use dif-The aim of this paper is to extend existing empirical evidence on the behaviour of participants in IOI and crowdsourcing processes in several ways.In fact, it focuses on the still under-researched field of BS and attempts to better clarify the variations in the behaviour of solvers (Breazeal, 2013).In particular, it investigates how the practices of solvers vary according to the type of solution (i.e.incremental or radical, systemic or modular), the characteristics of solvers (i.e.their experience as inventors), and their organizational context.This paper contributes to clarify the nature of the market of ideas created through BS.The results can help us understanding if BS creates the opportunity to exploit knowledge that otherwise would have been wasted and can, thus, be considered as a means to make the overall innovation system more efficient.
To investigate the phenomenon described above we conducted a survey to obtain data on solvers' individual characteristics, organizational context, motivations and problem solving practices.Our empirical setting is the online Open Innovation intermediary Innocentive (www.innocentive.com).In order for the results to be representative of successful broadcast search practices, only successful solvers were involved in our study.Questionnaires were addressed to the winning solvers of at least one of the challenges proposed on Innocentive's website.With an explorative intention, we analysed the data using factor analysis (Jae-On Kim and Mueller 1985) and t-test (Dewberry, 2004) as main techniques.Results contribute our knowledge of BS and Open Innovation in general.

Theory and hypotheses development
In BS an organization "discloses the details of the problem at hand and invites the participation of anyone who deems themselves qualified to solve the problem" (Jeppesen and Lakhani, 2010).Like in other crowdsourcing processes, an organization exploits resources made available by a large population of external individuals (Tran et al. 2012).These resources are not available inside its boundaries and developing them internally would be unfeasible (Mahr et al. 2010).However BS has peculiar characteristics that make it different from other crowdsourcing processes (Corvello, 2013).
Several studies have considered the motivations of participants in crowdsourcing processes (Surowiecki, 2004;Brabham, 2010;Leimeister et al. 2012).Both the role of intrinsic and extrinsic motivation has been considered (Frey et al. 2011).In the case of BS a monetary reward is always present.However, other types of motivations have been found to affect the behavior of solvers.Participants with different individual characteristics have been found to have different motivations (Brabham, 2010, Zhao andZhu 1012).Beside monetary rewards, also opportunities for new/better jobs

Method
A survey was conducted to collect original data from solvers that successfully submitted a solution to at least one challenge proposed by Innocentive.Because of the choice to interview only winners, the obtained data are representative of good practices in BS.A questionnaire was distributed by email.The questionnaires were sent to a first group of winners in July 2011 and to a second group in July 2012.In both cases a remainder email was sent after one month to increase the response rate.Response/non-response bias was examined using the Chi-square test of homogeneity (p < 0.05).Age, experience and education were compared between the first mailing (N = 73) to the second mailing (N = 20).There was no evidence of bias.
In our study we hypothesize that solvers' experience, solvers' organizational context and characteristics of the innovation affect solvers' idea generation/problem solving practices.In particular two aspects of the practices enacted by solvers in BS have been considered: the use of the services provided by the intermediary and the use of different sources of knowledge.These are the dependent variables in our study.Solvers' experience, type of organization and type of innovation (Incremental vs Radical and Autonomous vs Systemic) are the independent variables.
The questionnaire was divided into three sections: individual characteristics, idea generation/problem solving process and solution characteristics.In the first section respondents were required to provide information about their gender, age, education and professional situation.All these questions but the year of birth were formulated so that the respondent could choose among different categories.Respondents were also asked to rate with a score from 1 to 5 the importance of five motivating factors: monetary rewards, prestige/ reputation, opportunity for new/better jobs, satisfaction in proving the value of their ideas, enjoyment in solving a problem.Finally respondents were asked the number of patents that list them as authors.The number of patents was interpreted as a measure of their experience as inventors.Section two focused on the idea generation/problem solving process.In particular twelve different phases of the idea generation/problem solving process (obtained through an analysis of the literature) were briefly described and respondents were asked to evaluate the usefulness of the services provided by Innocentive in supporting each phase using a score from 1 to 5 (1= totally not useful and 5 = very useful).Respondents were also asked to rate with a score from 1 to 5 which source of knowledge they considered most useful for their idea generation process/problem solving (1 = not useful at all and 5 = very useful).Section 3 focused on the characteristics of the solution involved (incremental/ radical and autonomous/systemic).In particular we used the ferent services.As a consequence there will be a difference in the perceived usefulness of Innocentive's services between employees of research and non-research organizations.
H4: There is a statistically significant difference between solvers working in research and than non-research organizations in the perceived usefulness of the services provided by the intermediary in BS.
The last variable we hypothesize has an impact on solvers' practices is the nature of the solution they submitted.Innovations can be classified as incremental or radical (Garcia and Calantone, 2002).Usually radical innovations are perceived as more valuable.They are also expected to have a stronger effect on prestige and reputation.Radical innovations are expected to require more complex knowledge.Similarly problem solving for radical innovation is expected to be more complex than in the case of incremental innovations.As a consequence the following two hypotheses can be formulated: H5: There is a statistically significant difference in the perceived usefulness of different sources of knowledge between solvers perceiving their innovations as incremental compared to those perceiving their innovations as radical.
H6: There is a statistically significant difference in the perceived usefulness of the services provided by the intermediary in BS between solvers perceiving their innovations as incremental compared to those perceiving their innovations as radical.
Another common classification of innovations is systemic versus autonomous (Garcia and Calantone, 2002).If autonomous innovations do not influence nor are influenced by the context in which they are used, systemic innovations require/cause changes also in their context.Developing successful systemic innovations requires the acquisition of knowledge related to the context of the problem to be solved.Compared to autonomous innovations, the sources of knowledge are expected to be different.Also the interactions with other actors are expected to change.As a consequence the two following hypotheses can be formulated: H7: There is a statistically significant difference in the perceived usefulness of different sources of knowledge between solvers perceiving their innovations as autonomous compared to those perceiving their innovations as systemic.H8: There is a statistically significant difference in the perceived usefulness of the services provided by the intermediary in BS between solvers perceiving their innovations as autonomous compared to those perceiving their innovations as systemic.

Descriptive statistics
The questionnaire was distributed by email to 300 individuals.Twenty-five emails could not be delivered.Ninety-three complete questionnaires were returned, making a response rate of about 34%.The sample included solvers from large (more than 250 employees) and small or medium organizations (less than 250 employees).Winners from non-research organizations are more numerous than winners from research organizations.The demographic characteristics of the respondents are summarized in table 1.
Experience was measured as the number of patents listing the respondent as an author.The average number of patents per solver was 1.86 with a standard deviation of 1.54.The mode was 1.It is interesting to note that many Innocentive solvers are experienced inventors (they have at least one patent, on average almost two).
In the questionnaire we asked to rate five motivational factors that could have pushed the solver to participate in the challenge.The responses are summarized in table 2. As expected "monetary rewards" is an important motivational factor.However the opportunity to prove the importance of one's own ideas is considered even more important."Opportunities for new jobs or career advancements", instead, is the least important motivating factor.measures proposed by Dewar and Dutton (1986) to measure how radical the solution was.Respondents were asked to rate on a five point scale whether their solution: 1) had no new knowledge contained in the artifact or process; 2) represented an improvement over existing technology and 3) represented a major technological advance.To measure how systemic the solution was we asked to rate on a five point scale the following items by Gopalakrishnan and Bierly (2001): 1) The solution is independent of other products and services; 2) Users need to know other products or processes to use it effectively; 3) Knowledge about other systems is necessary to implement this innovation effectively.Cronbach alpha (Philips, 1971) was used to measure internal consistency and both scales were found reliable.The alphas were .80 for "Incremental vs Radical" (3 items) and .75 for "Autonomous vs Systemic" (3 items).
Data from the survey were analyzed using SPSS statistical software for analysis (George and Mallery, 2000).A series of t-tests were conducted in order to explore the responses and the relationships among the analyzed variable.In order to simplify the analysis (Dvir, et al. 2003), factor analysis (principal component analysis) was applied (Timm 1975;Harman 1976) to the eleven sources of knowledge and to the twelve services offered by the intermediary.The resulting factors served as dependent variables for the t-tests.All the independent variables were transformed in dichotomous variables by finding the mode and defining for each variable two sub-samples: individuals which had a score below the mode and individuals that had a score equal or greater than the mode.component was labeled "supporting collaboration" (items 1 and 2 load on this component).Factors 3, 4 and 12 load on the second component.These items represent the interface between the solver and the seeker.As a consequence the component has been labeled "Managing interfaces".The remaining items are related to the problem solving process carried out by the solver.They load on the same component that, as a consequence, has been labeled "Supporting problem solving".Composite scores were created for each of the three factors, based on the mean of the items that had their primary loadings on each factor.Cronbach alphas showed good reliability in all the three cases: .98 for Supporting collaboration (2 items), .76 for Managing interfaces (3 items) and .89for Supporting problem solving (7 items).
Also the principal component analysis on the items in table 5 (Sources of knowledge) yielded 3 factors that explain 60% of the variance.Again composite scores were created for each of the three factors, based on the mean of the items with their primary loadings on each factor.The items loading on the first component (items number 1, 2, 3, 4, 9) include institutional sources of knowledge like universities, government research organizations or the scientific literature.
As a consequence the first measure was labeled "Institutional sources".The items loading on the second component (items 5, 6, 7) include sources of knowledge implying a person-to-person exchange.As a consequence this measure has been labeled "Social sources".The remaining items refer to sources of knowledge related to the characteristics of the final output and the correspondent measure has been labeled "Output-related sources".Also in this case the reliability has been tested using Cronbach's Alpha.The alphas were .74for Institutional sources (5 items), .75 for Social sources (3 items) and .75 for "Output-related sources".
Table 5 summarizes the results of the t-tests with "Experience as an inventor" as independent variable.Our results suggest that there is a statistically significant difference in the way winning solvers that are experienced inventors or not use the services provided through the web-based platform.
In particular, compared to their experienced colleagues, winners without experience as inventors find more useful the As discussed above, this study focuses on the practices enacted by solvers in BS.We considered two specific aspects of these practices, which we consider relevant to understand idea generation/problem solving in BS: the acquisition of new knowledge used to generate a solution and the use of the services provided by the intermediary organization.Table 3 summarizes the respondents' perception of the usefulness of Innocentive's platform in supporting several aspects of the idea generation/problem solving process.The services provided by the intermediary are perceived as useful above all in understanding the problem at hand.Indeed, formulating the seeker's problem into a clear and attractive challenge is one of the core services the intermediary provides.The platform is also useful in stimulating solvers 'creativity.Even if Innocentive provides services to create teams of solvers and to support collaboration among them, the respondents do not consider these tools particularly useful.
Finally, table 4 reports data on the sources of knowledge used by winning solvers during BS.Scientific literature is the most used source of knowledge, followed by Universities and higher education.Also, the solvers often use patent literature.It is somehow surprising that person-to-person knowledge exchange (i.e.knowledge acquisition from colleagues) is not considered as useful as documental sources.

Statistical analysis
Statistical analyses in the form of independent sample t-tests were performed to further explore the data and the potential relations between the variables.One of the aims of this study was to try to understand how differences in solvers' experience and organizational context as well as in the characteristics of the innovation, affect solvers' practices (in particular practices of knowledge acquisition and use of the platform).
Given the high number of dependent variables our questionnaire considered, factor analysis was used to simplify the analysis (Dvir, et al. 2003) before carrying out the t-tests.
The principal component analysis on the items in table 4 yielded 3 factors that explain 71% of the variance.The first   8) are associated with a significantly higher use of the services provided by Innocentive to support interactions with the customer.This result could be expected since systemic innovations require understanding of the context of the innovation.A weakly significant relation exists between systemic innovations and a higher use of institutional and output-oriented sources of knowledge.

Discussion and conclusions
The aim of this exploratory study was to deepen our understanding of BS by considering the relation between the practices enacted by winning solvers and contingent factors like the experience of the solvers, the organizational context they work in and the nature of the solution they proposed support Innocentive provides for solver to solver collaboration and for managing interfaces with the customer.
Table 6 distinguishes between solvers working in research organizations or not.In general the support provided by the intermediary is considered more useful by solvers working in non-research organizations.Also the differences in the use of institutional and social sources of knowledge are statistically significant with the workers of non-research organizations using them to a larger extent.
Finally table 7 and 8 summarize the impact of the nature of the proposed innovation.Radical innovations (see table 7) are associated with a significantly larger use of social sources of knowledge compared to incremental innovations.for the Innocentive challenge they won.In particular we have focused on differences in the use of services provided by the open innovation intermediary and on the sources of knowledge solvers use.The study has found that several differences exist in the practices enacted by solvers in BS and these variations can be explained taking into account the contingent factors mentioned above.It is one of the few studies based on a survey of solvers of BS.Moreover it is one of the first studies, to our knowledge, investigating the factors affecting solvers' practices in BS.Overall this study proves that a variety of approaches to idea generation and problem solving exist in BS, consistently with previous studies (Jeppesen and Lakhani, 2010;Corvello, 2013;Frey et al. 2011).However this study extends the existent literature in several ways.First a more fine-grained understanding of the role of intermediary organizations has been provided.Previous studies had already considered the services provided by organizations like Innocentive (Zhao and Zhu, 2012;Leimeister, 2012;Corvello, 2013).In this study, however, we improve our understanding of the phenomenon by revealing through a factor analysis that these services can be divided into three categories.According to the function they perform for the solver, these services are used to: support collaboration; manage interfaces and support problem solving.Similarly we contribute to the research on knowledge acquisition in crowdsourcing (Zhao and Zhu, 2012;Mortara et al. in press), by singling out three types of sources of knowledge in BS: institutional, social and output-related.
By analyzing the relation between contingent factors and the variables resulting from the factor analyses we have con-ganizations, could consider objective empirical data (e.g.navigation logs) to study the behavior of solvers.Secondly, the number of responses is relatively small.Third only some contingent factors and some aspects of solvers behavior in BS have been considered.Future works should consider also other contingent factors (e.g.effort required by the problem-solving process, disciplinary background of the solver) and other aspects of solvers' behavior (e.g.problem-solving strategies).firmed several of our hypotheses thus expanding the literature on solvers practices in BS (Leimeister, 2009;Breazeal, 2013).Solvers which have no experience as inventors and who work in research organizations have been found to consider the support provided by Innocentive more useful than the solvers with experience as inventors or working in non-research organizations.Being an employee of a research organization also has an impact on the type of sources of knowledge used by a solver.The use of institutional sources (e.g.scientific journals), or social sources (e.g.colleagues and conferences) is common practice in the Academia and research organizations in general.These results suggest that winning solvers use in a challenge, knowledge they acquired during their main working/inventing activities.Experienced solvers or experienced researchers need less support from the intermediary.Besides it is likely that the knowledge acquisition practices commonly used by researchers are reproduced in BS.Our results suggest the possibility to extend to the case of BS the observation that the behavior acquired in an individual's environment is applied and adapted in new contexts (Banerjee and Campbell, 2009;Pantano and Corvello, 2013).
When considering the relation between the nature of the winning solution and the practices of the winning solvers, the most influential dimension is autonomous vs systemic.For solvers that proposed a systemic innovation, the most useful services offered by the intermediary organization are those that help in managing the interfaces with the customer.This result is consistent with the nature of systemic innovations that require a better understanding of the context of use of the new solution.Consistently output-related sources are more used by solvers that proposed a systemic innovation than by those which proposed an autonomous innovation.From a business point of view, the results of this study are especially relevant for intermediary organizations like Innocentive.A better knowledge of the practices enacted by solvers in BS can be useful for the development of better services and tools supporting solvers work.The value provided by this kind of intermediary increases if they are able to involve larger communities of solvers with stronger competences.This study can be a starting point to develop better services.Besides, our study focuses only on winning solvers.By analyzing their behavior, best practices for BS solvers can be singled out.Our results can be useful to drive intermediaries in stimulating learning processes within communities of solvers.However, this study is also relevant for seeker organizations as well as for communities of BSs solvers that can improve their interactions.
Although this study provides interesting results, it also presents some limitations.First, the study is based on respondents' perceptions and not on objective data.Future research, possibly carried out in collaboration with intermediary or-

Table 3 .
Usefulness of the services provided by the intermediary.

Table 4 .
Usefulness of different sources of knowledge.

Table 5 .
Impact of experience as inventors on winners' practices in BSs.

Table 6 .
Differences between solvers from research and non-research organizations.

Table 7 .
Differences between solvers perceiving their innovation as radical or incremental.

Table 8 .
Differences between solvers perceiving their innovation as autonomous or systemic.