Socio-Technical Factors in Electronic Software Biddings

Outsourcing of services has been an industry trend for more than two decades. In particular, public software acquisitions have followed this trend. Governments are significant consumers of software and these processes may fail due to different causes such as incomplete requirements elicitation, limited budget, lack of tenderers or even proper tenderers. Moreover, the fact of there being only one qualified tenderer may make the process suspect. The aim of this paper is to identify the variables which influence the result of a software tender process. We present a two-step research: first a quantitative approach and then a qualitative analysis. Initially we formulated the hypothesis that the quality of documents in public software tenders was related to the success of the bidding process. We studied 142 public tenders in Chile, which neither confirmed nor denied any relationship between success and quality of the tender document. In the second step, we applied a focus group and a semi-structured online questionnaire identifying N new potentially factors to success. For them, we found social theories supporting these relationships. The main contribution of this work is to propose a wider set of variables, not merely technical ones, which influence the success of software tender processes.

In the case of Chile, government institution purchasing is governed by the Public Purchases Law 19,886, approved in May 2003 and effective since October 2004. This law obliges government units to perform a public tender through the government electronic platform (site 3). In particular, public tenders for software products are available in the form of a summary and a group of attached files including procedures involved, project requirements and legal constraints.
From the customer's perspective, this procedure is considered successful when it results in the participation of several companies; similarly, and with no loss of generality, the process is not successful when there are few or no offers. Finally, a clear sign of failure is when the call is declared deserted, i.e., without suppliers that fulfil the minimum requirements.
From the supplier's perspective, and given the need for economic openness, the selection process of a tender becomes a problem of analysing a massive amount of data. While a national company can have, as a monthly average, between 10 to 20 tenders to analyze, an international company can easily have more than 100 calls to analyze within a short application period. Therefore, the success of the process does not depend exclusively on the client's requirements and constraints but also on the supplier's availability.
In technical literature, public tender processes for software products have been scarcely investigated [68]. Normally, the problem is studied from the perspective of requirement engineering activities. A common assumption is that the call for tenders stage starts after requirements engineering. Thus, the software requirements specification is understood as a previous stage to the call for tenders [4], [7], [14], [29], [73], [77]. However, other sources sustain that what is published in a public tender for a software product corresponds only to a first approach to a requirements specification [4], [11] which means that the requirements specification is part of the project's deliverables.
For the first part, we generate hypotheses to relate observable and objective measures of a successful tender process, to observable and objective measures software of requirements specifications. In Requirements Engineering there are classical properties such as the size of the requirements specification, its traceability, integrity and volatility [8]; many quality indicators of the requirements specification are discussed in the literature [23], [51], [72]. For that reason, we propose traditional requirements metrics and others related to the format of the requirements specification that could influence the success of the tender process. We present a correlation study in order to look for dependent variables.
We initially analyzed a group of 142 public tenders for software development, published by Chilean government institutions over the year 2012. We obtained low correlations between quality metrics and the success criteria identified. Given that the statistical study did not yield acceptable levels of confidence to accept or reject the initial hypotheses, as a second step we applied a focus group approach to the software providers and an electronic interview in order to gather additional variables influencing the results of bidding processes. This interpretative part of the research produced three variables over which we looked for related social theories supporting these qualitative findings.
The main contribution of this work is to discard technical variables as a relevant set of variables influencing the success of a software bidding process. As a concrete research stage, we propose a wider set of non-technical variables as possible factors influencing the success of the bidding process for future study.
The article is structured as follows: In Section 2, we summarize the main findings of a systematic mapping in order to describe relevant related work. In Section 3, we present our research design, as well as assumptions on the correlation between quality items and the success of the bidding process. In Section 4, we present the main results, the analysis of the correlations, and the impossibility of accepting or rejecting the formulated hypotheses. In Section 5, we present a set of socio-technical factors, combined with findings from interviews with suppliers, which are considered to form a set of alternative hypotheses. Finally, in Section 6, we review the main conclusions and propose how to address the agenda of future research.

Related Work
In order to analyse other works related to software tenders, we applied a repeatable and verifiable process of systematic mapping [71]. This included a formulation of a set of research questions that produced the following search string: public tender OR call for tender OR call for tenders OR public bidding OR public bid OR public tendering OR tender process OR software bidding OR requirement quality OR tender quality OR bidding quality OR bid quality. In order to avoid different functionalities of search engines we used only the Scholar Google service, using parameters with at least one of the words applied to only the titles, excluding patents and cites. The initial results showed a list of 281 records (site 10). We then established the next exclusion criteria: citations (the result from Google Scholar includes citations of un-indexed documents), documents previous to the year 2000, documents without author, documents not coming from journals, magazines or traceable proceedings, documents that did not include software as a deliverable. Finally we obtained 19 documents, detailed in Appendix I. In these papers, we reviewed the abstract, introduction and conclusions in order to identify publishers and main contributions. Table 2 presents a summary of the resulting papers grouped by editorial, main goal, methodology approach and software engineering topics.
We classified the goals of the papers reviewed in two categories: (i) to improve requirements elicitation and (ii) to improve the bidding process. For these categories, we associated one category per paper. We found six research methodologies used in these papers: 1. Case study. We classified those papers that explicitly mentioned this methodology and also those papers that showed detailed examples from real life as case study.
2. Focus groups. We classified only those papers that explicitly declared this methodology as focus group.
3. Qualitative content analysis. We classified those papers that had bidding documents as object of study and that used descriptive statistics to show their results as qualitative content analysis.
A third classification item was the variables influencing the bidding process. We have grouped the papers into 6 categories: (i) acquirer experience; (ii) bid format; (iii) requirements specification and quality; (iv) communication between parties; (v) economic issues; and (vi) security and transparency. Some papers argued for the influence of more than one of these variables, therefore the final account does not match to the total of studies. Finally, as a general way of showing the source of results, we noted the publishers of selected papers. Table 2 summarizes all these findings. From these 19 selected documents, we found only one quantitative study [50]. The amount of quantitative studies in a discipline shows the predictive power of findings, of course under statistical assumptions. Qualitative studies normally discover variables or refine current concepts into the discipline in order to define new variables. This quantitative study investigated how differences in format (reduced specification versus full specification) of the bidding process affects companies' bids for software projects [50]. All the other approaches are qualitative ones focused into already defined variables.
Under a software engineering perspective, there is a relevant position about quality as a good predictor of success on software projects. This systematic review has confirmed this position at this early stage of software process, we have found 10 studies (28%) relating quality and requirements influencing the result of the bidding process.
Therefore, as much as we know, there are no study approaches available at the present time which will find correlations between quality of requirements, expressed in the call-for-tender document, and the success of the bidding process. We point this out, because it involves very different research goals: while qualitative approaches look for understanding some phenomenon, mainly in the sense of producing plausible explanations by discovering (identifying) new variables, quantitative approaches look for predicting the phenomenon based on preconceived variables. Under this perspective, to look for dependencies means to accept that a set of variables are influencing the phenomenon (the result of the bidding process), therefore what is next is to collect data to test those hypothetical relationships. However, also is true that when a quantitative analysis fails, means that data collecting failed, or worst, the assumed involved variables are not those really influencing the behaviour under investigation.
To formulate the quantitative part of the study, we considered the most used variables from previous findings (specification and quality requirements, economic issues) would influence the success of bidding processes [75], [79]. We dismissed those less frequently used, such as Bid format and Security and transparency. We also dismissed the variables acquirer experience and communication between parties because they are unobservable variables into tender documents which were our objects of study. In particular, for the case of communication between parties, we assumed that this takes place once the tender has been awarded.

Research Design
This research design corresponds to a quantitative approach to relate requirements metrics, gotten from the call for tender document, to the success of the tender process; this kind of research requires that main assumptions be explicit and hypothetical relationships among known variables be formulated. The next subsection presents this content.

Research Assumptions and Hypotheses
This research was performed with the assumption of favourable results from quality assurance in software construction processes [57]. In particular, we see the tender document of a software product as a software device, that is, an intermediate result in the software process, specifically of an early stage in the specification of the problem to be resolved. In the same way, we understand that the outcome of the bidding process can be either successful or not, and that it is precisely the tender document that is the most important input in which suppliers can offer solutions. Thus, we formulate the general research hypothesis where the success (S) of the call for tender is a monotonically increasing function on the quality of the tender document (Q) as it is shown on equation 1. It means that, given two tender documents: d1 and d2, which have their corresponding qualities q1=Q(d1) and q2=Q(d2), and, being s1 and s2 the corresponding results of d1 and d2, i.e., S(d1) = s1 and S(d2)= s2, then q1 <= q2 implies s1<=s2. S = f(Q) , f monotonically increasing (1) Elaborating on this general hypothesis means finding measurements for quality (Q) and success (S). If we assume that we have more than one way to estimate the quality of a tender, then we can assume that we have an array of n quality metrics for the tender document namely m1, m2 to mn.. The same happens when we come to measure the success of the project, therefore, without loss of generality, we assume that s1, s2 and sk are different metrics for the success of a call for tender. Thus, we have a refined way of the equation 1 in equation 2.
(s1,s2,...,sk) = f(m1,m2,...,mn) (2) At this point, we prepare a set of null hypotheses for each metric si in terms of each metric mj . We formulate this analysis as a classic study to measure the confidence of the connection between two variables (interval/ratio) given a sample using the Pearson correlation r [34].
Thus, we establish a set of null hypotheses of the type.
Hoi,j: r ( mi , sj )<= 0 Therefore we have the following set of alternative hypotheses: Hai,j: r (mi , sj ) > 0 This corresponds to the traditional way of formulating quantitative studies for proving correlations based on data samples. Normally, it is expected that the null hypothesis be rejected, which means to accept its corresponding alternative hypothesis. In this case, to accept that a positive correlation, which implies a monotonically increasing dependency between success of the tender process and quality of the tender document.
Once we have formulated the problem in this way, we need to select the set of corresponding metrics for quality of tender documents and metrics for the success of the tender process. The next step will be to choose the data sample. This paper is available online at www.jtaer.com DOI: 10.4067/S0718-18762019000300104

Metrics for the Quality of the Tender
Software metrics are measurements, derived from software products, processes or resources, whose purpose is to provide a quantitative evaluation of the measurements of certain attributes of those products, processes and resources [8]. Specifically, in requirements engineering, different quality approaches have been used to estimate the good or bad qualities of a requirements specification. All of them take some desirable characteristics be present in the specification text, i.e., getting the value of the metrics implies a content analysis process. However, the step to convert a desirable characteristic of the software requirements specification to one or more metrics has been always debatable. In this subsection we present different references to support that the following set of desirable characteristics of call for software tenders may be dimensioned by already defined software engineering metrics.
The set of desirable characteristics to be considered in software call-for-tender documents are: 1. They must include software functions specifications. It is expected that a software call for tender specifies which are those functions that the software should accomplish.
2. They must include the specification of quality characteristics. It is expected that not only functionalities be specified, but also those features involving crosscutting concerns, such as general constraints, such as technology, and non-functional specifications such as security, usability, performance, among others. All of them constitute a general set of quality features.
3. They must be a complete specification. Although completeness is a feature hard to measure, it is expected that software specifications include diverse specification items which may be certainly observed, e.g. organizational context, desired functions, user profiles, involved business process.
4. They must be a precise specification. It is expected that a software specification does not present ambiguity in its sentences.
5. Finally, although the size is not a desirable characteristic by itself, we have added the goal of having size metrics for two main reasons: (a) all previous characteristics (1-4) need to be expressed into text, and the general size allows having a measurable context (enabling a density value) for them, and (b) Other studies have included the size of the call-for-tender document as relevant characteristic to determine the success of the bidding process [50].
To this set of characteristics, we have selected a set of already defined metrics that approach them. m1: The number of imperatives. The quantification of the number of imperatives is a classic metric in requirements engineering [15], [33], [51]. This metric is a quantification of the number of imperative expressions into the software specification, such as must, have to among others. The assumption is that the number of imperatives is proportional to the number of functional requirements expressed in the specification, and this metric therefore becomes a measurement of the size of the problem to be resolved. However, it can also be used to measure the density of requirements in a specification, in such a way that a high density (many imperatives into few text) can indicate that the majority of the functionalities are stated but not explained. On the other hand, a very low density could indicate deficiencies in the way that the software requirements was specified.
m2: The numbers of words describing quality. Non-functional requirements constitute a relevant topic of study in requirements engineering and there is a general agreement about their technical names (defined in ISO/IEC 9126 [45] and in ISO/IEC 25000 [44]) and what they really involve [17], [31], [51]. As technology specifications are also considered non-functional requirements we add to this metrics the amount of technological nouns, such as names of operative systems, database managers, and programming languages.The goal of the measurement of this metric is essentially to quantify the concepts of quality factors that are mentioned in the requirements document.
m3: The coincidence between call-for-tender titles and the sections proposed in the IEEE830 standard [42]. Although IEEE830 was proposed in 1993, and revised in 1998, it is still a common reference in requirements engineering research [31], [32]. Besides, we assumed that the updated requirements engineering standard ISO/IEC/IEEE29148:2011 was still unknown to government technical units at 2012. Therefore, in spite of completeness involves a semantic and pragmatic point of view, this metric will approach, at least from a syntax point of view, the intention of completeness of a requirements specification. This metric delivers the ratio between the existing titles in the requirements specification and the desired titles from the standard. The range of values is shown in percentage terms (0% to 100%). For the evaluation of title similarity, a proposal was developed to increase the flexibility of the comparison between titles by drawing up a list of synonyms for each term used in the titles of the standard.
m4: The measurement of the number of elements related to imprecisions [69]. This metric is based on the quantification of elements prone to imprecision, uncertainty and multiple interpretations, such as weak phrases in a document. It is characterized as being the definition of precise requirements to avoid ambiguity. We measured by number of elements related to imprecisions. It is a direct measurement on an absolute scale, and its result is a value in the natural number range. In order to keep the hypotheses on a hypothetical monotonic increasing function, its negative value has been used.
m5: The measurement of the ratio between quality expressions and a wide set of quality expressions in a software requirements specification. The object of measuring this metric is basically to calculate the total percentage of different quality expressions addressed in the requirements document. Non-functional words, such as security, interoperability, usability, and so on were considered. Thus this metric approaches quality attributes beyond functionality.
m6: The measurement of the number of words. Measurement by size is frequently used because it has been proved effective in software-related metrics [57]. The size of a requirements specification can easily be measured by the number of words, pages and lines of text. In relation to the whole, the size has implications on the consistency and details of the specification. In order to approach the size using other metrics, we have added metrics from m7 to m11.
m7: The measurement of the number of pages. This metric counts the pages of a requirements document. The number of words would be not enough because it is a common practice adding figures and diagrams into requirements specifications. In order to approach this magnitude, we also count the number of pages.
m8: The measurement of the number of lines of text. This metric counts the lines of a requirements document.
Although it is expected that the number of lines and the number of words are to related variables, their difference gives information about text density, because a greater number of lines implies that more itemized explanations or examples are part of the requirements specification.
m9: Title Count. This metric counts the number of titles in a requirements specification document. Although titles do not coincide to some specification standard, their account gives information about how much a text is structured, therefore, more titles implies an effort for better structuring an explanation of the contents.
m10: Subtitle Count. This metric counts the number of subtitles in a requirements specification document. Is is a complementary metric to m9, but provide additional information about the balance of complexity among text sections, for example, 5 different sections in a document, and only one of them having 10 subsections, shows an unbalanced text in terms of structure. This specific ratio is established in our last considered metric.
m11: Average number of subtitles per title. This metric consists of the ratio between metric m9 and metric m10, in other words, the number of subtitles to each title of a requirements specification document. This metric delivers the proportion between the total number of the subtitles and the total number of the titles contained in the document. Table 3 presents a map of metrics to requirements specification characteristics, size-related metrics may be used to calculate density and proportions for a better approach to a magnitude of a desired characteristic, thus, they have been added in more than a Table's cell.

Success Variables of Public Tenders for Software Products
To describe the success variables of the public software tenders process, we will explain the tender process. To do this, in Figure 1, we have represented the process by using BPMN notations. In this notation actors (roles) are represented by swimlanes, in this case Supplier and Customer. Empty circles represent the initial and possible ends of the procedure, clocks represent deadlines or time constraints, diamonds represent making decisions activities and rectangles represent sub-processes, finally, flows represent information messages and/or sequence controls. These multiple interactions are managed by the ChileCompra platform. First, the customer or acquirer (usually a public administration) publishes the call for tenders and classifies it according to topics and areas (upload C4T document). All public calls for tenders managed through ChileCompra are by definition open, i.e., anyone may submit an offer. The platform supports a suppliers' database, which is also classified by areas and topics of expertise. The platform then sends an email to any supplier whose area of expertise matches that of the new offer (flow from customer to supplier). Furthermore, suppliers can perform a manual search of public calls for tenders at any time according to a set of predefined criteria (search bases for proposals). Once the supplier has decided to enter a public tender process, the acquirer prepares the proposal according to the call for tender document specifications (prepare offers). During this stage, some public tender processes allow suppliers to send questions to the customer, who may answer publicly, to all suppliers at once through the platform, or privately, via email. Also, the customer may organize a public interest manifestation meeting, where the customer and the tenderers can gather to further clarify the actual problem the project aims to solve. Once the public tendering process has been prepared, the tenderer submits its proposal through the platform (Post offer as bases). At a given date (clock), the customer opens all the tenders received and evaluates them according to the set of criteria that was published in the call for tenders (Opening of offers). The evaluation result is finally communicated to all the suppliers by ChileCompra. The customer must publish not only the selected supplier, but also a chart presenting the scores of every tender submitted (Proposal award).  In the tender process explained above, we can identify the stages that involve decision making: the choice of submitting a tender or not, by the supplier, and the choice of awarding the project or not, by the customer. These are two indicators normally used to describe the success of the tender process [41] and we therefore used them as success metrics.

Journal of Theoretical and Applied Electronic Commerce Research
s1: The measurement of the number of suppliers that submit a public tender for a software product. This success factor may be gotten in the Opening of offers stage, in which the acquirer should report how many tenders have applied. In the case of the Chilean e-procurement platform, a formal document reporting the appliers must be uploaded.
s2: The measurement of awarding of the tender. This success measure may be gotten in the Opening of offers stage, in which the acquirer communicates to tenders who award the call. However, also it is possible that none of tenders be awarded because none of them fulfil minimum requirements of the call.

Results and Analysis of Correlation
Our proposed metrics were applied to public tender documents for software products belonging to Chilean government institutions by gathering documents from the ChileCompra platform (site 3). The decision to analyse Chilean documents was mainly due to set technical factors. In order to avoid sample size problems we considered a universe of 142 public tender documents (detailed in Appendix C: Public Tenders for Software Products). In order to process these documents we developed a software tool (detailed in Appendix B: Software Interface to measure 142 LPS), in which the inputs are a set of documents in pdf format and the outputs are the above metrics (m1 to m11). The first step was to separate titles from paragraphs and produce a xml version of the input. Then the documents were processed by counting the selected keywords for the different metrics. In the case of m3, a set of synonyms were considered in order to compare current titles to IEEE830 sections. In figure 2 we show fragments of the output, the intermediate XML file and the resulting values of some of the above metrics. We organized some descriptive-statistics results of the metrics by type of institution, as seen in Table 4, where the average number of providers is ns1 and the % award of the tender is % s2, the number of tenders is N. From the set of 142 public tenders for software products, we observed that 84 belonged to municipalities, from these, 41 tenders (48%) did not have a document structure allowing the identification of goals, functionalities or other common requirements sections. In other words, their similarity to the structure of the requirements specification, IEEE830 is zero. Also in this case, from the set of 84 municipality tenders, 25 tenders (almost 30% of them) were not awarded. A low value of %s2 for municipalities would be explained because this kind of organizations, most of the time, do not have enough technical teams in order to produce good technical specifications. However, in terms of considered metrics, it does not seem that this group has a different behaviour.
A surprising result is the low value of the average of imperative expressions on call for tenders, because these values mean that requirements were not expressed as requisites for the software product. However, in order to better explain this result, we reviewed the individual values and we can see that many of them were zero. We also reviewed other metrics and also we found many values in zero. Therefore we decided to distinguish between calls having metrics in zero and those having greater values. Thus, in Table 5 we show this zero-non-zero analysis considering again the number of imperatives (m1), the number of quality factors (m2), the measurement of similarity (m3), weak phrases (m4), and the average ns1 and percent s2 with the same meaning that they have in Table 4.
This second analysis also does not allow one to get a clear difference among the zero and non-zero groups in correspondence to the success of the their corresponding bidding processes. However, both of these views (Table 4 and Table 5) are just descriptive statistical analyses, i.e., a qualitative result, of the analyzed bidding processes. To contrast the metrics values to the success of the process, we need a more robust method to look for some explanation model which allows, under some statistical certainty, the prediction of the success (or failure) of a bidding process. With this analysis, we plan to establish relationships between the variables that occur before and after the creation of a tender.  According to the inferential statistical approach, to establish that two variables are associated or related is a different study from one to establish that one of these variables provokes the value of the second one [6], [80]. In the first case this kind of relation is called correlation. It is traditional and common to interpret the concept of correlation as positive correlation when a variable tends to increase as the other increases, and its interpretive value lies between the limits of 0 and +1. Proximity to the value of absolute 1 indicates a high level of association, and a lower value indicates a low level of association between the variables, where 1 is the perfect relationship and 0 means that the variables are independent. Conversely, a negative value means that when a variable tends to increase the another one decreases. In this case the minor expected value is -1.
Although, we have made a correlation study, the involved variables occur in different times, first is the call for tenders and then the evaluation of proposals occurs. Therefore, the metrics of the call-for-tender document may be obtained before the fact of choosing a tender or rejecting all of them. Under this assumption, to confirm a positive correlation means to confirm a monotonically increasing dependency function and a negative correlation would mean a monotonically decreasing function. Obviously, a function different of a constant can not accomplish both conditions. In Table 6, we show the results of the correlations between the metrics whose measurements were obtained before submission of offers, which we consider most important in this study, and those variables that occurred later, such as the number of suppliers and the eventual award of some tender.
Based on these results, we cannot determine a statistical or quantitative relationship between our initial hypothesis and results obtained as the confidence levels are low. This is not due to a bad sample since we see that, in spite of low differences between metric values and success results (Table 5), we have two metrics with high confidence levels (m3 and m11); however they present negative values, meaning that the quality measured by these metrics would be inversely proportional to the success of the tender process. To the latter we add that the shortage of imperative verbs and non-functional requirements in the tender documents, together with a summary of the problems already described and the data analysis. We reassert that we are facing a problem that needs to be addressed, and to help solve it we identify the possible key variables in the tender process that could help in decision-making.
In Table 5, it does not surprise us that for m3 (measures of similarity to the standard), the number of tenders not awarded is lower when m3 = 0, and the average of suppliers is higher when m3 > 0. However, the success difference was only 4% (26%-22%) and the confidence of a positive result appears to be too low (60%).
Therefore, we cannot reject the null hypotheses and, consequently, we do not accept the alternative hypothesis of proportionality between document quality and a successful tender process. Moreover, the greater confidence values are showing the real possibility that worst quality of call-for-tender documents implies better results on number of interested providers and awarded providers. In spite of that, it should be observed that all correlation values are low values, i.e., that the supposed relevant variables, as those measuring quality of the call-for-tender documents, are not really the relevant variables to be considered in a relevant and predictive model.
Until here, and on the light of this failed analysis, it seems to be obvious that quality variables are not the relevant variables in order to predict a successful bidding process. However, it shows an open contradiction to all assumptions about that high quality of the stages into a process should result in a successful process. However, this result, may be not the only one showing a quantitative result on this line. The experiment in [49] shows a quantitative approach to look for differences between two formats for the call-for-tender document: a reduced format (incomplete requirements) and a full format. It shows that, for the reduced format it has 17 applicants and for the full version, it has 13 applicants. Anyway, it does not seem to show differences in the success of the process but in the price of the biddings.
Also claimed in this study, this could be explained by the existence of other variables that we have not included and have remained hidden. We conclude that the tenders, regardless of how extensively or informally written they are, have attraction and risk factors that each supplier uses to decide whether to make an offer according to its area of specialisation and its technical ability to carry out a software development project. We hypothesize therefore that certain social and organizational factors exist in the administrators, officials and bureaucrats who work in government institutions, which can interfere with the success and transparency of a tender process, and predominate much more than the quality of the tenders. Hence, these factors are key to the success of the tender process.

Looking for Hidden Factors
This research design corresponds to a qualitative analysis, with the purpose of looking for unexplained behaviours in the first analysis. Although the evidence collected could be interpreted as an inversely proportional relationship, due to negative correlations between the quality for the document and success of the tender process, correlations close to zero are a sign that used variables were not the relevant variables in the success of the bidding process.
While quantitative research looks for relationships between predefined variables, qualitative research principles are mainly guided to improve the understanding by investigating the perspectives of the people in the context within which they act enabling the discovering of variables and the development of new hypothesis during the study [53]. Therefore, we have followed a qualitative approach formulating a general research question about which are the variables influencing the application and award of public tender processes in the software domain.
To approach an answer, qualitative research counts on a wide scope of research methodologies such as, ethnography, focus group, interviews, brainstorming, case study, action research, among others [84]. We have applied two qualitative techniques in order to gather explanations, to discover hidden variables and to formulate new hypothesis dealing with above findings. Firstly, we have applied a focus group to software suppliers and, secondly, a semi-open electronic interview also to software suppliers (different groups) was applied.

Qualitative Instruments and Results
Focus group is a technique to interview a group of people giving them opportunity to react and respond to each other [53]. In this application, six participants were selected based on availability and experience in public tender processes, their profiles are summarized in Table 7. The moderator had previous knowledge of all participants, since they had met in different professional and scientific events in the past. Following focus group technique, one of us played the role of moderator and another that of observer. The session was recorded (audio and video).
Following the protocol, the goal of the research and interaction form was explained to the participants. The issues raised for discussion were deemed to be easy enough to foster confidence in the participants, allowing them to reveal their true opinions in only one session.
Data was coded by two researchers independently and then merged. Coding was performed using the Transana tool. This tool allows the video to be reviewed interactively, thus for a selected video segment the analyst can write the main ideas and then create and/or apply a set of codes. The main findings show that beliefs about lack of capacity of public agents and lack of transparency guided some practices and decisions of applying or not to specific calls.
In order to get a first confirmation, as a second step, we formulated an additional qualitative stage, thus the first general research question was refined into different questions to know the relevance of the perception of lack of capacity of the public actors and a lack of transparency to make the decision of applying or not to some specific bidding process. Consequently, we designed a semi-structured electronic questionnaire using close questions to confirm variables, ranking the relevance of each one from 1 to 7, and open-ended questions in order to gather strategies to deal with lack of capacity and lack of transparency. Also we included additional open-ended questions to get additional variables if any. In order not to force answers to lack of capacity and lack of transparency we added several questions related to other variables for applying, like application period, development time, certification constraints among others. The questionnaire was answered by 25 engineers, with average experience in public sector tenders of 8 years. However only 10 of them completed all answers. In the group of incomplete answers we recognize to one of the participants of the focus group, thus we did not process his answers. Although they were selected for convenience, all of them belonged to companies participating in tender processes in the Chilean system. The questionnaire was applied through the Qualtrics platform (site 11).
In this second part of the qualitative part, to analyze lack of capacity and lack of transparency, we found a third variable taking place: corruption. Therefore , corruption, and the belief of that, was added as a third variable to consider. We remark here a phrase from one of the participants that comments: the provided information was not enough for us to build an offer to apply, however, I indirectly, heard that another provider had formal information to apply more than the already publish it. What was interesting, was that the same participant-provider, in another open-ended question about strategies, He commented, We have contacted other professionals from customers in order to get additional information to apply having a better offer. It is relevant to remark that Chilean law prohibits public offices to contact the potential providers during a bidding process, however, under the point of view of this provider, it is clear that this rule affects the structure of his opportunity, at the same time, there is not a punishment for private actors in the action of asking for additional information, but for public actors of providing them. Moreover, under his experience, under his subjectivity, the software supplier saw that it was an accepted practice, therefore, we recognize the condition of using particular actions on a public office for private gain, which is the condition for corruption [59].
In order to summarize findings from focus group and questionnaires, we present in Table 8 the references to these previously unconsidered variables as the main findings of these two qualitative instruments: (i) lack of competence, (ii) lack of transparency and (iii) corruption. Moreover, we realize that these variables may be present on both sides of the relationship, on the side of the public customer and on the side of the private supplier. Due to this, we examined only the perceptions of suppliers, there are unexplored quadrants that we prefer to let them clear in Table  8. In order to fill the table, we have coded the participants of the focus groups as p1 to p6, and participants of the questionnaire as q1 to q10. In the case of focus group, we have added a specific participant in the corresponding variable cell when he argued for the variable as a reason to the classical question of to bid or not to bid. In the case of participants of the questionnaire, we consider answers from 5 to 7, i.e., the highest score for the perceptions of the influence of the variable and also when they used open-ended questions to mention the variable. Unexplored. Unexplored.
In spite of the relative success of this qualitative part, in the sense of really reaching a set of previously hidden variables, it is also clear that they belong to the ambit of the social construction theories [5], i.e., the variables are mainly human perception having human implications about how people perceive values, norms, powers among other cultural categories. In particular, in the information technology field it is possible to find quantitative studies using this Carlos Cares  [76], [83]. Moreover, the philosophical principles that support this knowledge generation perspective is called critical realism, and its use in information systems research is supported by Mingers [67].
Therefore, as a complementary work to these findings, we present a theoretical background coming from social sciences in order to support that already mentioned variables are related to the software bidding process. However, it should be clear that these arguments only give additional support to above findings however, they are far to constitute established relationships. To do that, at least one new quantitative cycle should be made, starting from the generation of a new set of hypothesis to new correlation, or better, regression analyses. In order to this reach this goal each one of the following argumentations ends with a new hypothesis in the same way that initial quantitative hypothesis were formulated.

Belief in Lack of Capacity on the Part of Customer (V1)
One factor considered was belief in a lack of technical capacity on the part of customer. In other words, unfamiliarity of the supplier with the type of solution requested and the type of customer (for example, a municipality in a rural zone) creates a belief of customer incompetence. From a theoretical perspective related to analytical sociology [32] [54], we suggest considering the beliefs expressed by suppliers about the operation of public organisms and the capacity of public administrators and their technical offices. In theoretical terms, the beliefs are defined as "a proposition about the world that is considered to be true" [35] and, in the same sense, as "the adherence to the full proposition in the way that I believe X, whatever the nature of X is" [9].
These beliefs influence suppliers' decisions and actions, for example, to bid or not to bid. Furthermore they give meaning to their decisions and actions, for example, there is a high risk that a system's final decision could be delayed or that the range of the requirements might remain relatively stable. These beliefs are formed based on the information available from the past experiences of the individuals (suppliers), whether obtained directly or indirectly (by intuition or conversation with others). Another element that we group in the issue of beliefs about capacity is the belief of a position's instability. We found this judgment in the interviews and focus groups, associated with the idea of instability due to the possible change of a minister, a mayor or the government as a whole. It is basically the belief that the public system's capacity is not stable. In addition, we also found a related judgment about high heterogeneous capacities due to high differences in budget structures, e.g. between a provincial Municipality and a Ministry. Seen in this light, we can introduce the concept of competence reputation (CR). From this perspective, it seems rational to establish the conjecture that the success of the tender process is proportional to competence reputation, which would depend on the leader's stability and budget structure.

Belief in a Lack of Transparency in the Tender Process (V2)
One highly relevant argument places in doubt of the transparency of the tender process. This refers to the opacity of proposal evaluations, i.e., the suspicion that the suppliers had already been selected before the tender evaluations were finalised (a cooked call). One symptom (or rule of thumb) by which these calls can be recognised is a very short application period. In this case, as in the former, the belief is applied to the public administrators' actions, using past experience as well as their general reputation for transparency in the social construct of the corresponding public administrators' behaviour.
Confidence is defined as an expectation about how others will behave; this expectation is formed from the information that one has about them, especially their past behaviour [24] and/or their precedents and credentials regarding quality of performance and level of preparation to undertake the required tasks. The former can also be based on game theory, considering the parties to be facing a situation in which they must decide whether or not to cooperate with one another. This situation can be analysed as a game of trust, in which the initial decision to cooperate in an exchange or transaction is based on the confidence existing at the start of the game [74]. Confidence in the leader then becomes a key element in deciding whether or not to participate in the tender process. In this sense, the cooperation of the public administrators, i.e., their compliance with the established rules, is an investment in their reputation that sustains long-term, cooperative relationships [1] [85]. If on the other hand the promises of the public organisms are not believable, but encourage distrust, the suppliers can decide, by retrospective induction, that the most appropriate action is not to participate in the tenders. This helps to explain existing data about the number of participants in the tenders and how they are awarded.
In this context, it should be noted that cases of corruption among public administrators are considered newsworthy in Chile and are widely reported in the press [3]. On the other hand, honesty and the smooth operation of the public system are not newsworthy and so do not appear in the mass media. Belief in generalized corruption is thus found only in specific cases in software tenders. From this perspective, we developed the hypothesis that the success (S) of the tenders is better when the reputation for transparency (TR) of public administrators is bigger. In math terms it can be expressed as there is a monotonically increasing function f, such that: (C2) S = f(TR)

Belief in Inefficiency and Corruption of the Public System (V3)
We define a third variable as the belief in inefficiency of the public system, and corruption as a probable explanation of that inefficiency. We initiate this reasoning when we establish that suppliers' opinions demonstrate their beliefs. In the theoretical terms that we have briefly established, we can indicate that if these beliefs are based on direct experience, it is likely that the organisms and administrators concerned are untrustworthy in terms of their capacities and/or preferences. If that is so, then practices like corruption or inefficient, low-quality management may exist and may form part of the organizational culture of some public organisms. In this section, we can identify these practices as a factor that influences the reduced success of tenders.
Theoretically, we can add that practices can be defined as modes of action and can cause individuals to seek concrete actions, as the framework in which organizations' and individuals' motivations and aspirations are articulated [20]. Thus, it is necessary to ask why these practices are produced, and the answer indicates that formal rules and personal subjectivity are important in these organizations. Formal rules operate like rules of the game, among other things establishing incentives to encourage a specific end, indicating what is allowed or accepted, institutionalising promises between the parties, and defining enforceability mechanisms to guarantee compliance. Subjectivity refers to the set of motivations, preferences, aspirations and expectations that a framework produces, from which an individual interprets and develops a practice [20]. From this perspective on practices, we can explain, in part, the rise of corrupt practices and low-quality management as a product of the formal rules that affect the structure of opportunity of the public administrators within public organizations, influencing their expectations, motivations and preferences, and therefore their actions .
A public organism could suffer a deficit in the configuration of the formal rules that form the basis of its operation if these rules do not clearly and bindingly establish the control and evaluation mechanisms governing public administrators. This could, consequently, produce a context of opacity articulated fundamentally from the asymmetry of information. These organisations could act against the public interest by developing practices of corruption and/or low-quality management (procedural errors, low levels of effort, etc.), which contribute to the formation of the suppliers' beliefs about the lack of trustworthiness of public administrators and public organisms, and in turn affect the success of tenders in terms of the number of participants and how contracts are awarded. The lack of formal control and evaluation rules on the part of the administration could also produce, as an unwanted side-effect, the permanence of corrupt practices and low-quality management. In this sense, we make the assumption that we can quantify with metrics the existence of formal rules (disincentives, in general) of control and demand under what we call quality management of the public system (PMQ), by which we establish the conjecture that the success (S) of tender processes is better when the quality management of the public system is better. In math terms it can be expressed as there is a monotonically increasing function f, such that: (C3) S = f(PMQ)

Belief in Inefficiency and Corruption of Suppliers (V4)
From the previous line of reasoning, we establish that the practices of the public administrators could produce lessons on how to accomplish things, by adapting to a specific environment and then reproducing it. The lessons would be defined by corporate culture, in the sense proposed by Kreps (1996) [58], since individuals have a tacit, shared belief about how others will behave, especially suppliers, which produces a coordination mechanism in the face of contingencies. This corporate culture could be expressed in low-quality administration practices. Gambetta and Origgi (2013) [26] indicate that in a context of repeated interactions, a tacit culture can emerge which rhetorically promises high quality but accepts and replicates mediocrity, without public recognition. The actors, knowing that their abilities and efforts are low-quality, can adapt to the environment through a social norm that minimizes the costs of their efforts by demanding little. This provides mutual benefit as a short-term consequence, but in the long-term it stabilizes the low quality of an organization or market in the hands of groups of mutually satisfied mediocre actors. This theory seems to fit perfectly with the statement of one of the suppliers interviewed: when we asked about the quality of call documents and how he addressed the proposal when this document was poorly specified, he responded that a low-quality call was an opportunity to provide a previously developed product. This practice is obviously only profitable in the short-term; we therefore consider it to be a factor for inefficiency.
The theory of believable promises [22] indicates that a party that assumes a promise is believable given certain conditions -basically, the existence of formal, binding rules -and is required to fulfil the promise because that is the most convenient course of action. This course of action should culminate in compliance due to the structure of incentives and sanctions defined in advance, since the sanctions attracted by the opportunistic behaviour of noncompliance represent a great cost to profit. The rules eliminate the alternative course of action to compliance by making it absolutely unprofitable. In the case studied, formal rules do exist. Chilean law indicates that a supplier, regardless of its offer, is required to fulfil the requirements of the call that is awarded; however there is no supervising organism to verify compliance.
Moreover, if we accept that it is likely that public administrators may secretly select a specific supplier in advance, then it is also likely that this supplier will have participated beforehand, providing information and even solutions outside the established norms. We therefore conceptualize quantifiable, progressive norms and controls over the process which represent control of acquisition management (AMC), and conjecture that the success (S) of the tender processes is better when the control of acquisition management is better. In math terms it can be expressed as there is a monotonically increasing function f, such that: Suppliers conceive their offers for development projects with uncertainty due to incomplete information [10]. This statement refers to the fact that they do not have all the information that they require about who they have to deal with as a relevant uncertain in addition to technical factors. Therefore, If we group these four factors to the quality (Q) of the call-for-tender document, because we still believe that a high quality initial specification may influence the success of the tender process when these new social factors keep low values, then we come to a new hypothesis about that the success (S) of tender processes is better when values of these variables are bigger. In math terms it can be expressed as there is a monotonically increasing function f, such that: This hypothesis includes the elements as a whole, creating a more complete setting in which the suppliers make a decision whether or not to apply. We express this uncertainty in the following questions: Will they really be evaluated justly and transparently based on the technical merits of their proposal? What will happen if they are awarded the tender? Will the public leader fulfil his promises completely and on time? Will unforeseen ex-post contingencies arise regarding the execution of the contract? In this situation, we must resolve the intermediate problems: Do we study the tender or not? Will there be a corrupt supplier in the tender? If so, it seems clear that the framework of uncertainty is not created by the quality of the technical requirements documents, but rather by the elements indicated above, which complicate positive results for suppliers a priori. These elements, on the other hand, can be indeterminate, since they can ultimately depend on specific conditions that make a tender process and contractual execution highly contingent. The uncertainty means that the actors, with limited reasoning (limited ability to process information in a complete environment to which they must adapt themselves), know what could probably and possibly occur, but they do not know what will actually happen. This comes from staying in this niche market and, therefore, increasing the conditions for success results in heuristics, like carrying out only low-cost (pre-made) applications for those public services with a reputation for greater transparency. However, in any of these scenarios, it is clear that products being acquired are not the best ones that public sector may use, whether due to best proposals are not considered or even they do not become a proposal.

Limitations of the Study
All gathered information in the quantitative part refers to Chilean call for tenders in the its public electronic platform. Besides, in order to generate these new hypotheses we conducted a qualitatuve study where only Chilean suppliers were interviewed. Therefore, the first failed quantitative study and the explanations generated by our two qualitative initiatives, are only applicable to the Chilean system and, moreover, applicable to the current socio-political structure of values and power, thus we need to reduce the scope of implications. However, due to the fact that we have found additional variables belonging to the social world, we may speculate that similar societies would present similar behaviours. In any case, a mixed qualitative-quantitative research, in the sense of qualitative discovering and quantitative confirmation, may continue because, even in the case of arriving to quantitative confirmations, it seems reasonable to study the conditions for software bidding success under different cultural settings. Now, considering a broader point of view, and being in agreement that having a set of qualified providers is an acceptable measure of success, it is also true that a better measure of success of a software bidding process is to get and to use the software product for producing the desired change in the target organization. We did not work with information on the final success of the projects, i.e., the success related to a positive use of the acquired software because it complies customers' expectations and business goals. Although we tried to get this perceptions, it was not possible to us by neither using electronic questionnaires nor phone interviews. However, it seems clear that we need a different approach to get this information because we need to distinguish between success as viewed by the purchaser and success as viewed by the user community, and the (social) ROI of the resulting software.
This analysis outlines the impacts of the research presented in this article, arguing that it is necessary to continue researching in order to deepen understanding of these social factors. Specifically, it is necessary to find evidence to determine whether the proposed theoretical mechanisms to explain the operation of the identified social factors operate as we have indicated. In this way, the creation of knowledge about the conditions for success in public tenders will advance. These conditions for success refer to the quality of the written documents as well as the practices of public employees, which cause them and the governmental institutions in which they work to be seen as trustworthy. In this way, suppliers will have incentives to participate in the tender processes. Furthermore, we hope to widen the geographical aspect of the research, since a limitation of this study was that it was carried out in only one country, and therefore relates to only one socio-technical ethos.

Conclusions and Future Work
In this article, we highlight the importance of public tender processes for software products as a recognized part of the software life cycle. We followed a systematic review protocol in order to obtain a measure of this research topic and its main focuses.
Our first conclusion is that, as a research topic, the process of software project tenders has attracted little attention. We found only 19 documents in which the research topic involved software tenders. Most of these (53%) supported the idea that the quality of requirements specification included in the call for tenders influences the success of the bidding process; the remainder (37%) supported the idea that economic variables are also important. Only two studies made correlation studies using small samples (less than 50 cases).
We therefore proposed a first study in order to obtain statistical evidence of these common assumptions. We extracted 142 public software tenders from the Chilean government's public purchasing website, in which 30% of the public software tenders made by municipalities were declared deserted (not awarded). It was therefore an interesting set of cases to study.
We proposed a set of requirements-related metrics, in order to look for correlations between these and the success of the bidding process. However, statistically, we could not establish useful correlations between the quality of the requirements documents and the success of the tender process for software products. In other words, the common assumptions from literature, relating quality of specification documents and success of the bidding process was not supported by our study.
Thus our initial findings were not useful to reject the formulated null hypotheses, and consequently to accept any of the formulated alternative hypotheses. The natural conclusion was that other variables have a bigger impact on the success of the tender process. For this reason, we adopted two qualitative research approaches in order to obtain, from software providers, a different set of variables that influence their decision to apply or not to apply. We then looked for theoretical frameworks that deal with these variables in order to generate a sustainable conjunctures on this phenomenon.
As a result, we formulated a set of new hypotheses remarking the possible highest influence of variables coming from the social world, such as lack of technical capacity of parties, lack of transparency, and corruption, all of them expressed as beliefs of the same. To study these variables requires a multidisciplinary and multi-methodological approach. This implies not only particular efforts on generating creative metrics for reputation, transparency, and corruption, all of which are classical social variables, but also qualitative approaches to gather reliable information from bidding process and their final results.
Future work is needed in order to generate a reliable theory of variables influencing the success of a very particular kind of acquisition: a software project. In the light of the social theories reviewed here, they would appear to belong to the social setting. As a first stage, to have a coarse-grained measure of transparency we are formulating a maturity model for assessing transparency in software acquisitions, due to their observable variables, such as software functions, suitability, learnability, security, among others, and also its very different condition of being an intangible product, seems to justify a differentiated approach.