Revista Ingeniería de Construcción Vol. 25 No. 1, pag. 05-20

A learning model for design-build project selection in the public sector

Alfonso Bastias*¹, Keith R. Molenaar**

* Universidad del Desarrollo, Santiago. CHILE **University of Colorado, Boulder. USA

Abstract

The primary method of public sector project delivery in the United States (U.S.) has traditionally been design-bid-build delivery. The public sector has historically separated design and construction contracts. In the 1990s, the U.S. public sector began to experiment with design-build project delivery, which combines design and construction in one contract. In 1997, a decision support system was developed to provide a formal selection model for public sector design-build projects. The model supports public owners in determining which projects are appropriate for design-build delivery. This initial model was static in nature and was based on a regression analysis of 104 projects. The analysis resulted in a predictive model with five performance criteria: overall satisfaction; administrative burden; conformance to expectations; schedule variance; and budget variance. Since 1997, the number of design-build projects has increased dramatically and public sector design-build methods have evolved. The original model can be improved with new data and a new framework to provide for an adaptive model as the industry continues to evolve. This paper presents a formalized application and use of learning capabilities to supplement the original static model. This model adjusts parameters and functions using artificial intelligence as the main knowledge engine. This approach can be adapted to many applications of decision support in the design and construction industry.

Keywords: Design-build, learning system, neural networks, decision making, construction engineering and management

1. Introduction and motivation

Public sector owners in the United States (U.S.) have historically been constrained to design-bid-build project delivery. Owners hire designers through qualifications-based selection to design a project. Upon completion of the design, owners procure contractors based upon the lowest bid. In the early 1990s, public owners began to experiment with design-build project delivery in which they contract with one entity, the design-builder, to complete the design and construct the project. A number of high-profile successes led owners to increase the number of public sector design-build projects. However, many owners experienced failures because they chose projects that were not appropriate for design-build project delivery.

In 1997, a decision support system was developed to provide a formal selection model for public sector design-build projects. The model supports public owners in determining which projects are appropriate for design-build delivery thereby increasing their chance of success (Molenaar and Songer 1998). The resulting web-based decision support system is named the Design-Build Selector (DBS) (Molenaar and Songer 2001). This original research analyzed 104 projects through a retrospective case study approach to derive a predictive model with five performance criteria: overall satisfaction; administrative burden; conformance to expectations; schedule variance; and budget variance. Owners can input characteristics of their projects in a web-based decision support system to benchmark their design-build project candidates against the original 104 case study projects. They can vary their importance of the project goals through a pairwise comparison of the five performance criteria. The output of the model yields a score that is compared to the case study projects through a combined performance criteria score and for each of the performance criteria.

The intent of the present research is to develop a learning engine for the DBS so that the model can adapt as public sector project delivery evolves. Since 1997, the web-base DBS tool has been visited by over 12,000 people and the DBS tool itself has been used on more than 200 projects representing over $5 billion in design and construction. There is now considerable amounts of new project data that can be used improve the original static model.

A review of ninety-three Decision Support Systems (DSS) in the construction field over the past 30 years shows that the majority are static, with fixed parameters, functions, and business rules (Bastias 2006).

Static models can quickly become obsolete, requiring manual adjustments to be relevant in the dynamic environment of the construction engineering and management field. A better approach to solving the problem of changes in the decision environment is to develop dynamic models based on learning systems (Bastias and Molenaar 2005; Bastías 2006; Taylor and Bernstein 2009).

This paper presents a formalized application and use of learning capabilities to supplement the original DBS static model. Through this presentation, it is hoped that this general approach can be adapted to many applications of decision support in the design and construction industry.

2. Background of public sector design-build

A project delivery method is the comprehensive process by which designers, constructors, and various consultants provide services for design and construction to deliver a complete project to the owner. Two primary delivery methods in the U.S. public sector are design-bid-build and design-build. Figure 1 graphically depicts project delivery methods by showing the contract and communication flow inherent in each method.

Figure 1. Project delivery methods

Using the design-bid-build project delivery method, an owner engages a designer to furnish design services and then awards a separate construction contract on the basis of the designer's completed construction documents. The owner is responsible for the details of design and is responsible to the construction contractor for the quality of the construction documents. Figure 1 shows that the contracts place the owner between the designer and the builder in the project delivery process. The linear nature of the process make design-bid-build a lengthy endeavor. The design-bid-build process also allocates responsibility for the accuracy of the design details during construction clearly on the owner. As a result, the owner is accountable to the contractor for the cost of any design errors or omissions. There is little incentive for the builder to minimize change order cost. In fact, there can be quite the opposite effect. When design-bid-build projects are awarded on a low-bid basis, a winning bidder may look to post-award changes as a means to increase profits. Design-bid-build also provides no constructor input to the design because the general contractor is selected based on a low bid after the design is complete. Thus, the owner must rely on the designer alone for cost estimating and constructability review during the design process.

In contrast, design-build is a project delivery method in which the owner retains both design and construction services in the same contract. Figure 1 shows that the contracting approach is direct. Design-build delivery involves the contractor early in design and provides inherent constructability input to the design process. The design-builder is the legal entity that owns the details of design during construction and as a result, is accountable to the owner for the cost of any errors or omissions encountered in construction. As the owner no longer owns the details of design, its relationship with the design-builder must be based on a strong degree of mutual professional trust. The design-builder can literally control the project delivery process. As a result, the design-build delivery method has proven to be highly successful in compressing the project delivery period.

Due to the minimal amount of design at the time of contract award, the design-builder is generally selected through a best-value procurement that combines cost and price. Public sector projects most frequently use a lump sum pricing provision in design-build, but guaranteed maximum pricing is also viable.

Although design-build has been used for centuries, it was not until the late 1960s that the public sector began using design-build in the U.S. Through the 1980s and 1990s various federal and state agencies experimented with design-build for military housing, dormitories, lodges (motels), warehouses, courthouses, mail distribution facilities, vehicle maintenance facilities, laboratories, medical clinics, federal courthouses, and highways (Molenaar et. al., 1999). However, the projects were typically done with special legislation or legal provision because design-build was not widely allowed by public sector procurement laws. It was not until the 1996 Federal Acquisitions Reform Act gave federal authorities legal authority to engage in design-build projects (Molenaar et. al 1999). The rapid growth and novelty of design-build in the public sector created the need for decision support in choosing which public sector projects are appropriate for design-build.

3. Original DBS solution

The original DBS decision support system identified five performance criteria (dependent variables) that are directly related to overall project delivery performance. The performance criteria are obtained evaluating 36 project characteristics (independent variables) through a questionnaire that is oriented to obtain objective information about a specific project. The questionnaire is divided into four sections as shown in Figure 2 (Molenaar and Songer 1998).

Figure 2. Hierarchy of design-build selection questions

A definition of success can take on many criteria. Cost, schedule, and client satisfaction are the main categories used to define success in the DBS. The five performance criteria are (Molenaar and Songer 1998):

1.  Overall Satisfaction: The owners overall satisfaction of the project. The scale ranges from 1 (not satisfied) to 6 (better than expected).

2. Administrative Burden: The administrative burden for a potential design-build project as compared to other projects. The scale ranges from 1 (low) to 6 (high).

3.  Conformance to Expectations: The conformance to expectations for a potential design-build project as compared to other projects. The scale ranges from 1 (did not conform) to 6 (better than expected).

4.  Schedule Variance: The variance in the project's schedule from the time the design-builder was hired to project completion. The scale ranges from -4 (<10% under schedule) to +4 (>10% over schedule).

5.  Budget Variance: This equation predicts the variance in the project's budget from the time the design-builder was hired to project completion. The scale ranges from -4 (<10% under budget) to +4 (>10% over budget).

Each performance criteria is modeled by a multiple linear regression. The performance criteria regression models are based on the original 104 case studies (Molenaar and Songer 1998). To obtain an overall project rating, the five individual regression equations are combined through a linear model in which the weights for each of the five performance criteria are uniquely determined by the user for each project. The final score is predicted as follows.

The results of the overall score points the users to decision support information that can be used to increase the probability of individual project success (Molenaar 1997; Molenaar and Songer 1998). While the user can generate different weightings for the overall model based on their individual project needs, the individual performance criteria equations are static and based only on the 104 projects that were originally modeled to create the DBS decision support tool.

4. Learning solution

The point of departure for this research focuses on the model's learning capabilities. Adaptive models can change over time, adjusting their parameters and functions to increase output accuracy as more information becomes available. In general, there are three kinds of data: input data, time factor data, and output data. Given a process with N input-output data pairs, (see Ecuation 2), the main objective of adaptive modeling is to choose a model where (see Ecuation 3) (Harris, Hong et al., 2002). Techniques for approaching this equation have been developed in the artificial intelligence field.

In the Ecuation ~\,D_N represents a set of pair input/output, where x(t) is the input for time t and y(t) is Output for time t.

In the Ecuation 2, w is an unknown parameter vector for the model structure/fxff), w)

Artificial intelligence is a branch of computer science that has been developed for the purpose of solving learning problems. Although there are many different approaches in artificial intelligence such as evolutionary computation, fuzzy logic, and genetic algorithms (to name a few), the most prolific application of artificial intelligence for learning concepts is the neural network (Jain and Martin 1998). Neural networks learn by definition, through a training and/or adaptation, depending on the learning algorithm used in each case. Previous research has studied the general applicability of neural networks in the construction field (Moselhi, Hegazy et al., 199

and also for more specific solutions to construction problems such as bidding (Moselhi, Hegazy et al., 1993), estimation (Chao and Skibniewski 1995), and project procurement (Kumaraswamy and Dissanayaka 2001). Subsequently, neural networks have been mixed with other technologies, such as fuzzy logic and genetic algorithms, to produce more sophisticated and realistic learning systems (Jain and Martin 1998; Harris, Hong et al., 2002; Cheng and Ko 2003).

Figure 3. Basic learning system diagram

Figure 3 shows a basic diagram of a learning system. There are three main components: input, learning engine, and output. This diagram is the basic approach of the framework being proposed. It serves as reference to create the necessary steps for identifying and developing the structure and procedures of a learning system. The three components are described as follows.

Input: The data input is the most sensitive information provided to the model. Its accuracy will directly impact the output. Any model is sensitive to input variables, so a deep analysis of the type and quality of information is necessary for the creation of a successful model.

Learning Engine: The most complex component of any type of learning system, the learning engine, is designed to perform changes in parameters and functions, using the feedback information. In most cases some type of artificial intelligence is used as the engine (e.g., neural networks, fuzzy logic, genetic algorithms or a combination of techniques).

Output: The output is generally easily measurable and produces valid feedback. The outcome is the information used to make the decision. Outcomes will vary from problem to problem, and because of the nature of learning system, they will be determined by the ability to provide the system with reliable input information.

Artificial neural networks are relatively crude electronic models based on the neural structure of the brain. The network functions like a "brain" and learns from experience. This brain modeling also promises a less technical way to develop machine solutions.

The original mathematical structure for the DBS is relatively close to the structure presented by neural networks. The data set for input/output in the DBE can remain the same with a neural network solution, but the internal structure changes. The algorithms and the functions used by the neural network are defined by the data from the past projects. While the mathematical structure of the original DBS linear model and the neural networks are similar, neural networks have the capability to learn using new information that can be provided through a new input/output dataset. Specific learning algorithms and learning rates defined for each neuron in the network govern this learning.

The main drawback in the use of a neural network is the amount of data required to train the network. To produce sufficient accuracy in results with a low number of training data points, it is important to use an effective and efficient distribution of the training set, which is relatively problem dependent. Additional challenges with neural networks are paralysis and local minima. Paralysis refers to the case when the weights unpredictably come to a standstill status and do not adjust during training. Local minima, refers to the case when the weights settle on a less than optimum status (local optimum v/s global optimum) (Jain and Martin 1998).

In most cases, neural networks are adjusted and/or trained, so that a particular input leads to a specific target output, as shown in Figure 4. Adjustment of the network is based on a comparison of the output and the target, until the network matches the target, or until an allowed error occurs. There are two types of learning: supervised and unsupervised. Supervised learning requires a "teacher." The teacher might be a training set of data or an observer who grades the performance of the network results. Either way, having a teacher is learning by reinforcement. When there is no external teacher, the system must organize itself by some internal criteria designed into the network. Another important internal characteristic of the neural network has to do with its layers. A higher number of layers, in most cases, increases the accuracy of the outcomes but requires more data input to train the network properly. (Hüllermeier, Renners et al., 2004; Hinton, Osindero et al., 2006)

Figure 4 Artificial neural network diagram

Some notable advantages and disadvantages are:

•  Advantages: Neural networks learn system behavior by using input-output data. The representation of the human thinking process is good enough for solving many problems that are either unsolved or inefficiently solved by existing techniques. The networks incorporate the previous information to meet and advance further solutions.

•   Disadvantages: The major problem is the nature of the "black box"; the general understanding of what there is in the black box is incomplete. It is much harder to determine the appropriate structure and layer of the network for solving the problem. Manipulating and learning parameters for learning and convergence become increasingly difficult. Another important disadvantage is in respect to the number of data set needed to train the neural networks. Depending number of component and layer, the neural network might require a high volume of information, and usually in the construction industry this is a factor to be controlled.

The data set input/output remains the same between the original and the new DBS solution. A set of 25 input are related to five outcomes. So, the neural networks have 25 input nodes associated with five output nodes. A categorical type of node reflects an exclusive variable, for example, type of contract. As previously mentioned, the initial setup was developed using 104 projects and validated with 18 projects in 1997. Since then, many new projects have been completed giving the model new information to improve the accuracy of the results.

The most complex and tedious task in the construction of a neural network is the selection of the number of hidden layers or nodes and learning parameters, such as learning rule, and learning rate. There is a tradeoff implicit in the neural network construction. If the number of hidden nodes increases the network get more stability in the weight and bias. However, if the number of nodes is too small, the network loses its ability to learn. In order to design "the least complicated network with acceptable performance," a trial-and-error procedure is necessary (Cimikowski and Shope 1996).

After a comprehensive analysis and design of the network, a trial-and-error procedure was carried out by adding nodes to the hidden layer and checking for improved network performance. During these experiments, changes were monitored in the performance indicator (final score), the mean square error (MSE), the mean error (MEAN), and standard deviation (STD) error. The proposed solution includes a set of five neural networks associated to each performance criteria. All networks have the same structure, with an input layer composed by three neurons, and two hidden layers, where the inner layer has two neurons and the output layer only one.

Figure 5 shows the network design for the Overall Satisfaction performance criteria. Matlab™ with Neural Network toolbox® was used to develop the network. The algorithm used by the network is back propagation with purelin as the transfer function, translm as the transfer training function, and trains as the adapting function. The performance function minimized the MSE. The input layer has a vector of eight data points with three neurons.

Figure 5. Network for the indicator "Overall Satisfaction"

The neural networks must be trained in order to adjust weight and bias. The training process requires, in most cases, a considerable amount of data, however, this amount can be reduced if the data set is representative of the problem to be solved. The adaptation is that the previously-trained network is adjusted using new information through feedback. The simulation is just the evaluation of the network using certain input.

The data set for the new model considers 152 projects in the DBS solution. In order to maintain a parallel between the original solution and this new approach, the network was trained with the same initial set of data (i.e. the 104 original projects). It was then adapted with 1 8 projects and evaluated with 29 projects. Table 1 shows the results in the evaluation for three cases: the static solution; the initial neural network; and the neural network adapted with additional data sets.

Table 1. Analysis of error

Figure 6 shows the five neural networks in the DBS solution and represents the general topology. The output of each neural network is transformed to a scale of 1 -100, then to a linear model. The final score is calculated as an average of each performance criterion on an adjusted scale. This calculation is done to make a direct comparison as the weights are different for each project. As previously discussed, a pairwise comparison was used dynamically in the DBS to generate the weights for each unique project.

Figure 6. Final topology of neural networks

As shown in Table 1, the learning model shows a distinct improvement from the original regression model. As an example, a large public university spent $12,000,000 on an extension of its educational outreach building. The information from this project was used to analyze the performance of the current and new approach. The neural network presents a more accurate prediction of project success and therefore provides more accurate advisory information. The error in the final estimation decreased from 7.1 % to 0.5% for this particular case. The output from the DBS provides an overall score and based on predictions for each of the five performance measures. The output for this project showed that it was a good candidate for design-build. Some of the project characteristics pointed to improvements that the owner could make in the area of schedule performance.

For more information on this advisory information, see Molenaar's research (Molenaar and Songer 2001). The important aspect of the new DBS learning solution is not only the additional accuracy, but more importantly how the new information was used as feedback to continuously improve the output. The characteristics of the owner, project, market and relationship will be changing as the public sector evolves and becomes more familiar with design-build delivery. If the neural network model can be provided with adequate data, this model will provide more accurate predictions and provide more applicable advisory information.

5. Conclusion

This research advances the area of decision support systems in construction engineering and management by applying a learning model to a problem that was originally solved with a static model. Although the construction engineering and management discipline is very dynamic in nature, the large majority of the decision support systems are static. The DBS serves a representative case study because it is typical of decision support systems in this area.

While this paper provides only one case study of a learning model applied to construction engineering and management, the authors see many opportunities for further application. The DBS case study is an example of a strategic decision involving the selection of a project delivery method. Similar dynamic problems exist at this strategic level as seen in the literature. This methodology could be applied to the selection of international projects by a contracting company (Han and Diekmann 2001) or the decision to prequalify contractors by an owner (Russell et. al 1990). At the strategic level, there are certainly barriers such as the length of time needed to collect outcomes as feedback, the amount of data required for reliable output, the number of confounding variables in the decisions, and the acceptance of the decision support information by the decision makers. However, as shown in the DBS case, these barriers can be overcome with long-range planning and consistent data collection.

The application of this learning methodology could also be applied at the production level.

Similar studies that could benefit from this methodology at the production level include resource sharing in linear construction (Perera 1983), multiple heavy lifts optimization (Lin and Haas 1996), time-cost-quality tradeoff analysis for highway construction (El-Rayes and Kandil 2005). The length of time needed to collect outcomes as feedback and the number of confounding factors is smaller at the production level. However, other barriers for implementation could include the short time frame to implement the results and the difficulty in changing production systems during operations. Nevertheless, these barriers are minimal when compared to the potential gains that could be found by applying learning mechanisms to decision support systems in the industry.

Ultimately, the construction engineering and management field will benefit from the incorporation of learning capabilities in decision support system models. The use of artificial intelligence to apply learning capabilities in its decision models is one appropriate way; however, other methods can be used. The key is not to let modes remain static and to use new information to increase the accuracy of future results. The DBS case demonstrates that, with proper planning, dynamic decision support systems can improve performance without creating overly burdensome data collection and analysis efforts.

6. Acknowledgments

The authors would like to thank all of the professionals who contributed their time and case study data. Without their support, this research would not have been possible.

7. References

Bastias A. (2006), Towards the application of learning systems for decision support in construction engineering and management. Civil Engineering. Boulder, University of Colorado.

Bastias A. and K. Molenaar (2005), Classification and Analysis of Decision Support Systems for the Construction Industry. International Conference on Computing in Civil Engineering, Cancun, American Society of Civil Engineers.

Cimikowski R. and P. Shope (1996), "A neural-network algorithm for a graph layout problem." IEEE Transactions on Neural Networks 7(2): 341-345.

Chao L.-C. and M. J. Skibniewski (1995), "Neural Network Method of Estimating Construction Technology Acceptability." Journal of Construction Engineering and Management 121(1): 130-142.

Cheng M.-Y. and C.-H. Ko (2003), "Obj'ect-Oriented Evolutionary Fuzzy Neural Inference System for Construction Management." Journal of Construction Engineering and Management 129(4): 461-469.

Harris C, X. Hong et al. (2002), Adaptive modelling, estimation, and fusion from data: a neurofuzzy approach, Springer Verlag.

Hinton C, S. Osindero, et al. (2006), "A fast learning algorithm for deep belief nets." Neural Computation 18(7): 1527-1554.

Hüllermeier E., I. Renners et al. (2004), "An evolutionary approach to constraint-regularized learning." Mathware & soft computing 11(2-3): 109-124.

Jain L. C. and N. M. Martin (1998), Fusion of Neural Networks, Fuzzy Sets, and Genetic Algorithms: Industrial Applications, CRC Press, Inc.

Kumaraswamy M. and S. Dissanayaka (2001), "Developing a decision support system for building proj'ect procurement." Building and Environment 36(3): 337-349.

Molenaar K. (1997), Public sector design-build: a model for proj'ect selection. Civil Engineering. Boulder, University of Colorado.

Molenaar K. and A. Songer (1998), "Model for public sector design-build proj'ect selection." Journal of Construction Engineering and Management 124(6): 467-479.

Molenaar K. and A. D. Songer (2001), "Web-based decision support systems: Case study in proj'ect delivery." Journal of Computing in Civil Engineering 15(4): 259-267.

Moselhi O. and T. Hegazy et al. (1991), "Neural networks as tools in construction." Journal of Construction Engineering and Management 117(4): 606-625.

Moselhi O., Hegazy T. et al. (1993), "DBID: analogy-based DSS for bidding in construction." Journal of Construction Engineering and Management 119(3): 466-479.

Taylor J. and P. Bernstein (2009), "Paradigm Traj'ectories of Building Information Modeling Practice in Proj'ect Networks." Journal of Management in Engineering 25: 69-76.

Corresponding author: E-mail: abastias@udd.cl