pAeISSN: 2723 Ae 6609 e-ISSN: 2745-5254 Vol. No. February 2025 http://jist. Analysis of Change Order Decision-Making Using The Analytic Network Process in The Operator Training Simulator Project at The Refinery Development Master Plan in The Oil And Gas Industry Mahrtahmed Wira Panjijaya1*. Udisubakti Ciptomulyono2 Institut Teknologi Sepuluh Nopember. Indonesia Email: mahrtahmed. panjijaya@pertamina. *Correspondence Keywords: Change Order. Operator Training Simulator. Refinery Development Master Plan. Delphi. DEMATEL. Analytic Network Process (ANP). Sistem Pendukung Keputusan. ABSTRACT The Refinery Development Master Plan (RDMP) project at PT X aims to increase the capacity and efficiency of the refinery. A key aspect of this Residue Fluid Catalytic Cracking (RFCC) unit project is the development of an Operator Training Simulator (OTS) to ensure practical operator training. However, the change order decision-making process associated with developing the OTS is often complex and influenced by various factors. This research uses a Decision Support System by combining a Delphi - DEMATEL-based approach with Analytic Network Process (ANP) techniques to analyze the change order decision-making The Delphi method gathers expert opinions and identifies key factors that influence the decision-making process, and the DEMATEL method is used to see the interrelationships between Meanwhile, the ANP method structures and prioritizes these factors. The results of this study provide a structured framework for decision-making in the OTS development change order process that managers and stakeholders can use to ensure effective and efficient decision-making. Introduction PT runs the Refinery Development Master Plan (RDMP) project. X is one of the strategic initiatives that aims to increase the capacity and efficiency of oil refineries to meet the increasing national energy demand. (Harsyandi, 2. One key unit that is an essential component in this project is a Residue Fluid Catalytic Cracking (RFCC) unit. This unit increases the added value of oil products by converting residues from crude processing or crude oil in the primary process into high-value products such as fuel, fuel, propylene, and other by-products. (FakhrounNisa, 2015. Wiryatama et al. , 2. In large-scale projects such as RDMP, implementation and change management are complex challenges (Yan et al. , 2. To ensure the smooth operation of the RFCC unit. PT. X needs to equip operators with adequate training. One of the essential aspects of the Refinery Development Master Plan (RDMP) project is the Operator Training Simulator (OTS) for RFCC units, which aims to train and ensure that operators can operate RFCC units effectively and safely (OAoSullivan, 2. OTS plays a critical role in simulating normal operating conditions, emergency response training, and improving operator skills. Indonesian Journal of Social Technology. Vol. No. February 2025 Analysis of Change Order Decision-Making Using The Analytic Network Process in The Operator Training Simulator Project at The Refinery Development Master Plan in The Oil and Gas Industry As a project develops, changes or modifications are often known as change orders. Change orders on OTS RFCC units can be triggered by various factors, such as changes in design, technical specifications, or newly identified operational needs. (Chen et al. Effective decision-making is critical because of the complexity and potential impact of these change orders. (Kermanshachi et al. , 2. A Change Order is a formal document used in construction project management to record and approve changes during a project's execution. (Ervianto, 2023. Ir Sulistijo Sidarto Mulyo & Santoso, 2018. Ismaeil & Sobaih, 2. These changes can include design, material, schedule, or other changes that affect the project's initial scope. (Ruskandi et al. , 2. A Change Order is approved if it has received approval from the project owner . and the contractor after discussing the impact of the proposed (Clough et al. , 2. Change orders can impact the project owner's finances, schedule, and quality of the project's final results. (Rahman et al. , 2. Decision-making related to change orders in an operator training simulator project requires a proper approach to identify and assess the various factors that affect the projectAos success. (Ravikanth, 2. Due to the complexity and interdependence between these factors, the methods used to support decision-making must handle this complexity well, such as ANP (Analytic Network Proces. , because ANP can handle complex relationships and interdependencies between various elements and criteria that affect decision-making. In a simulator operator training project, many factors are interrelated and influence each other, so ANP is a perfect fit to address this. ANP also has a flexible network structure compared to traditional hierarchical methods such as AHP (Analytic Hierarchy Proces. This flexibility allows researchers to incorporate various elements and relationships in the context of a simulator operator training project. Several studies have shown the effectiveness of ANP in project management, particularly in the context of decision-making involving many interrelated criteria and factors. For example. I introduced and developed ANP as a tool to deal with complex and dynamic decision (Ibrahim et al. , 2020. Saaty, 1999, 2. Where after the Factory Acceptance Tests (FAT) were carried out on the Operator Training Simulator (OTS) project, it turned out that several punches/corrections were found in the form of additions and changes to items, but not the vendor could follow up on all of them because they were not covered in the initial contract agreement. From the FAT results, six items require additional costs, including: New requirement by Owner during FAT (Including Model Update and Re-tunin. Additional Instructor / FOD Graphics . Page. New requirement by Owner for Modeling (Including Model Update. Re-tuning and Resource Mobilizatio. Additional Database Connection to OTS for Energy Recovery Package (ERP) OTS model adaptation to Energy Recovery Package (ERP) Scenario Addition for Energy Recovery Package (ERP) Indonesian Journal of Social Technology. Vol. No. February 2025 Mahrtahmed Wira Panjijaya. Udisubakti Ciptomulyono Due to the addition of work, an additional cost from the contractor/vendor of 1,224,675 USD is required. Therefore, according to the procedure of the TKO company, guideline No. B-001/X1000/2020-S9 Implementation of Change Order Management New Unit Scope of Work EPC ISBL-OSBL in point 10 if the cumulative cost value of the Change Order Proposal is more than an amount equivalent to three percent . %) of the Contract Price, then a review of the Change Order Proposal is required from the Holding regarding the Change Order Proposal to give approval or rejection of the Change Order Proposal which has been analyzed techno-commercially-legally by PT X through an official letter from the Director of Development within 14 Calendar Days. One of the methods that can be used to improve the quality of decision-making is the Analytic Network Process (ANP) (Chen et al. , 2. ANP is a multi-criteria decisionmaking technique that considers interdependencies and feedback between decision elements, thus providing a more holistic and comprehensive approach compared to traditional methods (Zhang et al. , 2. According to Panjijaya, . The results show that accepting part of the change order is the best alternative with the highest weight . , followed by receiving the change order . , rejecting the change order . , and creating a new contract . This model is expected to assist companies in making strategic decisions on similar projects in the future. This study analyzes change order decision-making in OTS RFCC units using ANP. It hopes that implementing ANP will identify key factors affecting change orders and provide more accurate and targeted recommendations for change order management. This is expected to improve project efficiency, reduce operational risks, and ensure that OTS can optimally meet operator training needs. In addition, this research will contribute to the development of decision-making methods in the energy industry environment, especially in the context of oil refinery development projects. Thus, this research is beneficial not only for PT. X but also for the energy industry as a whole in managing complex and dynamic project changes. In this study, several ANP approaches will be proposed to assess related criteria, such as cost with schedule, cost with quality, etc. Furthermore, from the requirements obtained, identification is carried out by weighting using the Delphi method, after which, from the criteria obtained, it is continued using the ANP method to select alternatives that have been determined whether the change order will continue or not to continue. Moreover, if you continue with the change order, it will impact the increase in costs and longer work time. In contrast, if you do not decide to change the order, it will affect the quality of the Operator Training Simulator, which will make it difficult for operators to conduct training for smooth start-up and factory operations. Method Indonesian Journal of Social Technology. Vol. No. February 2025 Analysis of Change Order Decision-Making Using The Analytic Network Process in The Operator Training Simulator Project at The Refinery Development Master Plan in The Oil and Gas Industry This research is a problem-solving tool for deciding whether to change the order. The method used is the MCDM (Multi-Criteria Decision Makin. The population in this study is all research objects and subjects with specific characteristics according to those determined by the researcher. From this population, conclusions will be drawn, and samples will be taken. In this study, the sampling method used is nonprobability sampling with a purposive sampling technique. The purposive sampling technique is a technique for determining samples with specific considerations (Wesli. The research respondents were selected because they were directly involved or considered to have the ability and understanding of the production of RFCC units. The respondents of this study are experts at PT. In the ANP method, the validity reference is not assessed from the adequacy of the number of respondents but from the ability of the respondents to master the problem being researched. Based on these considerations, the following respondent data are as follows: Table 1. List of Tanaga Experts Department of Human Main Duties and Functions Resources Manager Production Ensure the proposed change/ change order does not interfere with the production flow and product Manager Engineering Evaluate the technical aspects of the change order, including its impact on the design and technical Responsible for the design, testing, and implementation of engineering systems. Section Head Coordinate departmental activities related to change RFCC Ensure that the proposed changes are implemented effectively in their departments. Senior Engineering Provides an in-depth analysis of the technical implications of change orders. Suggest alternative solutions or adjustments to minimize risk. Senior Supervisor Assess the impact of changes on daily operations and supervise teams. Implement changes and monitor their effectiveness by providing training and support to the team. Source: Self-Prepared The determination of criteria is based on company guidelines and literature obtained from the results of processing questionnaires from experts that have been processed using the Delphi method and then used the DEMATEL method to see the relationship between the criteria. Then, criteria weighting and alternative assessments will be collected using a Indonesian Journal of Social Technology. Vol. No. February 2025 Mahrtahmed Wira Panjijaya. Udisubakti Ciptomulyono questionnaire, which will be processed using the ANP method with the help of "Super Decisions" software. Results and Discussion This chapter discusses data collection and data processing in implementing decision-making in change orders/change requests in the Operator Training Simulator project at the Oil and Gas Industry Refinery Development Master Plant. The data collection consists of PT's internal data. X regarding the guidelines for implementing change orders and taking some literature was re-identified using the Delphi method according to experts' opinions and analyzed using the DEMATEL and ANP methods. Overview of Research Objects The object of research in this study is a company established to carry out the development of the Balikpapan & Lawe-lawe Refinery Development Master Plan (RDMP) project and run the Refinery Unit (RU) V Balikpapan refinery processing business. East Kalimantan which is hereinafter referred to as PT. X, which was established on May 7, 2019. Currently, the ownership shares of PT. Subsidiaries of stateowned companies own X 99. 997% engaged in the energy sector, especially the oil and gas industry. Indonesia's most significant dividend contributor. Currently. PT. X is developing the Operator Training Simulator RFCC unit where. At the monitoring and controlling stage of the project, it was found that there was a discrepancy in the project implementation stage, so PT. X, as the owner, asked the contractor to make work adjustments. So that the contractor submits a change order for additional work costs that require prior approval by the owner. Making decisions related to accepting or rejecting this change order is one of the challenges for decision-makers at PT. The reason is that this decision not only impacts the cost and schedule of the project but also the quality of the final work results. This study aims to delve deeper into the factors that affect decision-making related to change orders from the perspective of experts (Engineering Manager. Marketing and Start-Up Manager. Section Head. Senior Engineer, and Senior Superviso. By understanding the perspectives of experts, it is hoped that a more comprehensive understanding of the decision-making process related to change orders can be obtained and critical factors that need to be considered can be Data Respondent The data of the respondents involved in the study will be explained in the age, working period, position, and job descriptions of each respondent as follows: Manager Engineering Age: 39 years. Working period: 15 years Job Description: Responsible for ensuring that the technical aspects of the simulator run smoothly and by the standards set, such as technical planning, system development, quality management, and coordinating with other functions. Manager Commissioning and Start-Up Indonesian Journal of Social Technology. Vol. No. February 2025 Analysis of Change Order Decision-Making Using The Analytic Network Process in The Operator Training Simulator Project at The Refinery Development Master Plan in The Oil and Gas Industry Age: 52 years. Working period: 31 years Job Description: Responsible for coordinating the entire commissioning process and ensuring that the simulator functions correctly according to the specifications that have been set, as well as coordinating and supervising the whole process of testing, configuring, and preparing the simulator to operate optimally. Section Head RFCC Age: 38 years. Working period: 15 years Job Description: Responsible for ensuring that the RFCC (Residuum Fluid Catalytic Crackin. part of the simulator operates accurately and realistically to provide an optimal training experience for the operator. Senior Engineer Age: 31 years. Working period: 8 years Job Descriptions: Responsible for providing in-depth technical support, ensuring optimal simulator performance, and ensuring simulators accurately represent the actual operation of RFCC units. Must have in-depth knowledge of RFCC processes, expertise in modeling and programming, and the ability to solve complex problems. Senior Supervisor RFCC Age: 40 years. Working period: 16 years Job Description: Ensuring that all instruments and personnel work harmoniously to achieve project objectives. Data Analysis Results analysis is a research analysis process from the results of surveys conducted by researchers on research objects. In this study, the analysis process is carried out in 3 stages, namely by looking for criteria and sub-criteria that are influential in making change order decisions using the Delphi Method. The second analysis stage uses the DEMATEL method to find relationships between criteria and sub-criteria obtained at the Delphi stage. ANP processes data from the Delphi. DEMATEL, and pairwise comparison processes to receive the best choice rating. Determination of Criteria and Subcriteria The Delphi method is used to identify criteria and subcriteria in decision-making. The determination of criteria and subcriteria starts with distributing questionnaires in three rounds. In the first round, questionnaires were distributed explaining the researcher's intentions and objectives and containing all answers to all criteria and sub-criteria written by experts even though they were not on the list. In the second round, the results of the answers in the first round are on the list and are used as questions in the second round. the second round, indicators or subcriteria are grouped into criteria that have a similar scope of meaning. The selected specialist requests the determination of these criteria. After the results of the second questionnaire from the experts were collected, a questionnaire was prepared for the third round. The questionnaire questions were Indonesian Journal of Social Technology. Vol. No. February 2025 Mahrtahmed Wira Panjijaya. Udisubakti Ciptomulyono distributed to experts in the third round, and the results were reviewed. Then, the collected answers were re-analyzed. if there were results in these three rounds with a value of less 5, then it was eliminated from the final answer results. Determination of Relationships between Criteria and Subcriteria Chloroperia and subcriteria were determined in the previous stage and used to analyze the relationship between criteria and subcriteria using the Dematel method. Moreover, defining relationships between criteria and subcriteria is critical in understanding the complex dynamics within decision-making processes, particularly in multifaceted projects like the Operator Training Simulator (OTS) under the Refinery Development Master Plan (RDMP). This study uses the DEMATEL (Decision Making Trial and Evaluation Laborator. method to systematically analyze and visualize the causal relationships between the identified criteria and subcriteria. Determination of Weight Values Between Subcriteria with ANP After Dematel data is processed, a relationship between factors will be generated to determine the change order policy. This relationship is based on information from the subcriteria linkage matrix between these factors, which is also shown on the NRM chart. These relationships are then used to compile the ANP questionnaire using Super Decision The first step in preparing the ANP questionnaire is to form a relationship between factors outlined in the Super Decision software. An overview of the relationship is shown in Figure 1. Figure 1. ANP Model on Change Order Policy Based on the model shown in Figure 1, we can see that three levels are included in several clusters according to the relationship figure. In each criterion included in the cluster, subcriteria are outlined in the nodes. The first level is the cluster goal or the purpose of this analysis, namely the Change Order Policy. The goal is placed in the red goal cluster, and then a light blue change order node is created. At the second level is a Indonesian Journal of Social Technology. Vol. No. February 2025 Analysis of Change Order Decision-Making Using The Analytic Network Process in The Operator Training Simulator Project at The Refinery Development Master Plan in The Oil and Gas Industry criterion cluster with subcriteria in the form of nodes. In this study, there are four criteria and 10 subcriteria. The cost criteria are placed in the Cost Criteria cluster. The subcriteria are compiled in several green nodes, namely the increase in project costs (B. , the increase in overhead costs (B. , and the addition of the expenses for contractors (B. Quality criteria are placed into the Quality Criteria cluster in which there are subcriteria that are input in the form of nodes, namely simulator quality degradation (K. , rework (K. , and loss of efficiency due to work interruption (K. The time criteria are placed into the Time criteria cluster in which two subcriteria are input as nodes, namely the completion of the pending project (W. and the change or extension of the work time (W. The risk criteria are placed into the risk criteria cluster in which two subcriteria are input as nodes, namely effects related to organizations or disputes (R. and conflicts between parties (R. In the third level, there is an alternative cluster that contains four change order options, outlined in purple nodes: accepting change orders (A. , rejecting change orders (A. , accepting partial change orders (A. , and creating new contracts (A. The questionnaire can be generated by running the program, as shown in Figure 2. The questionnaire form can then be used to collect data. Figure 2. ANP Questionnaire from Super Decision Software Data was processed after the respondents' ANP questionnaire data was obtained. Data processing using the ANP method is carried out using Super Decision software. Data processing is carried out as follows: Recapitulation of ANP paired comparison data The paired comparison values on each criterion and sub-criteria were obtained from the respondentsAo assessment on a scale of 1-9 on the ANP questionnaire. The recapitulation results were prepared by calculating the geometric average score of the respondent's assessment. The recapitulation results are shown in the Appendix. The average result value is then input into the super decision for calculation. Indonesian Journal of Social Technology. Vol. No. February 2025 Mahrtahmed Wira Panjijaya. Udisubakti Ciptomulyono Recapitulation of ANP paired comparison data A paired comparison matrix between the subcriteria was created using the ANP The results of the respondents' assessment were recapitulated by calculating the geometric average. The value of this calculation result is then entered into the matrix cell form in the super decision, as shown in Figure 3. The form of the paired comparison matrix of the calculation results with the super decision related to the project cost is shown in Table 3 The consistency value of the respondents in assessing the paired comparison by looking at the Consistency Index (CI) size. The assessment is consistent if the CI value < 1, and if the indigo CI > 0. 1, it can be said that the respondents' assessment is Based on Figure 3, the CI value is 0. 07013, so it can be noted that the respondent's assessment is consistent. Figure 3. Data Entry on the Super Decision Form Table 2. Paired Comparison Matrix Related to Project Costs on Cost Criteria B1 - Penambahan B2 - Kenaikan B3 - Penambahan biaya biaya proyek biaya overhead untuk kontraktor B1 - Penambahan 1,000E 00 4,680E 00 3,610E-01 biaya proyek B2 - Kenaikan biaya 1,000E 00 1,730E-01 B3 - Penambahan 1,000E 00 biaya untuk kontraktor Source: Data processed Compose an Unweighted Supermatrix The unweighted supermatrix is the third step notated with the W matrix. Then, a transpose matrix is created. In this study, the calculation of the unweighted supermatrix was carried out using super decision software. The results of the unweighted supermatrix are shown in Figure 4. Indonesian Journal of Social Technology. Vol. No. February 2025 Analysis of Change Order Decision-Making Using The Analytic Network Process in The Operator Training Simulator Project at The Refinery Development Master Plan in The Oil and Gas Industry Figure 4. Unweighted Supermatrix Defining the Weighted Supermatrix Weighted Supermatrix is obtained from the value of the group matrix used as the weight of the unweighted supermatrix. The weighting of the weighted supermatrix is done by multiplying the value of the group matrix cells by the value of each unweighted supermatrix cell. The weighted supermatrix results can be seen in Figure 5. Building a Supermatrix Limit The Supermatrix limit is obtained by multiplying the weighted supermatrix by its When the priority value in each column is the same, the result of the supermatrix limit is obtained. (Results supermatrix limit can be seen in Figure . Indonesian Journal of Social Technology. Vol. No. February 2025 Mahrtahmed Wira Panjijaya. Udisubakti Ciptomulyono Figure 5. Weighted Supermatrix Figure 6. Limit Supermatrix Determining final priorities The final priority value is obtained from the supermatrix limit and then normalized based on the group so that the total priority value known for each group is one. The final Priority value can be seen in Figure 7. Indonesian Journal of Social Technology. Vol. No. February 2025 Analysis of Change Order Decision-Making Using The Analytic Network Process in The Operator Training Simulator Project at The Refinery Development Master Plan in The Oil and Gas Industry Figure 7. Priorities Output The ANP method prioritizes the subcriteria's weight by obtaining the subcriteria's weight value of project management success. The ANP method does not describe hierarchically but in a network. The relationship between the subcriteria was obtained from the analysis results using the Dematel method. The results obtained from the ANP are in the order of priority of weights in each subcriterion. The results of the weighting are shown in Figure 8. Indonesian Journal of Social Technology. Vol. No. February 2025 Mahrtahmed Wira Panjijaya. Udisubakti Ciptomulyono Figure 8. Weighting Level of Change Order Policy Factor The results of data processing using DAMP showed that the priority sub-criteria in the change order policy were the completion of delayed projects (W. , which had the highest weight value of 0. 306471, and the overhead cost increase sub-criterion (B. , which had the lowest weight value of 0. Priority recommendations in the Change Order Policy Proposed recommendations contain proposals for strategic steps that can be taken related to alternative factors that have the highest ranking to improve the project management aspect. Alternatives have been determined at the beginning of the study, including accepting the change order, rejecting the change order, accepting part of the change order, and making a new contract. By conducting an ANP analysis, the alternative factor with the highest weight compared to other alternative factors accepts a part of the change order (A. The results of data processing using the ANP method using Super Decision are shown in Table 2. Indonesian Journal of Social Technology. Vol. No. February 2025 Analysis of Change Order Decision-Making Using The Analytic Network Process in The Operator Training Simulator Project at The Refinery Development Master Plan in The Oil and Gas Industry Figure 9. Change Order Policy Priority Rating The ranking results using the ANP method above show that the third alternative, namely accepting a part of the change order (A. , is a recommendation on the alternative change order policy. This alternative has a global weight of 0. The second alternative priority is to accept a change order (A. with a worldwide weight of 0. The third and fourth alternatives are to reject the change order and create a new contract, with each alternative weighing 0. 0392 and 0. Sensitivity Analysis In this study, the sensitivity analysis simulation was carried out by changing the weight of the subcriteria covered in the criteria with the highest weight. (Moradi et al. Changes are made with deviations of 25%, 50%, -25%, and -50% on these The criteria with the most significant weight are the Time and Quality Criteria. Regarding time criteria, there are two subcriteria for completing the delayed project (W. and changing/extending the work time (W. Meanwhile, the quality criteria have three subcriteria, namely the deterioration of the quality of the simulator (K. , rework (K. , and loss of efficiency due to work interruption (K. High-weight criteria tend to have a more significant influence on the decision outcome. Change Order Policy Strategy Recommendations The explanation of the highest priority is valid . ensitivity tests have been carried ou. , and then it is elaborated on with technical conditions. Accept Partial Change Orders Technical Explanation There are 2 points of technical problems underlying the change order, as follows: Indonesian Journal of Social Technology. Vol. No. February 2025 Mahrtahmed Wira Panjijaya. Udisubakti Ciptomulyono Table 3. Technical Explanation Commodities & Specifications Weekdays Additional Requirement by Owner during FAT (Factory Acceptance Tes. 1 New requirement by Owner during FAT (Including 168 hari Model Update and Re-tunin. 2 Additional Instructor / FOD Graphics . Page. 122 hari Requested by Owner 3 New requirement by Owner for Modeling (Including Model Update. Re-tuning, and Resource Mobilizatio. 50 hari Price (US$) Additional Work for Energy Recovery Package (ERP) Late Deliverable additional database Connection to OTS for Energy 50 hari Recovery Package (ERP) OTS model adaptation to Energy Recovery Package 19 hari (ERP) 31 hari Scenario Addition Energy Recovery Package(ERP) Source: Data processed Conditions and Facts on the Ground After coordination with the engineering function, it was agreed that the first point, "Additional Requirement by Owner during FAT (Factory Acceptance Tes. ," can be done by the internal engineering function, so there is no need for a change order for this point. As for the second point, "Additional Work for Energy Recovery Package (ERP)," because internal engineering does not have experience processing the work, which is, incidentally, still new, the engineering function does not undertake this second point. Conclusion This stage is the final step of the research. The study's conclusion results from research on data processing related to change order decision-making. Analyzing the change order decision-making process in the Operator Training Simulator (OTS) project in the Refinery Development Master Plan (RDMP) highlights the importance of using a structured method in managing complex projects. Delphi. DEMATEL, and Analytic Network Process (ANP) methods allow for a comprehensive evaluation of critical factors influencing the decision-making process. The results show that the most significant subcriterion in the change order policy is the delayed project (W. completion with the highest weight of 0. In contrast, the subcriterion with the lowest weight is the Indonesian Journal of Social Technology. Vol. No. February 2025 Analysis of Change Order Decision-Making Using The Analytic Network Process in The Operator Training Simulator Project at The Refinery Development Master Plan in The Oil and Gas Industry increase in overhead costs (B. This shows that project completion time has a more significant influence on decision-making than cost-related considerations. Based on ANP analysis, the priority recommendation is to accept a partial change order (A. with a global weight of 0. This alternative is considered the most balanced in considering cost, time, quality, and risk factors and offers a pragmatic approach to maintaining project integrity while still accommodating essential changes. The sensitivity analysis validated the strength of the ANP model, showing that the time and quality criteria greatly influenced the decision results. The change in weighting on these criteria indicates a significant shift in the priority ranking, which confirms the need for more attention to project completion times and quality standards during change order REFERENCE