Determinations

---

---

TABLE OF CONTENTS

• DETERMINATION OF THE TRIBUNAL
• STATEMENT OF REASONS
• • PROCUREMENT PROCESS
  • • Background
    • Re-evaluation
  • POSITION OF PARTIES
  • • Department's Position
    • MIL/Fleetway's Position
    • SWI's Position
  • TRIBUNAL'S DECISION
  • DETERMINATION OF THE TRIBUNAL

---

Ottawa, Monday, March 19, 2001

File No.: PR-2000-039

IN THE MATTER OF a complaint filed by Siemens Westinghouse Incorporated under subsection 30.11(1) of the Canadian International Trade Tribunal Act, R.S.C. 1985 (4th Supp.), c. 47;

AND IN THE MATTER OF a decision to conduct an inquiry into the complaint under subsection 30.13(1) of the Canadian International Trade Tribunal Act.

DETERMINATION OF THE TRIBUNAL

Pursuant to subsection 30.14(2) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal determines that the complaint is valid in part. However, the Canadian International Trade Tribunal finds that the breaches committed by the Department of Public Works and Government Services and the Department of National Defence in re-evaluating the technical merits of proposals did not have a material impact on the scoring of Siemens Westinghouse Incorporated's proposal with respect to sections 6 and 7 of Annex "H" of the Request for Proposal. Therefore, there is no basis upon which to disturb or reject the scoring of the said sections or to overturn the overall result of the re-evaluation of the technical merits of Siemens Westinghouse Incorporated's proposal.

Pursuant to subsection 30.16(1) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal awards Siemens Westinghouse Incorporated its reasonable costs incurred in filing and proceeding with this complaint.

Michel P. Granger
Michel P. Granger
Secretary

The statement of reasons will be issued at a later date.
 
 

Date of Determination:	March 19, 2001
Date of Reasons:	April 17, 2001
Tribunal Members:	Patricia M. Close, Presiding Member
	Pierre Gosselin, Member
	James A. Ogilvy, Member
Investigation Manager:	Randolph W. Heggart
Investigation Officer:	Paule Couët
Counsel for the Tribunal:	John Dodsworth
Complainant:	Siemens Westinghouse Incorporated
Counsel for the Complainant:	Ronald D. Lunau
	Carina de Pellegrin
	Mary Rose Ebos
Interveners:	MIL Systems, a Division of Davie Industries Inc.
	Fleetway Inc.
Counsel for the Interveners:	David Sheriffs Scott
	J. Bruce Carr-Harris
	Vincent DeRose
Government Institution:	Department of Public Works and Government Services
Counsel for the Government Institution:	Christianne M. Laizner
	Susan D. Clarke
	Ian G. Mcleod

 
 

Ottawa, Tuesday, April 17, 2001

File No.: PR-2000-039

IN THE MATTER OF a complaint filed by Siemens Westinghouse Incorporated under subsection 30.11(1) of the Canadian International Trade Tribunal Act, R.S.C. 1985 (4th Supp.), c. 47;

AND IN THE MATTER OF a decision to conduct an inquiry into the complaint under subsection 30.13(1) of the Canadian International Trade Tribunal Act.

STATEMENT OF REASONS

On November 3, 2000, Siemens Westinghouse Incorporated (SWI) filed a complaint with the Canadian International Trade Tribunal (the Tribunal) under subsection 30.11(1) of the Canadian International Trade Tribunal Act ¹ concerning a procurement (Solicitation No. W8483-6-EFAA) by the Department of Public Works and Government Services (the Department) for the provision of in-service support (Class Design Agency and Class Technical Data Agency services) for the Halifax and Iroquois class ships of the Department of National Defence (DND).

SWI alleged that, during the re-evaluation of the technical merits of the proposals, contrary to Article 506(6) of the Agreement on Internal Trade,² the Department developed and applied a methodology and weighting that were not set out in the Request for Proposal (RFP). SWI also submitted that these actions contravene the Tribunal's determination in File No. PR-99-034³ and the findings of the Federal Court of Appeal (the Court)⁴ and that its proposal met the minimum mandatory experience requirements in section 1.3 of section C of the RFP. In addition, SWI alleged that the new evaluation (the re-evaluation) team applied the evaluation criteria contained in the RFP inconsistently with respect to its proposal.

SWI requested, as a remedy, that the Tribunal postpone the award of any contract in relation to this solicitation until the validity of the complaint is determined. Furthermore, it requested that the Department be instructed to re-evaluate the technical merits of the proposals, as determined by the evaluation team, in accordance with the evaluation methodology set out in the RFP and proceed thereon with this procurement, as provided in the RFP and the AIT. In the alternative, SWI requested that the Tribunal issue a determination that its proposal was wrongly disqualified and recommend that the contract be awarded to SWI, the lowest-priced responsive proposal. In the further alternative, SWI requested compensation for lost profits. SWI requested reimbursement for its costs for preparing a bid in response to this solicitation and for filing and pursuing the complaint.

On November 14, 2000, the Tribunal informed the parties that the complaint had been accepted for inquiry, as it met the requirements of subsection 30.11(2) of the CITT Act and the conditions set out in subsection 7(1) of the Canadian International Trade Tribunal Procurement Inquiry Regulations. ⁵ That same day, the Tribunal issued an order postponing the award of any contract in relation to this solicitation until the Tribunal determined the validity of the complaint. On November 27, 2000, the Tribunal granted MIL Systems, a Division of Davie Industries Inc., and Fleetway Inc. (MIL/Fleetway), intervener status in the matter.

On December 21, 2000, the Department filed a Government Institution Report (GIR) with the Tribunal in accordance with rule 103 of the Canadian International Trade Tribunal Rules. ⁶ On January 15, 2001, SWI filed comments on the GIR with the Tribunal. On February 7, 2001, MIL/Fleetway filed comments with the Tribunal on SWI's response to the GIR and, on February 20, 2001, SWI filed comments in response with the Tribunal.

Given that there was sufficient information on record to determine the validity of the complaint, the Tribunal decided that a hearing was not required and disposed of the complaint on the basis of the information on record.

PROCUREMENT PROCESS

Background

This complaint deals with the re-evaluation of the technical merits of SWI's proposal submitted in response to Solicitation No. W8483-6-EFAA. This re-evaluation was conducted pursuant to the Tribunal's determination in File No. PR-99-034R,⁷ wherein the Tribunal recommended that "the Department of Public Works and Government Services and the Department of National Defence re-evaluate the technical merits of the proposals submitted by MIL Systems (a Division of Davie Industries Inc.) and Fleetway Inc., a joint venture, and by Fleetway Inc. and the proposal submitted by Siemens Westinghouse Incorporated, in accordance with the evaluation methodology set out in the [R]equest for Proposal, and proceed thereon with this procurement as provided in the Request for Proposal and the Agreement on Internal Trade".

In its statement of reasons dated March 21, 2000, the Tribunal made the following finding with respect to the evaluation of the proposals:

The Tribunal finds that the Handbook^[8] significantly altered the evaluation methodology set out in the RFP. This is a breach of Article 506(6) of the AIT, which requires that the tender documents clearly identify the methods of weighting and evaluating the evaluation criteria and that such a clearly stated methodology be used in the evaluation of proposals. In the Tribunal's opinion, the Handbook introduced an evaluation and weighting methodology that could not be anticipated or derived from the methodology set out in the RFP. Because, in the circumstances, bidders could not reasonably expect such an evaluation approach, it was impossible for bidders to structure their proposals accordingly. Therefore, in the Tribunal's opinion, it was unfair to evaluate proposals against such an unannounced and unexpected evaluation yardstick. Furthermore, the Tribunal is of the view that, by staggering the scores into narrow bands away from both ends of the evaluation scale, i.e., 0.8 for a pass and 0.3 for a fail instead of the anticipated 1.0 for a perfect response and 0.0 for a totally devoid one, the Department has altered the evaluation methodology announced in the RFP in favour of less experienced bidders.⁹

Subsequent to the issuance of the Tribunal's determination, SWI and MIL/Fleetway filed judicial review applications with the Court. The Court upheld the Tribunal's determination with respect to the propriety of the evaluation methodology as follows:

[T]he Tribunal's finding that the change in evaluation methodology adopted in the handbook constituted a material change is unassailable. Reasonably viewed, the change could favour less experienced bidders. In theory, a pre-qualified but relatively inexperienced bidder could benefit from the change for the reason that it is guaranteed a minimum of 35% and a maximum of 85% even if it were deserving of less because of its relative inexperience. On the other hand, it is theoretically possible that a more experienced bidder could be disadvantaged by this scoring system if it is assumed that this bidder is more likely to obtain scores in excess of the minimum and maximum because of its enhanced experience. As well, the Tribunal cannot be faulted for concluding that it is impossible to assert with certainty whether any of the proposals would qualify under the evaluation methodology set out in the Request for Proposals.¹⁰

The Court did not, however, agree with the Tribunal that the Siemens bid was non-responsive and ordered that the Siemens bid be re-evaluated also. For this and the above reasons, the Department terminated the contract with SWI and commenced the re-evaluation of the technical merits of both the MIL/Fleetway and Siemens bids.

Re-evaluation

In May 2000, the Department and DND started setting up an independent structure for re-evaluating the technical merits of the proposals. It was decided that the Department would lead and coordinate the re-evaluation. An official from the Department was assigned as chairperson of the re-evaluation team (comprised of five experienced DND officials and one experienced departmental official). To create a "firewall" to separate the re-evaluation team from the evaluation team, its notes and documentation, or any other official involved in any way with the original evaluation, it was decided that the chairperson of the re-evaluation team would act as liaison with the assigned departmental legal counsel who, in addition to providing legal advice to the re-evaluation team, would act as a primary channel of communication with the Department and DND officials, when necessary. Alternatively, the chairperson of the re-evaluation team would communicate, when necessary, with departmental and DND officials through a designated departmental point of contact.

In late May 2000, an independent fairness monitor was appointed and contracted from the private sector to oversee the entire re-evaluation process. On July 11, 2000, the re-evaluation team completed the development of the evaluation handbook (the re-evaluation handbook). The re-evaluation handbook established the rules for the re-evaluation, set out the assumptions that would apply, and enumerated the roles and responsibilities of the members of the re-evaluation team, the independent chairperson and the fairness monitor (see page 4). The re-evaluation handbook and related scoring documentation were developed independently of any reference to or examination of the evaluation handbook, or review of the bids.

The following principles set out in the re-evaluation handbook applied to the re-evaluation:

The re-evaluation was conducted on the basis of the following assumptions set out in the re-evaluation handbook:

Three types of documents, i.e. a Detailed Evaluation Plan (the Plan) Scoring Table, a Detailed Evaluation Plan Matrix and Personnel Evaluation Detailed Scoring Tables, were also developed by the re-evaluation team.

The following provisions of the RFP are particularly relevant to this case:

SECTION C - EVALUATION CRITERIA

To be considered responsive, a proposal must (a) meet all the mandatory requirements of this Request for Proposal and (b) obtain the required minimum marks for technical merit. Proposals not meeting (a) or (b) above will be given no further consideration and deemed non-responsive.
A proposal will be declared non-responsive if it fails to provide the supporting evidence required by a mandatory item or for non-compliance with any mandatory requirement set out in this Request for Proposal (RFP) document or its Annexes.
The lowest priced responsive proposal will be recommended for award of a contract.
1.0 MANDATORY REQUIREMENTS
. . .
1.3 The Bidder must provide with its proposal, evidence of having the following qualifications and experience:
a) Completion of (ie within the last 5 years), or currently managing, at least one (1) contract valued at $1 M or more, in the fields of engineering support and technical data management of Canadian Forces vessels;
b) Provision of a Project Manager with significant experience (5 years in the last 8) in the management of design, engineering and drafting projects resulting from contracts valued at $1 M or more;
c) Provision of a Technical Data/Configuration Manager with significant experience (5 years in the last 8) in the management of marine technical data with at least one (1) year's experience in the management of Canadian Naval technical data;
d) Provision of a Quality Manager with significant experience (5 years in the last 8) in Quality Assurance of engineering and technical documentation;
e) Provision of at least one (1) qualified Engineer and one (1) qualified Technologist at the Senior and Intermediate level for each of the following marine disciplines: Naval Architecture/Hull Engineering, Marine Engineering, and Ship Electrical/Electronics Engineering; and the provision of at least one (1) qualified Engineer (ie. Electrical, Electronic, Computer or Mechanical) at the senior and intermediate level with experience in the field of Ship Combat Systems Hardware Integration. Senior level personnel must have at least 10 years experience (in the last 15 years) and Intermediate level personnel must have at least 5 years experience (in the last 8 years) within the applicable marine field with at least one (1) year's experience in Naval warship engineering projects;
f) Provision of at least one (1) Draftsman and one (1) Technical Writer at the Senior and Intermediate level with marine technical documentation experience. Senior level personnel must have at least 10 years experience (in the last 15 years) and Intermediate level personnel must have at least 5 years experience (in the last 8 years) with at least one (1) year's experience in Canadian Naval technical documentation;
g) Provision of at least one (1) person with significant experience (5 years in the last 8) in Technical Data Management and Life Cycle Material Management work processes of DND/DGMEPM [Director General Maritime Equipment Program Management]; and
h) Provision of at least one (1) person with significant experience (5 years in the last 8) in translation of marine technical documentation.

Note : Proof of personnel qualifications shall be provided in the form of a degree/diploma or certificate (in the applicable field) from a recognized educational institution or a minimum TQ6B qualification (for technologists only) from the Canadian Forces. Proof of the required experience shall be in the form of a detailed résumé for each person proposed.

. . .
1.7 A9001T 31/03/95 Education/Experience Certification

Bids, to be considered responsive must contain the following certification:
"The Bidder hereby certifies that all statements made with respect to education and experience are true and that any person proposed by the Bidder to perform the Work or part of the Work is either an employee of the Bidder or under a written agreement to provide services to the Bidder.

. . .

Canada reserves the right to verify the above certification and to declare the bid non-responsive for any of the following reasons:

- unverifiable or untrue statement
- unavailability of any person proposed on whose statement of education and experience Canada relies to evaluate the bid and award the Contract.
1.8 Proposals must be compliant for technical merit, ie: achieve a pass mark of at least 60% for each of the eight (8) major technical evaluation criteria and an average mark of at least 70% overall when assigned against rated requirements as per paragraph 2.0 below.
. . .
2.0 TECHNICAL MERIT
Proposals shall be evaluated for technical merit as per Annex "H" attached. This evaluation shall assess the Bidder's comprehension of the Work; its knowledge of the DND organization and its vessel fleet; its knowledge of the DND process for technical documentation and configuration control; and the extent of additional experience above the minimum (mandatory) requirement. The presentation of the proposal shall be evaluated with respect to supporting details, logic, clarity and overall completeness/quality.
Annex "H" of the RFP reads, in part, as follows:
I. MINIMUM REQUIREMENTS
To be considered technically compliant, Bidders must score at least 60% in each of the eight sections listed below for Rated Requirements and must attain a minimum of 70% overall. The categories of Rated Requirements are:

II. RATED REQUIREMENTS
Proposals shall be rated for technical merit in accordance with the SOW [Statement of Work], and the Bidder's organization, resources and facilities tendered.
Documents in I. (a) through (h) above shall be evaluated against the following criteria: comprehension of the work, how the work will be performed, personnel tendered, and the logic and clarity of presentation.
. . .
2.0 Project Management Organization Plan - 350 Points
The Project Management Organization Plan describes the roles and responsibilities of the Bidders personnel and subcontractors, and how the Contract will be managed. Some of the key areas for which information is to be provided are identified in SOW para 4.2.
The Project Management Organization Plan shall include a subcontract Plan to define the functions necessary to manage all subcontract activities required in support of the Class Design Agent and Technical Data Agent functions.
The Project Management Organization Plan will be evaluated based upon the following:

. . .
6.0 Data Link Access Plan - 100 Points
The Data Link Access Plan shall outline and describe the plan and implementation schedule for the provision of a Data Transmission Link to the Louis St. Laurent Building in Hull, Quebec, to provide remote access to the Technical Data Configuration System. The Plan shall also identify the standardized packaging methodology (Transfer Package protocol) for the delivery/packaging of the various electronic data elements. The Link shall have the capability to transfer files, ie. Drawings, publications, TDANs [Technical Data Action Notices], Data Lists, status reports, etc., as outlined in SOW para 4.6.
The Data Link Access Plan will be evaluated on the following set of criteria:

7.0 Transition Plan - 300 Points

a. The Transition Plan shall address all of the resources required to transition and take custody of the Technical Data as per the SOW Section 5;
b. The Transition Plan shall identify for each of the above Plans, the implementation schedule to put in place the elements which are not presently part of the Bidder's organization/facility; and
c. the Transition Plan shall identify the Implementation Plan for the development, loading and vetting of the Technical Data Configuration System (TDCS).

The Transition Plan will be evaluated for methodology, completeness, feasibility and the time frame required, as follows:

The individual assessment of the proposals began on July 12, 2000. It is at this stage that the members of the re-evaluation team looked at the bids for the first time. At the end of July 2000, they determined that some pages were missing from the résumés submitted by SWI. The chairperson and legal counsel determined that the evaluation team had also determined that these pages were missing from the original bid. Therefore, it was decided that the re-evaluation of SWI's bid would proceed without seeking to replace the missing pages.

By mid-September 2000, all individual scoring was completed, and the members of the re-evaluation team held consensus¹³ review meetings, in accordance with the re-evaluation handbook.

On September 15, 2000, the re-evaluation team's scoring was finalized and the Personnel Evaluation Detailed Scoring Tables were prepared and signed by the evaluators. These tables show that SWI's proposal failed section 2.0 (Project Management Organization Plan), section 6.0 (Data Link Access Plan) and section 7.0 (Transition Plan) of Annex "H" of the RFP.

On September 19, 2000, the fairness monitor stated, in part, the following in his final report:

I hereby certify that:
a. none of the re-evaluation team members had been involved in the previous evaluation or identified any involvement with the parties that would jeopardise his independence in performing the re-evaluation;
b. all documentation used for the re-evaluation was a clean copy of the original, without any significant markings that may have been made by the original evaluation team;
c. the detailed re-evaluation process and tools were fair and flowed directly and logically from the solicitation package and evaluation plan; and
d. the re-evaluation process was executed in an equitable manner.

On October 5, 2000, SWI was advised that its proposal was evaluated as being non-compliant in the re-evaluation. On October 13 and 19, 2000, SWI objected in writing to the results of the re-evaluation.

POSITION OF PARTIES

Department's Position

With respect to SWI's allegation that the re-evaluation handbook applied weighted criteria to the rated requirement categories that were not set out in either Section C or Annex "H" of the RFP, the Department submitted that the criteria applied in the re-evaluation were the criteria identified in the tender documents at both Section C and Annex "H". The Department submitted that the criteria identified in the Plan could be directly linked to the provisions of Section C and Annex "H". Furthermore, although the Plan and the RFP are not identical in all respects, the Department argued that there is a reasonable relationship between the two documents. The Department submitted that the evaluation criteria identified in the Plan clearly address the same type of criteria, as set forth in Annex "H" and section 1.9 of Section C of the RFP. Therefore, the Department added that the terms used in the Plan could not be construed as an attempt to add unknown criteria after the RFP was published. The Department submitted that the assessment of the feasibility of the proposed plan, the depth and experience of staff and the adequacy of resources and facilities are necessary components of the assessment of a bidder's capacity to do the work.

With respect to SWI's contention that the re-evaluation of its proposal with respect to sections 2.0, 6.0 and 7.0 of Annex "H" of the RFP contravened Article 506(6) of the AIT, because the evaluators assessed each subcriterion of these sections against undisclosed criteria and weighting, the Department submitted that the evaluators assessed each subcriterion in accordance with the criteria set out in Section C and Annex "H" and that the percentage of the score attributed to the criteria assessed was reasonable, logical and consistent with the evaluation criteria found at sections 1.8, 1.9 and 2.0 of Section C and sections 2.0, 6.0 and 7.0 of Annex "H". The Department submitted that the subcriteria evaluated and the percentage distribution of points against the subcriteria were necessary to ensure consistency among evaluators and to provide an objective measurement device.

Moreover, the Department submitted that the fact that the Plan was developed in response to the Tribunal's determination dated July 21, 2000, and, therefore, after the publication of the RFP, does not contravene the principles of transparency and fairness of the AIT because: (1) the specific percentage distribution of points for the subcriteria against which each section was evaluated is consistent with the RFP and the relative importance of the criteria therein; (2) there was no introduction of new capability requirements not contained in the RFP; (3) sufficient transparency was evident in the criteria and weighting of scores set forth in the RFP; and (4) the development of the Plan was necessary for a proper implementation of the Tribunal's recommendation dated July 21, 2000.

Concluding on this point, the Department submitted that the Plan did not add undisclosed weighting or criteria, nor did the evaluators contravene requirements of fairness and transparency in re-evaluating the proposals in accordance with the Plan. The Plan, the Department submitted, provided precise common objective indicators for assessing the criteria in Section C and Annex "H" of the RFP and, while not identical in all respects, there was both consistency and a reasonable relationship between the provisions of the Plan and the RFP.

With respect to SWI's allegation that the re-evaluation methodology was premised on a misinterpretation of the Tribunal's March 21, 2000, statement of reasons, the Department submitted that SWI's suggestion of simply exchanging a 0.8 for a 1.0 as a "passing" score in the methodology, and a 0.3 for a 0.0 as a "failing" score, would not substantially alter the scoring methodology. The Department submitted that the approach proposed by SWI fails to address the concerns expressed by the Tribunal that the pass/fail methodology used in the evaluation significantly altered the methodology set out in the RFP in a manner that could not be anticipated or derived by bidders. Furthermore, the above approach also fails to address the concerns expressed by both the Tribunal and the Court that the scoring system used for the evaluation favoured less experienced bidders. Indeed, the Department submitted that it should be inferred from Tribunal and Court decisions that responses assessed as a "pass" should properly have been applied a factor from 0.0 to 1.0 or a score from 0 percent to 100 percent.

The Department added that "[c]onsensus (within a range of 20%) is important as a tool to prevent inconsistency and inaccuracy in marking". Accordingly, the Department submitted that there is no basis for a complaint on the ground that the consensus scoring or the averaging of the scores of all evaluators was unfair or contravened the requirements of the trade agreements. Furthermore, there is no evidence to support SWI's contention that the bidders were prejudiced because there were only five evaluators on the re-evaluation team.

With respect to SWI's allegation that the re-evaluation team failed to apply criteria consistently with respect to its proposal on the basis that the qualifications of certain key management personnel, i.e. the project manager, the technical data/configuration manager and the quality assurance manager (key personnel), were deemed sufficient to pass the mandatory requirements of the Letter of Interest (LOI) and RFP evaluation, but failed the re-evaluation of technical merits, the Department submitted that the evaluation of the mandatory requirements is not pertinent to this complaint. The Department added that, as a result of the Tribunal's determination to re-evaluate only the technical merits of the proposals, it was assumed that all bidders were compliant with the mandatory requirements of the RFP. Therefore, the mandatory requirements were not re-evaluated as such.

Furthermore, the Department submitted that the re-evaluation with respect to section 2 of Annex "H" of the RFP necessarily required consideration of the "roles and responsibilities of the Bidders personnel and subcontractors". In this context, a number of key personnel proposed by SWI were found not to have the experience in five of the last eight years, as set out in section 1.3 of Section C of the RFP and in Table 2 of the Personnel Evaluation Detailed Scoring Tables used for evaluating section 2.0c of Annex "H" of the RFP. On this basis, the re-evaluation team awarded failing scores for technical merit when assessing the experience of these individuals and, as a consequence, SWI's composite score for section 2.0 of Annex "H" did not achieve the requisite 60 percent passing mark, as set out in the RFP.

With respect to SWI's allegation that the failure to re-evaluate the technical merits of the Fleetway proposal contravened the Tribunal's recommendation, the Department submitted that the decision to conduct the re-evaluation of the technical merits of the MIL/Fleetway proposal and then apply the scoring from that re-evaluation to the Fleetway technical proposal was reasonable and fair to all bidders. This decision, the Department asserted, was made for reasons of efficiency and expediency, in order to avoid unnecessary duplication of effort and in an attempt to complete the re-evaluation as quickly as possible without compromising the need to conduct a thorough and objective re-evaluation. This decision was made by the re-evaluation team only after receiving the advice of its legal counsel that the contents of the technical proposals of Fleetway and MIL/Fleetway were equivalent. More importantly, the Department submitted, there is no prejudice to SWI as a result of this decision.

With respect to SWI's allegation that the evaluation methodology designed by the re-evaluation team cannot be deemed to be fair and equitable because the Department had knowledge of the contents of at least two prospective bids that were to be evaluated at the time that the re-evaluation team prepared the re-evaluation handbook and scoring documentation, the Department submitted that there is no basis in fact to this allegation, as was acknowledged by SWI in paragraph 3 of its "Summary of Complaint".

SWI alleged that there is a possibility that the proposal before the re-evaluation team was not the same proposal that was before the evaluation team because of a security incident, which occurred after the completion of the evaluation. In response, the Department submitted that, to the best of its knowledge, the re-evaluation team assessed proposals identical to the original proposals. The Department submitted that the security incident referred to by SWI occurred at the premises of DND; however, the Department maintained a complete set of bidders' original proposals and was able to verify that the missing résumé pages were also missing from the set that it kept and that were not subject to any security incident. More importantly, the Department argued that, even if the security incident resulted in the removal of pages of résumés of proposed personnel, which is not admitted, the re-evaluation of SWI's technical proposal as non-compliant with respect to sections 6.0 and 7.0 of Annex "H" of the RFP would not have been materially altered, as the Department and DND's non-compliance determination was, in large part, based on factors not related to the résumés at issue.

The Department requested the opportunity to make further submissions with respect to costs.

MIL/Fleetway's Position

With respect to SWI's allegation that the re-evaluation handbook injected new subcriteria into certain categories of the rated requirements, MIL/Fleetway submitted that the re-evaluation handbook simply allocated relevant subcriteria to each of the categories in a logical way. It stated that the subcriteria were already identified in section 2.0 of Section "C" and Annex "H" of the RFP and that there existed no difference in substance between the language used in the re-evaluation handbook and in the RFP to identify these subcriteria.

With respect to the weighting attached to the subcriteria, MIL/Fleetway submitted that, contrary to SWI's allegation, they merely represented a logical tool for evaluators. The weighting of the subcriteria did not change the clearly disclosed weighting in Annex "H" of the RFP attributed to each of the categories of the rated requirements and, in fact, MIL/Fleetway asserted, such a rating was implicit in the RFP documents.

With respect to SWI's allegation that the evaluation methodology used for the re-evaluation did not comply with the RFP or the Tribunal's determination of March 21, 2000, MIL/Fleetway submitted that the Tribunal, in its determination, specifically rejected the pass/fail approach used in the evaluation and that to urge another pass/fail approach for the re-evaluation demonstrates that SWI's analysis on this point is fundamentally flawed.

With respect to SWI's allegation that key personnel evaluated as satisfactory under subsections 4(b), (c) and (d) of the LOI and sections 1.3b), c) and d) of Section C of the RFP cannot reasonably be declared unsatisfactory when evaluated by the re-evaluation team against exactly the same criteria under section 2.0 of Annex "H" of the RFP, MIL/Fleetway submitted that it is wrong to say that the LOI had the same criteria in that respect as those contained in the RFP. In fact, MIL/Fleetway submitted that the requirements in the LOI with respect to the above-mentioned key personnel were less onerous than the requirements under the re-evaluation of the technical merits of the proposal.

Furthermore, MIL/Fleetway submitted that, in conducting the re-evaluation, the Crown, as per the terms of the RFP, was required to assess the degree to which bidders exceeded the mandatory minimum experience requirements identified in the RFP. This assessment therefore required an evaluation of the degree to which key personnel were qualified beyond a mere acceptability. Similarly, MIL/Fleetway submitted that, although the language of sections 1.3b), c) and d) of Section "C" of the RFP is similar to the criteria identified under the Personnel Evaluation Detailed Scoring Tables, section 1.3 of Section C of the RFP was designed to demonstrate a bidder's compliance with minimum mandatory requirements. Passing a minimum mandatory requirement does not involve the more stringent assessment involved in the technical evaluation that was designed to assess the extent of additional experience. Concluding on this point, MIL/Fleetway noted that, in any event, the Crown, conforming to the Tribunal's determination, conducted a re-evaluation only of the technical merits of bidders' proposals to determine compliance with the rated requirements of the RFP.

With respect to SWI's allegations that the evaluators were aware of the contents of proposals in advance of the re-evaluation and that the Department, in breach of the Tribunal's determination, failed to re-evaluate Fleetway's proposal, MIL/Fleetway submitted that there is no evidence to support the former and that the latter did not and could not prejudice SWI.

Concerning SWI's allegation that a security incident affected its proposal, MIL/Fleetway submitted that there is no evidence to support this ground of complaint. In any event, MIL/Fleetway argued that, on July 27, 2000, SWI had full knowledge of all the facts relating to the alleged security incident and that the re-evaluation had begun. MIL/Fleetway argued that, if SWI had any concern about the integrity of its proposal arising out of the incident, it should have complained about this matter at that time. This was not done and, MIL/Fleetway submitted, this ground of complaint is therefore out of time.

MIL/Fleetway argued that a procuring authority is entitled to use an evaluation handbook, provided the evaluation criteria that it contains have a reasonable relationship to, or are reasonably consistent with, the RFP. Furthermore, MIL/Fleetway submitted that, while a solicitation must inform bidders of the basis of the evaluation, a procuring authority is not required to identify every single layer of detail involved in the factors to be taken into account in an evaluation and/or a re-evaluation.

In its comments of February 7, 2001, on SWI's response to the GIR, MIL/Fleetway submitted that both the Tribunal and the Court concluded that the pass/fail methodology used by the evaluation team was inconsistent with the RFP and impermissible. The defect found by the Tribunal and the Court was serious and, MIL/Fleetway argued, required that the technical proposals be entirely re-evaluated, not merely recalculated.

MIL/Fleetway submitted that, given the evaluation team's awareness of the pricing information for each of the proposals, the Crown was required to create an impartial re-evaluation team. Furthermore, MIL/Fleetway submitted that the size and composition of the re-evaluation team is consistent with the RFP and the AIT.

MIL/Fleetway further argued that the documents allegedly not provided to the re-evaluation team are irrelevant because they contain no information that is not already contained in SWI's proposal. In any event, MIL/Fleetway submitted that these documents would not have assisted SWI in meeting the rated requirements and, as such, SWI suffered no prejudice. MIL/Fleetway further argued that SWI's allegation that the re-evaluation team gave too much importance to personnel issues fails to recognize that personnel had to be evaluated with regard to different aspects of the rated requirements that call for different skills.

MIL/Fleetway also submitted that the re-evaluation properly assessed the qualifications of key personnel and that the final marks for the re-evaluation met the 20 percent consensus requirement. As well, MIL/Fleetway argued that the points given to key personnel for additional years of experience and for past contracts in excess of $1 million would become relevant if a bidder met the minimum technical requirement, which SWI failed to do. In any event, MIL/Fleetway added, these additional points were only applicable to section 2.0 of Annex "H" of the RFP and, as such, even if this aspect of the evaluation affected SWI's technical re-evaluation (which is denied), SWI would nevertheless have failed sections 6.0 and 7.0 of Annex "H". Again, MIL/Fleetway submitted, SWI has simply suffered no prejudice.

Finally, MIL/Fleetway argued that SWI's assertion that it should have passed under the rated requirements of sections 6.0 and 7.0 of Annex "H" of the RFP, because the results of the re-evaluation contradicted the results of the evaluation, is illogical. MIL/Fleetway argued that both the Tribunal and the Court have concluded that the evaluation was fundamentally flawed and that one or all of the proposals might be technically non-compliant on a re-evaluation. That is why, MIL/Fleetway submitted, a re-evaluation was required.

SWI's Position

SWI submitted that there is something inherently suspect about a re-evaluation process that leads to such a contradictory result from the evaluation in which its proposal was selected as the successful proposal by a qualified and experienced team of evaluators. Such a totally inconsistent outcome, SWI argued, cannot be reliable and results from a deeply flawed approach to the re-evaluation. SWI submitted that a truly fair and objective re-evaluation of the same proposal, using the same RFP criteria, should lead to the same or to substantially similar results as those achieved during the evaluation.

SWI submitted that the re-evaluation was inconsistent with the RFP and with the previous rulings made by the Court and the Tribunal. It introduced new and undisclosed subcriteria and weighting, a new computational procedure and more. SWI submitted that something was seriously wrong when the very same proposal goes from being the successful proposal to being disqualified as technically inadequate. It stated: "SWI's proposal did not change, and the criteria set out in the RFP did not change. The only things that changed were the evaluation team and the methodology they employed".

SWI indicated its belief that the re-evaluation team did not possess the same degree of technical association with the project as the evaluation team. Indeed, it argued that, during the evaluation, it was the technical authority set out in the RFP itself who determined that SWI had passed the technical evaluation. The re-evaluation team was half the size of the evaluation team and, for obvious reasons, being comprised of second choices, necessarily had less expertise to draw upon. As such, SWI submitted that the re-evaluation team was less well positioned to understand the requirements of the RFP and to judge the technical merits of its proposal. In addition, there appears to have been no information technology expert on the re-evaluation team, thereby possibly explaining why SWI's proposal failed section 6.0 of Annex "H" of the RFP during the re-evaluation.

SWI further submitted that the complete lack of continuity in the assessment of proposals also affected the fair re-evaluation of its proposal. SWI submitted that the re-evaluation was conducted in such a fashion as to destroy any continuity between evaluation teams and the information used by the evaluators. The "firewall" created between the evaluation and the re-evaluation teams prevented the re-evaluation team from having access to the technical expertise of the evaluation team, from seeking advice and guidance from the technical authority designated in the RFP and from seeking contractual advice respecting SWI's subcontracting agreements and agreements to have access to personnel.

Moreover, SWI submitted that, by deliberately isolating the re-evaluation team from the notes and documentation available to the evaluation team, the re-evaluation team was not provided with the same material that the evaluation team had before it. This decision, SWI submitted, was a serious error on the part of the Department, since the use of such material was expressly sanctioned by the Court. As a result of this difference in record, SWI submitted that the assessment of the mandatory requirements under the RFP done by the evaluation team was based on different documents than those used by the re-evaluation team. Thus, different parts of the SWI proposal were evaluated by different teams on the basis of different documentation. SWI submitted that the whole of its proposal should have been evaluated on the basis of the same documentation.

SWI contended that the evaluation methodology developed by the Department and DND to carry out the re-evaluation was not in conformity with the Court's judgement. For example, SWI argued that, contrary to the Court's decision, the result of the pre-qualification proceeding under the LOI and the result of the evaluation of the mandatory requirements in the RFP by the evaluation team were ignored by the re-evaluation team in its conduct of the re-evaluation.

SWI submitted that the error found by the Court to have been committed by the evaluation team was limited to a minor computational error in calculating scores. SWI argued that the scoring system contained in the RFP for evaluating the technical proposals is essentially based on a pass/fail grid, as attested by a government memorandum dated October 28, 1998, before the publication of the RFP on November 4, 1998. As a result, SWI argued, the pass/fail methodology used by the evaluation team was consistent with the RFP, and there was no other way of properly conducting the evaluation or re-evaluation. SWI submitted that the Court did not find fault with the manner in which the evaluation team had derived the 549 evaluation items from the SOW nor with the judgement of the evaluation team that SWI's proposal passed a major portion of those items. The Court, SWI submitted, only criticized the computational procedures as possibly favouring less experienced bidders. The error with the evaluation, SWI argued, was only in using a compressed scoring equation. Therefore, all that was required to comply with the Court judgement was to recalculate the conversion of the "passes" and "fails" into numerical scores without using the compressed formula.

With respect to the establishment by the Department and DND of a "firewall" for the conduct of the re-evaluation, including notes and documentation of the evaluation, SWI submitted that this approach deprived the re-evaluation team of important documents relating to SWI's previous contract experience and key personnel experience, the usage of which by the evaluation team was supported by the Court.

Furthermore, SWI submitted that the conduct of the re-evaluation breached the requirements of Article 506(6) of the AIT, in that the Department and DND used evaluation subcriteria and weighting not identified in the RFP or consistent with the criteria and weighting identified in the RFP. SWI submitted that, arguing as the Department does, that the weighting, as assigned, was reasonable and logical is not the test to apply in the circumstances. The test in the AIT is straightforward and requires the clear identification in the solicitation documents of the evaluation criteria and weighting methodology that will apply. In this case, it is clear that the weighting used in the re-evaluation was not disclosed in the tender documents.

SWI added that, contrary to assertions in the GIR, neither the subcriteria nor the weighting used in the re-evaluation flows clearly and logically from the criteria and methodology set out in the RFP. SWI submitted, for example, that some items of the SOW were weighted differently or counted in a number of sections of Annex "H" of the RFP, thus distorting the re-evaluation by imposing an evaluation grid unknown to bidders.

With respect to the failure of SWI's proposal in specific sections of the rated requirements during the re-evaluation, SWI submitted that the three sections concerned were, in fact, three of the least important sections set out in Annex "H" of the RFP, accounting for 20 percent of the total available evaluation points.

Specifically, with respect to the re-evaluation of section 2.0 of Annex "H" of the RFP, SWI submitted that the re-evaluation was, in part, based on a "demonstrated understanding" subcriterion worth 10 percent¹⁴ of that section's total available evaluation points, a criterion that was not disclosed in the RFP and demonstrably unreasonable. SWI argued that this fact is not insignificant. Similarly, SWI submitted that the application of the "demonstrated understanding" criterion to the evaluation of the qualification of key personnel was unannounced in the RFP and has no clear meaning in relation to the qualification of personnel.

With respect to the re-evaluation of the qualifications of key personnel, SWI submitted that the great divergence in the scores awarded by the different evaluators is, in itself, difficult to reconcile. Furthermore, certain low scores assigned to key personnel are only possible if one ignores the fact that the mandatory criteria of the RFP had been met by SWI's proposal. However, SWI submitted that, since it had been conclusively determined by the evaluation team that the key personnel proposed by SWI possessed the qualifications required by section 1.3 of Section C of the RFP and since this determination was supported by the Tribunal's recommendation to re-evaluate only the technical merits of proposals, then each of SWI's key personnel should automatically have been awarded at least passing grade points in the re-evaluation. This, SWI submitted, would have resulted in SWI's proposal passing section 2.0 of Annex "H" of the RFP.

SWI submitted that there was no reference in section 2.0 of Annex "H" of the RFP to any points being awarded based on the value of contracts. Therefore, this criterion was arbitrary and unknown to bidders. Moreover, to award no points to an individual possessing extensive qualifications, as is apparent from the résumé submitted in the proposal, is particularly absurd and contrary to Annex "H". Indeed, SWI submitted, how was it possible for SWI to successfully complete the transition phase of this requirement without qualified managers?

With respect to the re-evaluation of section 6.0 of Annex "H" of the RFP, SWI submitted that the GIR fails to explain why this section of SWI's proposal was unsuccessful. Moreover, with respect to the failing grade assigned by the re-evaluation team to SWI's proposed transition plan in section 7.0 of Annex "H", SWI submitted that this conclusion is simply insupportable and wrong, given the fact that it successfully completed the transition phase in accordance with its transition plan.

SWI submitted that bidders were told in the RFP that their capabilities¹⁵ would be assessed on the basis of a variety of different factors. For the re-evaluation team to pick one component out of many, namely, "depth and experience of staff", and to make that simple component worth a disproportionately large 80 percent of the available points under the "capability" category was clearly unfair and contrary to the terms of the RFP. Had SWI known that this criterion would, by itself, constitute such a significant element of the evaluation of sections 6.0 and 7.0 of Annex "H" of the RFP, it would have structured its proposal differently to highlight this depth and experience.

With respect to the decision not to re-evaluate the Fleetway proposal, SWI submitted that the MIL/Fleetway and Fleetway proposals are described in the GIR as "equivalent" or "virtually identical". This, SWI argued, is not the same as "identical" and does not justify taking the score awarded to one proposal and applying it to both. SWI also noted that it was the Department's legal counsel who advised that the contents of the two technical proposals were "equivalent". This determination, SWI submitted, is a judgement on technical issues that legal counsel is not qualified to make.

With respect to the completeness of SWI's proposal after the security incident and the Department's assertion that such allegedly missing résumé pages would not have materially altered the outcome of the re-evaluation of sections 6.0 and 7.0 of SWI's proposal, SWI submitted that it is difficult to accept this view in light of the fact that the depth and experience of staff, presumably as reflected in their résumés, accounted for 80 percent of the available points under the "capability" subcriterion in these sections.

In its comments of February 20, 2001, SWI submitted that the results of the evaluation cannot be ignored. Only some elements of the evaluation were found by the Court and the Tribunal to have been in error and the Department's mandate was only to correct the deficient elements in the evaluation and not to redo those parts of the evaluation where no error was found. One of the elements not found to be in error was the breakdown of the eight rated requirements of Annex "H" of the RFP into 549 subcriteria in accordance with the SOW in the RFP. SWI submitted that the original breakdown of criteria into 549 subcriteria and the assessment of passes and fails by the evaluation team should have been the starting point for the work done by the Department and DND to correct the errors identified by the Court. Furthermore, SWI submitted that the language of the RFP clearly supports the original breakdown of criteria based on the SOW as a proper decomposition of the SOW covering only the criteria, and all the criteria, without repetition. SWI submitted that the re-evaluation handbook does not demonstrate such a reasonable relationship between the methodology used for the re-evaluation and the RFP. SWI argued that this fact is not inconsequential because, if the Tribunal finds that SWI is incorrect with respect to the meaning of the prior Tribunal and Court decisions, the evaluation methodology employed by the re-evaluation team, when considered on its own, is obviously not in conformity with the criteria and weighting set out in the RFP.

In this latter respect, after discussing the application to this case of several determinations¹⁶ by the Tribunal, SWI argued that the views of the U.S. Navy Rear Admiral (Ret'd), retained by MIL/Fleetway to advise on the question of the conformity of the re-evaluation handbook to the evaluation methodology set out in the RFP, are just personal opinions that are easily open to questions. SWI submitted that the specialist in the theory of measurements that it retained, Dr. Jonathan Barzilai, demonstrated that the Rear Admiral's conclusions were insupportable and based on assertions that are unsubstantiated. Furthermore, SWI claimed that the conclusions of the two reports of Dr. Eugene L. Jurkowitz, dated January 15 and 31, 2001, are also of questionable value.

SWI submitted that MIL/Fleetway's allegation that SWI was not prejudiced by the weighting of the subcriteria because it cannot pass the section on rated requirements ignores the fact that SWI did pass, with high scores, that section during the evaluation, which was conducted by qualified evaluators. SWI argued that it is difficult to see how a party could suffer more prejudice than by going from being the successful bidder and successfully performing the transition phase of the contract to having a bid, which was previously declared compliant and successful, disqualified.

With respect to the "consensus" methodology used by the re-evaluation team, SWI submitted that the disparity in rating the experience of key personnel indicates that no consensus existed at all among the evaluators. In fact, SWI added, there might not even have been a common understanding on how to score the proposals. Furthermore, SWI submitted that the "consensus" mechanism contained in the re-evaluation handbook introduced arbitrary elements into the scoring process. It is illusory, SWI argued, to describe a process in which evaluators are worn down or subjected to pressure to change their scores so as to achieve an arbitrarily selected spread as achieving "consensus".

With respect to the creation and the isolation of the re-evaluation team, SWI submitted that, where the results of the evaluation and of the re-evaluation are in conflict, it is logical to think that the results of the larger and more experienced review team would be more indicative of the actual merits of SWI's proposal. Concerning the Department's rationale for creating a totally different re-evaluation team, SWI submitted that the re-evaluation team must also have known that SWI submitted the lowest bid, since SWI had been awarded the contract.

With respect to the availability of certain documents to the re-evaluation team in the conduct of the re-evaluation, SWI submitted that the issue contrary to MIL/Fleetway's assertion is not whether the Department was obligated to provide documentation reviewed during the LOI process to the evaluation team. Rather, the issue is whether it was fair to SWI to deny the re-evaluation team's access to the documentation that had been assembled and reviewed by the evaluation team. SWI argued that if the re-evaluation team had been provided with this information, this might have affected its assessment of SWI's experience and capabilities. This information could have been relevant in scoring the qualifications of SWI's key personnel, the feasibility of SWI's plans, the depth and experience of SWI's staff and the resources and capability of SWI to carry out the work. In the final analysis, SWI added, there has to be a cause for the very illogical outcome of the re-evaluation. In SWI's submission, the lack of continuity in documentation must have prejudiced SWI in the re-evaluation of its technical proposal.

With respect to the identical nature of the bids submitted by MIL/Fleetway and Fleetway, SWI submitted that this assumption has not been established as correct and, therefore, it was an error not to have the two proposals available to the re-evaluation team.

TRIBUNAL'S DECISION

Section 30.14 of the CITT Act requires that, in conducting an inquiry, the Tribunal limit its consideration to the subject matter of the complaint. Furthermore, at the conclusion of the inquiry, the Tribunal must determine whether the complaint is valid on the basis of whether the procedures and other requirements prescribed in respect of the designated contract have been observed. Section 11 of the Regulations further provides, in part, that the Tribunal must determine whether the procurement was conducted in accordance with the AIT.

In its comments of February 20, 2001, SWI submitted that the key questions to be answered by the Tribunal, and which essentially relate to Article 506(6) of the AIT, are as follows:

The Tribunal will address these questions in turn. The Tribunal wants to make it clear at the outset that a number of the above questions, specifically questions 1 and 4, as worded, do not fall within its jurisdiction.

As noted earlier, on March 6, 2000, the Tribunal found that a complaint filed by MIL/Fleetway in relation to this solicitation was valid and recommended that the technical merits of the MIL/Fleetway and Fleetway proposals be re-evaluated in accordance with the methodology set out in the RFP. Subsequent to this decision, both SWI and MIL/Fleetway filed judicial review applications with the Court. In its judgement delivered on June 23, 2000,¹⁷ the Court decided that SWI's proposal should not have been found non-compliant by the Tribunal, yet upheld the Tribunal's determination that the evaluation methodology used by the Department and DND in assessing the technical merits of the proposals was flawed and its results unreliable. Accordingly, on July 21, 2000, the Tribunal modified its recommendation of March 6, 2000, and included SWI's proposal in the re-evaluation.

The Tribunal notes that the issue is not to determine whether the directions of the Court or the recommendation of the Tribunal in relation to this solicitation have been properly implemented or fully complied with by the Department and DND. Rather, adopting the Tribunal's position in File No. PR-97-008,¹⁸ the Tribunal is of the view that the implementation by the Department of its recommendation of July 21, 2000, effectively extended the procurement process¹⁹ for this solicitation and gave rise to the possibility of new challenges by potential suppliers, which are entitled to submit protests concerning any aspect of the procurement process. In this context, the Tribunal will consider and decide the grounds of complaint, properly filed by SWI, which allege that the conduct of the Department and DND in completing the re-evaluation breached the provisions of the RFP and the AIT. In so doing, the Tribunal will, of course, be mindful of the previous Court and Tribunal decisions in relation to this solicitation. However, the Tribunal will not review the extent to which the directions of the Court and the recommendation of the Tribunal have been complied with or implemented, because they do not constitute an aspect of the procurement process.

Article 506(6) of the AIT provides that "[t]he tender documents shall clearly identify the requirements of the procurement, the criteria that will be used in the evaluation of bids and the methods of weighting and evaluating the criteria".

With respect to the first question raised by SWI, the Tribunal will not decide the extent to which the Department's and DND's actions in developing the re-evaluation methodology and in establishing the re-evaluation team complied with the Court's directions and/or the Tribunal's determination.

However, the Tribunal realizes that it is possible to read SWI's question as follows: in light of the Court's and the Tribunal's decisions, did the Department and DND comply with the provisions of the AIT and the RFP when they established the re-evaluation team and developed the methodology? The Tribunal is prepared to answer this question as formulated. It finds that the Department did not breach the provisions of the AIT and the RFP when it established the re-evaluation team and developed the methodology to conduct the re-evaluation. The Tribunal is of the view that, in the circumstances, the Department and DND were well advised to form a team with new members to conduct the re-evaluation. In the Tribunal's opinion, this approach, along with the presence of a fairness monitor to oversee the re-evaluation process, was probably an appropriate precautionary measure to minimize the likelihood of allegations of bias, apprehended or real, in the re-evaluation process. The Tribunal is not saying that it was necessary for the Department to modify or completely change the evaluation team to conduct the re-evaluation. But, by so doing or by establishing a five-member re-evaluation team, the Department did not breach the AIT or the RFP. There is no provision in the RFP prescribing the size of the evaluation/re-evaluation team and, in the Tribunal's opinion, SWI has not established that the number of evaluators was insufficient in this instance. Similarly, although SWI has indicated that the members of the evaluation team had a higher degree of technical familiarity with the project than had the members of the re-evaluation team, SWI has not established that the re-evaluation team was inept in the circumstances. Insofar as the re-evaluation methodology is concerned, the Tribunal is of the view that the RFP and related SOW were open to a number of reasonable "decompositions" of their component parts for evaluation purposes. In the absence of any particular or specific "decomposition" prescribed in the RFP, the Tribunal is of the view that it was open to the Department and DND to adopt any "decomposition" that was consistent with the provisions of the RFP, which brings us to the second question.

With one exception described in the next paragraph, the Tribunal finds that the evaluation methodology, including the "decomposition" of the RFP and SOW adopted by the Department and DND to conduct the re-evaluation, is consistent with, reasonably related to and logically connected with the evaluation methodology set out in the RFP. In that sense, the said methodology did not introduce criteria or weighting that were not announced in the RFP or that could not reasonably be anticipated by bidders. Except for a limited number of application errors, discussed below, which are not prejudicial to the global assessment of SWI's proposal, in the Tribunal's opinion, the methodology used for the re-evaluation conformed to the criteria and weighting set out in the RFP. The Tribunal realizes that the re-evaluation handbook introduced subdivisions to the criteria and weighting set out in the RFP. This is a common occurrence when evaluating complex procurements. As such, it is the Tribunal's opinion that, as long as the subcriteria and weighting could readily be anticipated by bidders and be derived from the broader criteria and weighting set out in the RFP and, provided such subsets did not augment or diminish the relative worth and importance of the criteria and weighting as announced in the RFP, this kind of refinement to the evaluation criteria and methodology set out in the RFP to guide evaluators in their assessment of proposals is not contrary to Article 506(6) of the AIT. In addition, it is not contrary to the broader transparency requirements of the AIT.

SWI alleged that certain criteria and weighting introduced by the re-evaluation methodology could not be anticipated or derived from those set out in the RFP and, in fact, deviate from the weighting announced therein. More specifically, SWI alleged that the Department introduced in the re-evaluation of section 2.0 of Annex "H" of the RFP a "demonstrated understanding" criterion worth 10 percent of the evaluation points for this section, which is not set out in Annex "H". The Tribunal finds that this criterion, strictly speaking, is not expressly provided for in the RFP for the said section and amounts to the introduction by the Department and DND of an unannounced criterion and the diversion of 10 percent of the evaluation points of the section away from their announced purpose. However, considering the relative importance of the subcriterion and the fact that SWI's proposal obtained more than 60 percent of the points assigned to this unannounced criterion, the Tribunal is of the view that the impact of this error on the evaluation of section 2.0 of Annex "H" of SWI's proposal, by itself, is not material.

SWI also alleged that the use by the Department of the "depth and experience of staff" subcriterion in several sections of Annex "H" of the RFP, including sections 2.0, 6.0 and 7.0, amounts repeatedly to the unfair assessment of the same criterion. Furthermore, SWI alleged that this inequity was compounded by the fact that the weighting assigned to this evaluation subcriterion was exaggerated and out of line with that announced in the RFP. The Tribunal has carefully reviewed section 2.0 of Section C, sections 2.0, 6.0 and 7.0 of Annex "H" and sections 4.2, 4.6 and 5.0 of the SOW and is satisfied that, read together, these provisions clearly convey to bidders the importance that the Department and DND would attach to the depth and experience of the staff proposed in the evaluation of proposals. For this reason, the Tribunal is satisfied that the use of this subcriterion was clearly signaled in section 2.0 of Section C and that the weighting that it was accorded in evaluating proposals was consistent with its relative importance as announced in the RFP. In the Tribunal's opinion, the requirement being procured is fundamentally one for the provision of services whose quality depends, to a large extent, on the experience and expertise of the staff proposed. Moreover, the Tribunal observes that, although the depth and experience of staff were assessed under several sections of Annex "H", these assessments related to different personnel and skills, depending on the specific section under consideration. Therefore, as such, these assessments do not amount to evaluating the same subcriterion repeatedly, but, rather, represent the repeated use of the same subcriterion in different contexts.

With respect to the third question raised by SWI, the Tribunal finds that the Department and DND contravened the "fair treatment of proposals" procedural obligations²⁰ of the AIT, when it was decided to isolate the re-evaluation team specifically from the documentation considered by the evaluation team at the time that it assessed the minimum mandatory experience requirements in section 1.3 Section C of the RFP. The Tribunal is of the view that isolating the re-evaluation team from the notes and documentation used and the personnel involved in the evaluation of the rated requirements was acceptable, except for the fact that the specific wording of technical evaluation criterion "2.0c C2"²¹ was virtually the same as that of mandatory criterion 1.3 Section C of the RFP. In order for SWI to pass mandatory criteria, the evaluation team relied upon other documents in the file, but not in the proposal. Considering that one of the assumptions set by the Department and DND to govern the re-evaluation of proposals was that "all of the bidders are compliant in all of the Mandatory requirements" and the fact that the re-evaluation team should have been aware of the absolute similarity of the two criteria, one mandatory in the RFP and the other a rated requirement in the RFP and the re-evaluation handbook, it was inappropriate for the Department and DND not to provide the re-evaluation team with the information that allowed SWI's proposal to be compliant with the mandatory requirement; the re-evaluation team should have at least given SWI a passing mark for this technical criterion. To act as if the assumption that the mandatory requirement had been passed or was non-existent or empty of meaning, in the Tribunal's opinion, was injurious and prejudicial to the re-evaluation of SWI's proposal with respect to the "depth and experience of staff" criterion used to evaluate the qualifications of key personnel in section 2.0 of Annex "H" of the RFP.

This criterion was assessed using the contents of the résumés of the key personnel proposed, as set out in Table 2 of the Personnel Evaluation Detailed Scoring Tables. The Tribunal notes that, in numerous instances in that table, the said key personnel received nil evaluation points, even though the very same key personnel were assumed to have satisfactorily met the minimum mandatory experience requirements. In the Tribunal's opinion, these key personnel could not reasonably be assumed to meet the minimum mandatory experience requirements and be rated 0 for the same requirement. The Tribunal finds that this materially downgraded the evaluation results achieved by SWI's proposal with respect to section 2.0 of Annex "H" of the RFP. This is a breach of the AIT and, for that reason, the complaint is valid in part.

However, the Tribunal is satisfied, given the structure of the evaluation framework, that this error did not substantively permeate and materially affect the overall evaluation and rating of sections 6.0 and 7.0 of SWI's proposal. Specifically, the Tribunal notes that the staff evaluated, with respect to the latter two sections, was comprised of personnel largely different from that identified as key personnel and represented categories of staff that were not materially affected by the evaluation of the minimum mandatory experience requirement. The Department did not make and, therefore, did not have to take into consideration any assumption relating to the minimum experience of the latter categories of staff in conducting the re-evaluation. Furthermore, the Tribunal observes, by relying on the individual evaluators' notes, that SWI's low scores in sections 6.0 and 7.0 of Annex "H" of the RFP rest fundamentally on the fact that these sections included conflicting statements, missed critical components, lacked details and thoroughness or simply failed to identify, completely or at all, which of the 25 staff members proposed by SWI would be responsible for performing the specific functions set out in sections 6.0 and 7.0 of Annex "H". The Tribunal also notes that SWI has not contradicted in a substantive manner any of these reasons in its submissions to the Tribunal.

With respect to the fourth question raised by SWI concerning the appropriateness of the Department and DND's decision to withhold the Fleetway proposal from the re-evaluation, contrary to the Tribunal's recommendation, the Tribunal will not deal with this question for the reasons stated in response to the first question. Nonetheless, addressing the fairness of the re-evaluation issue inherent in SWI's question, the Tribunal fails to see how SWI's proposal might have been prejudiced at the time of the re-evaluation by the fact that the Fleetway proposal was not re-evaluated separately for technical merits. It is clear from the solicitation documents that the evaluation methodology to be used in assessing proposals was not to be based on a comparative analysis and rating of the proposals received. On the contrary, each proposal was evaluated and re-evaluated on its own merits.

With respect to SWI's fifth question that relates to the unfairness of the approach used to develop the re-evaluation handbook, the Tribunal finds that there is no evidence on the record to support this allegation. The Tribunal is satisfied that the re-evaluation handbook was developed by the evaluators before they became aware of the contents of any of the proposals submitted.

Finally, the Tribunal finds that there is no basis to SWI's allegation that the integrity of the proposal that it submitted might have been compromised by a security breach at DND's premises, thereby prejudicing the re-evaluation of its proposal. The Tribunal is satisfied that the proposal that the Department used in its re-evaluation of the technical merits of SWI's proposal was identical to that submitted by SWI.

In conclusion, the Tribunal finds that SWI's complaint is valid in part. In considering the appropriate remedy in the circumstances, the Tribunal finds that the breaches committed by the Department and DND in conducting the re-evaluation of the technical merits of section 2.0 of SWI's proposal did not have a material impact on the Department and DND's scoring of sections 6.0 and 7.0 of Annex "H" of the RFP. Accordingly, in the Tribunal's view, there is no basis upon which to disturb or reject the scoring of these sections. Because achieving a 60 percent mark for each of the sections of Annex "H" was a mandatory requirement of the RFP and because the Department and DND properly evaluated and scored these two sections of SWI's proposal as failing to meet the mandatory 60 percent threshold, the Tribunal concludes that there is no basis upon which to overturn the overall re-evaluation results of SWI's proposal. Therefore, in the Tribunal's opinion, SWI's proposal was properly declared non-responsive for failing to meet the mandatory 60 percent requirement in respect of sections 6.0 and 7.0 of Annex "H".

The Tribunal has considered the issue of whether to grant MIL/Fleetway reasonable costs incurred in pursuing the matter and has decided not to award such costs. In the Tribunal's view, the submissions made by the Department were determinative in this instance.

DETERMINATION OF THE TRIBUNAL

In light of the foregoing, the Tribunal determines that the procurement was not conducted in accordance with the provisions of the AIT and that, therefore, the complaint is valid in part.

Pursuant to subsection 30.16(1) of the CITT Act, the Tribunal awards SWI its reasonable costs incurred in filing and proceeding with this complaint.

1 . R.S.C. 1985 (4th Supp.), c. 47 [hereinafter CITT Act].

2 . 18 July 1994, C. Gaz. 1995.I.1323, online: Internal Trade Secretariat <http://www.intrasec.mb.ca/eng/it.htm> [hereinafter AIT].

3 MIL Systems, Determination (6 March 2000), Statement of Reasons (21 March 2000) (CITT).

4 . Siemens Westinghouse v. Minister of Public Works and Government Services (23 June 2000), A-195-00 (FCA).

5 . S.O.R./93-602 [hereinafter Regulations].

6 . S.O.R./91-499.

7 . MIL Systems (21 July 2000) (CITT).

8 . Refers to the evaluation handbook used in the first evaluation of the proposals.

9 . Supra note 3 at 19-20.

10 . Supra note 4 at para.11.

11 . This assumption was necessary because the Tribunal recommended in its March 6, 2000, determination that the Department and DND re-evaluate only the technical merits of the proposals.

12 . This assumption was necessary to ensure that there would be no contact between the members of the re-evaluation team and the bidders. The evaluation team's questions and answers were permitted.

13 . According to the re-evaluation handbook, consensus is reached where the spread of the evaluator's individual scores is 20 percent or less.

14 . See protected exhibit 9 of the GIR.

15 . See protected exhibit 9 of the GIR and the listing of evaluation criteria in protected exhibit 8 of the GIR.

16 . DMR Consulting Group (18 September 1997), PR-97-009 (CITT); FPG/HRI Joint Venture (6 June 1996), PR-95-031 (CITT); Canadian Computer Rentals (3 August 2000), PR-2000-003 (CITT); Mirtech International Security (3 June 1997), PR-96-036 (CITT); and Bell Canada (21 February 1997), PR-96-023 (CITT).

17 . Supra note 4.

18 . Symtron Systems (10 September 1997) (CITT).

19 . Article 514(2)(a) of the AIT defines the term "procurement process" for purposes of bid protest procedures as follows: "begins after an entity has decided on its procurement requirement and continues through to the awarding of the contract".

20 . Article 518 of the AIT defines "procurement procedures" as "the processes by which suppliers are invited to submit a tender, a proposal, qualification information, or a response to a request for information and includes the ways in which those tenders, proposals or information are treated".

21 . As found in protected exhibit 8 of the GIR.

---

[ Table of Contents]

---

Initial publication: April 18, 2001