NORTHROP GRUMMAN OVERSEAS SERVICES CORPORATION

Determinations

NORTHROP GRUMMAN OVERSEAS SERVICES CORPORATION
v.
DEPARTMENT OF PUBLIC WORKS AND GOVERNMENT SERVICES
File No. PR-2007-008

Determination issued
Thursday, August 30, 2007

Reasons issued
Wednesday, September 12, 2007

TABLE OF CONTENTS

DETERMINATION OF THE TRIBUNAL
STATEMENT OF REASONS

IN THE MATTER OF a complaint filed by Northrop Grumman Overseas Services Corporation under subsection 30.11(1) of the Canadian International Trade Tribunal Act, R.S.C. 1985 (4th Supp.), c. 47;

AND FURTHER TO a decision to conduct an inquiry into the complaint under subsection 30.13(1) of the Canadian International Trade Tribunal Act.

BETWEEN
NORTHROP GRUMMAN OVERSEAS SERVICES CORPORATION	Complainant
AND
THE DEPARTMENT OF PUBLIC WORKS AND GOVERNMENT SERVICES	Government Institution

DETERMINATION OF THE TRIBUNAL

Pursuant to subsection 30.14(2) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal determines that the complaint is valid in part.

Pursuant to subsections 30.15(2) and (3) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal recommends, as a remedy, that the Department of Public Works and Government Services, within 30 days of the publication of this determination, re-evaluate the proposals with respect to rated criteria R13 and R44 of the Request for Proposal for all three bidders. The Canadian International Trade Tribunal further recommends that, in accordance with the Evaluation Plan: (1) regarding rated criterion R13, the Department of Public Works and Government Services permit the use of “root mean square” and any other technically supportable definitions of the term “error”; and (2) regarding rated criterion R44, the Department of Public Works and Government Services award points to only those stores that were cleared or certified prior to bid closing and for which adequate documentation was provided. If this re-evaluation results in the identification of a different winning bidder for the contract, the existing contract should be cancelled and awarded to that bidder.

Pursuant to section 30.16 of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal awards Northrop Grumman Overseas Services Corporation its reasonable costs incurred in preparing and proceeding with the complaint, which costs are to be paid by the Department of Public Works and Government Services. The Canadian International Trade Tribunal’s preliminary indication of the level of complexity for this complaint case is Level 2, and its preliminary indication of the amount of the cost award is $2,400. If any party disagrees with the preliminary indication of the level of complexity or the preliminary indication of the amount of the cost award, it may make submissions to the Canadian International Trade Tribunal, as contemplated by the Guideline for Fixing Costs in Procurement Complaint Proceedings. The Canadian International Trade Tribunal reserves jurisdiction to establish the final amount of the award.

Ellen Fry
Ellen Fry
Presiding Member

Pierre Gosselin
Pierre Gosselin
Member

Serge Fréchette
Serge Fréchette
Member

Hélène Nadeau
Hélène Nadeau
Secretary

The statement of reasons will be issued at a later date.

Tribunal Members:	Ellen Fry, Presiding Member
	Pierre Gosselin, Member
	Serge Fréchette, Member

Director:	Randolph W. Heggart

Senior Investigator:	Michael W. Morden

Counsel for the Tribunal:	Reagan Walker
	Nick Covelli

Complainant:	Northrop Grumman Overseas Services Corporation

Counsel for the complainant:	Barbara A. McIsaac, Q.C.
	R. Benjamin Mills

Interveners:	Lockheed Martin Corporation
	Raytheon Company

Counsel for Lockheed Martin Corporation:	Richard A. Wagner

Counsel for Raytheon Company:	Ronald D. Lunau

Government Institution:	Department of Public Works and Government Services

Counsel for the Government Institution:	David M. Attwater
	Susan D. Clarke
	Christianne M. Laizner
	Ian McLeod

Please address all communications to:

The Secretary
Canadian International Trade Tribunal
Standard Life Centre
333 Laurier Avenue West
15th Floor
Ottawa, Ontario
K1A 0G7

Telephone: 613-993-3595
Fax: 613-990-2439
E-mail:

STATEMENT OF REASONS

COMPLAINT

1. On April 17, 2007, Northrop Grumman Overseas Services Corporation (Northrop) filed a complaint with the Canadian International Trade Tribunal (the Tribunal) pursuant to subsection 30.11(1) of the Canadian International Trade Tribunal Act. ¹ The complaint concerned the procurement by the Department of Public Works and Government Services (PWGSC) (Solicitation No. W8475-02BA01/C), on behalf of the Department of National Defence (DND), for the initial supply of 36 advanced multi-role infrared sensor (AMIRS) targeting pods for DND’s CF-18s and 13 years of in-service support for the pods.

2. Northrop alleged that the evaluation of the bids was not done in accordance with the provisions of the published Evaluation Plan. Northrop claimed that its proposal did not receive the points to which it was entitled for rated criteria R13 and R39. Northrop also alleged that the proposal of the winning bidder, Lockheed Martin Corporation (Lockheed), received points for rated criterion R44 to which it was not entitled. As a remedy, Northrop requested that the contract be terminated, that the bids be re-evaluated and that the contract be awarded to Northrop if it was found to have submitted the winning bid. In the alternative, should the Tribunal not recommend that the contract be terminated, Northrop requested that it be recognized as the bidder that should have won the contract and that it be compensated for its lost profits. In the further alternative, Northrop requested that it be awarded its bid preparation costs. It also requested its costs for preparing and proceeding with the complaint.

3. On April 25, 2007, the Tribunal informed the parties that it had accepted the complaint, as it met the requirements of subsection 30.13(1) of the CITT Act and the conditions set out in subsection 7(1) of the Canadian International Trade Tribunal Procurement Inquiry Regulations. ² On May 1 and 21, 2007, respectively, Lockheed and Raytheon Company (Raytheon) requested intervener status in these proceedings. The Tribunal granted both requests. On July 3, 2007, PWGSC filed the Government Institution Report (GIR). On July 13, 2007, Northrop and Lockheed submitted their comments on the GIR.³ On July 19, 2007, PWGSC requested that it be allowed to submit additional comments, arguing that Northrop had introduced new arguments in its comments on the GIR. On July 23, 2007, Northrop filed its final comments.

4. Given that there was sufficient information on the record to determine the validity of the complaint, the Tribunal decided that a hearing was not required and disposed of the complaint on the basis of the information on the record.

5. The procurement in issue is subject to the obligations contained in the Agreement on Internal Trade ⁴ only.

PROCUREMENT PROCESS

6. The procurement at issue is for the initial supply of 36 AMIRS targeting pods for CF-18s and 13 years of in-service support (spares, equipment and training) for the pods. The procurement process has been underway since November 2002, when a Letter of Interest (LOI) was issued on MERX.⁵ At that time, four companies expressed an interest, three of which, Lockheed, Northrop and Raytheon, provided information regarding their products to DND. A second LOI was issued in November 2005 advising that a draft Request for Proposal (RFP) was available for review and comment. According to PWGSC, based on the comments received from the three remaining potential bidders, “. . . numerous changes were made to the RFP with a view to keeping the three main contractors . . . compliant . . . .”⁶ The RFP was released on May 16, 2006, and the bidding period closed on September 15, 2006, with three bidders submitting proposals.

7. According to PWGSC, between October 2 and November 3, 2006, the bids were evaluated and a review of the evaluations was undertaken. On November 10, 2006, PWGSC calculated the total points for the three compliant proposals and determined that, based on the evaluation methodology prescribed in the RFP, Lockheed’s proposal provided the best value. On March 22, 2007, PWGSC awarded the contract to Lockheed.

8. According to Northrop, it received an e-mail on March 22, 2007, advising it that the contract had been awarded. On March 30, 2007, Northrop received a debriefing from PWGSC. Northrop filed its complaint with the Tribunal on April 17, 2007.

PRELIMINARY MATTER—TRIBUNAL’S JURISDICTION

9. A preliminary question arose in this case as to whether Northrop had standing to make a complaint under section 30.11 of the CITT Act, alleging a violation of Article 506(6) of the AIT.

10. In its letter to PWGSC dated April 25, 2007, the Tribunal requested that PWGSC provide a submission regarding the Tribunal’s jurisdiction, given that the only applicable trade agreement was the AIT and that Northrop is a company apparently based in the United States. On May 2, 2007, PWGSC provided the Tribunal with its comments and brought a motion requesting an order of the Tribunal dismissing the complaint as, according to PWGSC, Northrop had no standing to file such a complaint under the AIT. On May 7, 2007, Northrop filed its comments on PWGSC’s submission. On May 9, 2007, PWGSC filed its final comments on the matter. On June 8, 2007, the Tribunal determined that it did have jurisdiction to conduct an inquiry and issued an order dismissing PWGSC’s motion.

11. The Tribunal is a creature of statute. Sections 30.1 to 30.19 of the CITT Act establish the basis upon which rests the Tribunal’s jurisdiction in respect of procurement complaints. These provisions create a dispute resolution mechanism for the procurement provisions of various trade agreements, including those of the AIT, and incorporate the rights and obligations created under these agreements into Canadian law.

12. The basic approach to statutory interpretation, as approved by the Supreme Court of Canada, was articulated in the following words: “. . . The words of an Act are to be read in their entire context and in their grammatical and ordinary sense harmoniously with the scheme of the Act, the object of the Act, and the intention of Parliament . . . .” ⁷

13. Subsection 30.11(1) of the CITT Act sets out the general parameters for the initiation of a procurement complaint and states the following:

Subject to the regulations, a potential supplier may file a complaint with the Tribunal concerning any aspect of the procurement process that relates to a designated contract and request the Tribunal to conduct an inquiry into the complaint.

14. “Potential supplier” and “designated contract” are key terms in this provision. They are defined in section 30.1 of the CITT Act as follows:

“potential supplier” means, subject to any regulations made under paragraph 40(f.1), a bidder or prospective bidder on a designated contract;

“designated contract” means a contract for the supply of goods or services that has been or is proposed to be awarded by a government institution and that is designated or of a class of contracts designated by the regulations.

15. As discussed below, the term “potential supplier” identifies the persons who have standing to initiate a procurement complaint under the dispute mechanism. The term “designated contract” identifies the subject matter in respect of which a complaint may be initiated.

16. Subsection 30.13(1) of the CITT Act provides as follows: “[s]ubject to the regulations, . . . the Tribunal . . . shall decide whether to conduct an inquiry into the complaint . . .” after it determines that a complaint complies with certain technical requirements listed in subsection 30.11(2). Among the requirements that must be satisfied is the necessity to identify the complainant and the designated contract concerned by the complaint.

17. As contemplated by the CITT Act, the Regulations set more precise parameters for the exercise of the Tribunal’s jurisdiction.

18. Subsections 3(1) and 7(1) of the Regulations are fundamental in circumscribing the Tribunal’s jurisdiction at the time of initiating an inquiry under subsection 30.11(1) of the CITT Act. Subsection 3(1) of the Regulations provides as follows:

For the purposes of the definition “designated contract” in section 30.1 of the Act, any contract or class of contract concerning a procurement of goods or services or any combination of goods or services, as described in . . . Article 502 of the Agreement on Internal Trade, . . . is a designated contract.

19. Article 502 of the AIT essentially limits the coverage of the procurement chapter of the AIT to procurement over specified dollar values and excludes certain procuring entities from coverage.

20. Subsection 7(1) of the Regulations sets further limitations on the Tribunal’s jurisdiction. It prescribes three conditions that must be met in order for the Tribunal to decide whether to conduct an inquiry. If these conditions are met, the Tribunal has the discretion to decide whether to conduct an inquiry. Subsection 7(1) provides as follows:

The Tribunal shall, within five working days after the day on which a complaint is filed, determine whether the following conditions are met in respect of the complaint:

(a) the complainant is a potential supplier;

(b) the complaint is in respect of a designated contract; and

(c) the information provided by the complainant, and any other information examined by the Tribunal in respect of the complaint, discloses a reasonable indication that the procurement has not been carried out in accordance with . . . Chapter Five of the Agreement on Internal Trade . . . .

21. Since Northrop is a bidder on the procurement in question, it will satisfy the definition of “potential supplier” in section 30.1 of the CITT Act if the contract is a “designated contract”. Because the procurement in question comes within the description in Article 502 of the AIT, it follows that it is a “designated contract” for the purpose of section 30.1 of the CITT Act. Therefore, Northrop is a “potential supplier”.

22. Nothing in the wording of the definition of “potential supplier” or “designated contract”, per se, imposes a nationality requirement on a complainant. However, the Tribunal must also ask itself whether there is any other provision that imposes such a requirement, either directly or implicitly, in light of the context, object and purpose of the legislation.

23. Paragraph 7(1)(c) of the Regulations imposes as a condition for the Tribunal’s jurisdiction that there be a reasonable indication of a violation of Chapter Five of the AIT. Accordingly, if there is any provision that imposes a nationality requirement on the relevant provision of Chapter Five, the Tribunal will not have jurisdiction unless, among other things, there is a reasonable indication that the nationality requirement has been fulfilled.

24. PWGSC appears to suggest that, since the definition of “designated contract” refers, in this case, to a contract described in Article 502 of the AIT, it implies necessarily that the Tribunal can only consider a complaint by a Canadian supplier. The essence of PWGSC’s argument appears to be that Articles 101(3) and 501 limit the Tribunal’s jurisdiction concerning Chapter Five to considering only complaints by Canadian suppliers.

25. Article 101(3) states as follows:

In the application of this Agreement, the Parties shall be guided by the following principles:

(a) Parties will not establish new barriers to internal trade and will facilitate the cross-boundary movement of persons, goods, services and investments within Canada;

(b) Parties will treat persons, goods, services and investments equally, irrespective of where they originate in Canada;

(c) Parties will reconcile relevant standards and regulatory measures to provide for the free movement of persons, goods, services and investments within Canada; and

(d) Parties will ensure that their administrative policies operate to provide for the free movement of persons, goods, services and investments within Canada.

26. Article 501 of the AIT states as follows:

. . . the purpose of this Chapter is to establish a framework that will ensure equal access to procurement for all Canadian suppliers in order to contribute to a reduction in purchasing costs and the development of a strong economy in a context of transparency and efficiency.

27. It is clear from the wording of both Articles 101(3) and 501 of the AIT that they do not constitute substantive provisions. They merely constitute an expression of what the parties to the AIT wanted to achieve and must be considered in that context.

28. Article 101(3) of the AIT contains principles that provide guidance in the application of the AIT. An analysis of the four principles enunciated within the article does not indicate if there is any intention to limit the ability for a private party—whether Canadian or non-Canadian—to pursue rights and obligations created under the AIT within the context of the dispute settlement provisions established thereunder.

29. Similarly, Article 501 of the AIT is silent on whether the parties, in pursuing the goal of equal access for all Canadian suppliers, intend to bar non-Canadian suppliers from the procedural protections created under the AIT.

30. Indeed, where the AIT discriminates in favour of Canadian suppliers, it does so on an exceptional and tightly controlled basis, as discussed below. Implementing the four principles in Article 101(3) of the AIT does not necessarily require that non-Canadian parties be barred from access to the dispute settlement provisions. Likewise, achieving the objective of equal access for all Canadian suppliers, as provided by Article 501, does not necessarily imply that the rights of non-Canadian suppliers need to be restricted. On the contrary, from an economic perspective, it can be argued that limiting access to Canadian suppliers could in fact be detrimental to the objectives of achieving “. . . a reduction in purchasing costs and the development of a strong economy in a context of transparency and efficiency.”

31. The Tribunal also considered the substantive provisions of Chapter Five of the AIT as a whole to determine whether the overall context of the chapter indicates an intention to limit its coverage to Canadian suppliers. In the Tribunal’s view, as discussed below, the substantive provisions of Chapter Five make it clear that the parties to the AIT did not intend to limit its coverage in this way, given that certain substantive provisions of Chapter Five clearly intend rights to apply to non-Canadian suppliers.

32. Article 504 of the AIT sets out a number of basic disciplines in respect of non-discriminatory treatment.

33. Article 504(6) of the AIT indicates that the parties to the AIT did not want, a priori, to limit the rights resulting from Chapter Five to Canadian suppliers only. The parties have taken a more carefully crafted approach in this article.

34. Article 504(6) states as follows:

Except as otherwise required to comply with international obligations, a Party may limit its tendering to Canadian goods or suppliers subject to the following conditions:

(a) the procuring Party must be satisfied that there is sufficient competition among Canadian suppliers;

(b) all qualified suppliers must be informed through the call for tenders of the existence of the preference and the rules applicable to determine Canadian content; and

(c) the requirement for Canadian content must be no greater than necessary to qualify the procured good as a Canadian good.

35. This provision clearly imposes a requirement for positive action on a party that wishes its procurement to be limited to Canadian suppliers. To impose such a limitation, the party must specify that the procurement opportunity is so limited and must also fulfill the requirement in Article 504(6)(c) of the AIT concerning the required level of Canadian content. These provisions indicate that, instead of imposing a blanket up-front limitation on non-Canadian suppliers, parties intended any limitation to be based on a specific analysis of the competitive situation.

36. If Chapter Five of the AIT were intended to offer protection only to Canadian suppliers rather than to all suppliers, there would be no need for Article 504(6), since there would be no need to limit the situations when tendering is confined to Canadian suppliers. Instead, Article 504(6) provides for two distinct levels of protection for suppliers: a basic level of protection that applies to all suppliers and a higher level of protection that only applies to Canadian suppliers.

37. Similarly, Article 506(2) of the AIT offers different levels of protection for Canadian suppliers and other suppliers in calls for tender. Article 506(2)(c) protects all suppliers by permitting the following:

(i) registration on the source list is consistent with Article 504;

(ii) all registered suppliers in a given category are invited to respond to all calls for tenders in that category; and

(iii) a supplier that meets the conditions for registration on the source list is able to register at any time.

38. Canadian suppliers receive additional protection under Article 506(2) of the AIT concerning access to the tendering information:

A call for tenders shall be made through one or more of the following methods:

(a) the use of an electronic tendering system that is equally accessible to all Canadian suppliers;

(b) publication in one or more predetermined daily newspapers that are easily accessible to all Canadian suppliers . . . .

39. Furthermore, looking at Chapter Five of the AIT as a whole, “supplier” and “Canadian supplier” are used as distinct terms throughout the chapter. Indeed, in Articles 504 and 506, both terms are used within the same article.

40. Article 518 of the AIT defines the two terms as follows:

supplier means a person who, based on an assessment of that person’s financial, technical and commercial capacity, is capable of fulfilling the requirements of a procurement . . .

Canadian supplier means a supplier that has a place of business in Canada.

41. Thus, the definition of “supplier” is more generic than that of “Canadian supplier”. It contains no association with a Canadian place of business or any other characteristic associated with Canadian nationality.

42. It is a well-known rule of interpretation that, when different words are used by the drafters, they are intended to mean different things.⁸ In the Tribunal’s view, the use of two distinct terms in Chapter Five of the AIT indicates that the parties intended to make a distinction between “Canadian suppliers” and all “suppliers”, rather than reading in “Canadian” wherever the term “supplier” appears.

43. Having concluded that there is no general requirement to limit rights under Chapter Five of the AIT to Canadian suppliers, the Tribunal needs to consider whether Article 506(6), the particular provision that is the basis of the complaint, has any provision that is intended to limit rights to Canadian suppliers.

44. Northrop alleged that the evaluation of bids was not carried out in accordance with the published Evaluation Plan, contrary to Article 506 of the AIT, which provides as follows:

1. Each Party shall ensure that procurement covered by this Chapter is conducted in accordance with the procedures set out in this Article.

. . .

6. . . . The tender documents shall clearly identify the requirements of the procurement, the criteria that will be used in the evaluation of bids and the methods of weighting and evaluating the criteria.

45. Article 518 of the AIT defines “bid”, “call for tenders” and “tender” in the following manner:

bid means a submission in response to a call for tenders

call for tenders means a call for competitive bids from suppliers . . .

tender means a response to a call for tenders.

[Emphasis added]

46. It is clear, when considering Article 506(6) of the AIT in light of the above-quoted definitions of “supplier” and “Canadian supplier”, that the parties to the AIT did not intend to limit application of the substantive right created under the provision to Canadian suppliers only. Article 506(6) refers to “suppliers” and not “Canadian suppliers” and, hence, does not impose a requirement for a complainant to be a Canadian supplier. Accordingly, the Tribunal concludes that Article 506(6) is intended to provide protection to all suppliers.

47. Based on the foregoing analysis, it is the Tribunal’s view that the complainant does not need to be a “Canadian supplier” in order for the Tribunal to have jurisdiction to inquire into a complaint under Article 506(6) of the AIT. Before reaching its decision, the Tribunal carefully reviewed its previous decisions concerning the preliminary matter under consideration, noting that the submissions of the parties addressed this issue in more detail than was the case in most or all of the Tribunal’s previous decisions.⁹ The Tribunal is mindful that its decision in the present case appears to be inconsistent with its previous decisions on the same issue. However, to be convinced that the position advanced by PWGSC was correct, the Tribunal would have needed to conclude from the wording, context, object and purpose of the legislative scheme that the parties to the AIT intended to limit the application of the dispute settlement mechanism to Canadian suppliers only. As explained in its analysis above, and after careful consideration of the arguments presented to the Tribunal by the parties, the Tribunal could not find any indication of such intention and, as a result, could only conclude in the manner in which it did.

TRIBUNAL’S ANALYSIS

48. Subsection 30.14(1) of the CITT Act requires that, in conducting an inquiry, the Tribunal limit its considerations to the subject matter of the complaint. Moreover, at the conclusion of the inquiry, the Tribunal must determine whether the complaint is valid on the basis of whether the procedures and other requirements prescribed in respect of the designated contract have been observed. Section 11 of the Regulations further provides that the Tribunal is required to determine whether the procurement was conducted in accordance with the applicable trade agreements, which, in this instance, is the AIT.

49. The relevant part of Article 506(6) of the AIT reads as follows:

. . . The tender documents shall clearly identify the requirements of the procurement, the criteria that will be used in the evaluation of bids and the methods of weighting and evaluating the criteria.

50. As it has stated in the past, the Tribunal is of the view that, unless the evaluators have not applied themselves in evaluating a bidder’s proposal, have ignored vital information provided in a bid, have wrongly interpreted the scope of a requirement or have based their evaluation on undisclosed criteria, the Tribunal will generally not substitute its judgement for that of the evaluators.¹⁰

51. Section 5.2.2.2 of the Evaluation Plan, found in part 6 of the RFP, imposed the following responsibility on bidders:

It is the bidder’s responsibility to have provided sufficient data to prove compliance and demonstrate capability for all Rated Requirements. Evaluation will be based solely on the data provided by the Bidder in his RFP response.

Rated Criterion R44

52. Rated criterion R44 of the RFP reads as follows:

The AMIRS should not limit the Carriage, Employment, or Jettison (CEJ) of stores or expendables from any carriage location on CF-18, including station #3, to below the limits of the CF-18 equipped with [a specified targeting system].

53. Section 5.2.11 of the Evaluation Plan applies to rated criterion R44 and states as follows:

. . .

5.2.11.1 General

The [Canadian Forces] desires that the CEJ of stores from weapons station #3 of a CF-18 aircraft (ECP-583 Modified) while carrying an AMIRS on weapon station #4 be no more restrictive than the current restrictions imposed by the carriage of the [specified targeting system]. . . . The Bidders should have provided all Test reports or Analysis reports to fully prove their claim. If a store was cleared/certified by analysis model, then an explanation should be provided on how the model was validated.

. . .

5.2.11.2 Scoring

5.2.11.2.1 The Bidder should demonstrate compatibility of all stores detailed in the CEJ worksheet of the RMM with station #3 of an F/A-18 A/B/C/D aircraft, with their pod mounted on station #4, [in accordance with] MIL-HDBK 1763 for the Carriage, Employment, and Jettison of every store. Scores for Carriage, Employment, and Jettison shall be awarded as follows:

5.2.11.2.1.1 Carriage ¹¹

5.2.11.2.1.1.1 For every store in the CEJ worksheet of the RMM that has been cleared or certified for carriage on Station #3 of an F/A-l8 A/B/C/D aircraft, with the AMIRS pod mounted on Station #4, to the stated flight regime detailed in the CEJ worksheet of the RMM and where the Bidder has provided the necessary Test Reports or Analysis Reports documentation to prove his claim, the particular store shall be awarded 6 points;

5.2.11.2.1.1.2 For every store in the CEJ worksheet of the RMM that has been cleared or certified for carriage on Station #3 of an F/A-l8 A/B/C/D aircraft, with the AMIRS pod mounted on Station #4; however, to less than the stated flight regime detailed in the CEJ worksheet of the RMM and where the Bidder has provided the necessary Test Reports or Analyses Reports documentation to prove his claim, the particular store shall be awarded either 2 or 4 points, depending on the level of completeness; and

5.2.11.2.1.1.3 For every store in the CEJ worksheet of the RMM that has not been cleared or certified for carriage on Station #3 of an F/A-18 A/B/C/D aircraft, with the AMIRS pod mounted on Station #4, the particular store shall be awarded a score of 0 points. Similarly, where the Bidder has not provided the necessary test reports or analyses to prove his claim, the particular store shall be awarded a score of 0 points.

NOTE

5.2.11.2.1.1.4 Where analysis or test reports documentation cannot be provided because of releasibility issues, the Bidder should have provided at least a summary description of the analysis or test report, along with the report title, number, and date, in order to be awarded the appropriate score.

. . .

5.2.11.2.1.3.5 The method of compliance can be either Inspection, Analysis, Similarity, or Test (Test can be either wind tunnel or flight test).

54. This criterion was also addressed in amendment No. 12, dated September 1, 2006. It reads in part as follows:

[Question] 35. Will the DND adopt a method to differentiate between analytical and flight test as means to substantiate CEJ certification due to the different levels of associated risks?

Answer: As background, during the construct of the evaluation plan, careful consideration was given to the scoring of all rated requirements. In regards to CEJ certification, our view then as now, is that it may be possible for some stores to be cleared purely by analysis and if any such analysis is accepted by the evaluator for that store as being sufficient, then there would be no reason not to award the allocated points for that store. On the other hand, if there is some doubt regarding the analysis vis-à-vis the stated flight regime requirements, then the evaluator would only award the partial points allocated for the particular store. As Canadian Forces DND Subject Matter Experts will be involved in the CEJ evaluation, I have every confidence that the scoring will be fair in all respects.

55. Northrop submitted that simply providing test and analysis reports, without clearance or certification at the time of bid submittal, is not responsive to rated criterion R44. Northrop submitted that Lockheed was clearly awarded points on the basis that its stores could be cleared or certified, not that they were already cleared or certified. It claimed that its complaint was never that the proof of existing clearance or certification came from a “government authority”, as alleged by PWGSC, but rather that Lockheed did not provide any clearance or certification in its bid. It also submitted that the RFP indicates that the Government intended to carry out integration and certification activities with respect to the pod itself, as per rated criterion R45, but that the same cannot be said for the issues of the stores, which was the focus of R44.

56. PWGSC submitted that the evaluators interpreted the terms “cleared” and “certified” to mean that the various weapons mounted on station 3 of the CF-18 had been proven to be airworthy for the CEJ of the stated flight regime while the bidder’s pod was on station 4 of the CF-18. It submitted that nowhere in the RFP, or elsewhere, does rated criterion R44 require proof of an existing clearance or certification from a government authority. It noted that both Lockheed and Northrop had, as part of their proposals, submitted analysis and test reports providing recommendations to substantiate their self-scoring.¹² PWGSC noted that Northrop alleged, in its complaint, that its AMIRS pod had been certified by a government authority prior to bid closing, but PWGSC could find no such certification in Northrop’s proposal.

57. Lockheed submitted that its proposal complied with the requirements of rated criterion R44. It submitted that its AMIRS pod had accumulated tens of thousands of hours of flying time and did not pose any greater risk than any other of the bidders’ products. It submitted that the Evaluation Plan provided for various methods by which a bidder could demonstrate that its AMIRS pod did not limit the CEJ of stores. Lockheed noted that it had included the requisite analysis and test reports with its proposal. Lockheed also argued that there was no requirement in the RFP that certification or clearance be obtained from a government authority, and it noted that, according to the GIR, Northrop did not provide such certification. It submitted that it appears that Northrop is attempting to add an additional requirement to rated criterion R44 that did not exist in the RFP.

58. In the Tribunal’s view, the wording of the detailed scoring methodology found in section 5.2.11.2 of the Evaluation Plan makes it clear that, for a bidder’s proposal to receive points from the evaluators, clearance or certification was required prior to bid submission and that the documentation supporting the claim was to accompany the proposal. The Tribunal notes that answer 35 of amendment No. 12 indicated that “. . . it may be possible for some stores to be cleared purely by analysis . . .”, but it believes that this statement merely indicates that analysis may be acceptable as substantiation for certification. It does not indicate that the analysis is an acceptable alternative to certification. Because the Tribunal is of the opinion that the RFP required clearance or certification at the time of bid closing, the Tribunal finds that PWGSC failed to evaluate the bids according to the RFP and, hence, failed to observe the requirement of Article 506(6) of the AIT.

Rated Criterion R13

59. Rated criterion R13 reads as follows:


Req #	REQUIREMENTS	Scoring Method
IR/Laser Boresight
R13	The sensors/Lasers boresight error should be less than 250 microrad/axis	<250; (2 pts) <=215; (4 pts) <=180; (6 pts) <=145; (8 pts) <=110; (10 pts)

60. According to PWGSC, the evaluators determined the scoring by using maximum error values¹³ found in the proposals as the indication of the worst-case boresight performance. Northrop, on the other hand, submitted that a “root mean square” (RMS) process was a more appropriate measure of error. Northrop awarded itself eight points based on its RMS process results, whereas PWGSC awarded Northrop fewer points based on the maximum error model.

61. PWGSC argued that, by definition, the RMS method only provides that 84 percent of the measurement values are within 250 microrad/axis, thus meaning that 16 percent of the measurement values would exceed the threshold and contradict the requirement that the boresight error be less than the stated level. Due to the difficulty in assessing this criterion, as well as the divergent scores from the three evaluators assigned to this criterion, PWGSC engaged an independent engineer who determined that the supporting documentation submitted by Northrop used RMS to substantiate its self-evaluation. PWGSC submitted that evaluators have an obligation to consider an entire proposal for evaluation purposes and, as the proposal did not include maximum error values, the evaluators were required to calculate these values using the available information found in Northrop’s proposal. PWGSC submitted that it did not rely on Northrop’s statement of compliance, as represented by its self-score, and that it used sound engineering calculations to make a proper assessment of Northrop’s proposal.

62. Lockheed supported PWGSC’s submission regarding this element of the complaint.

63. Northrop submitted that PWGSC “read in” the requirement to use the maximum error method based on an assessment of an unidentified engineer. It submitted that the evaluators, having determined that Lockheed’s proposal used the maximum error method, then attempted to apply this same method to Northrop’s proposal. Northrop claims that it used the RMS process because no product of this type is able to achieve a maximum error value under all potential conditions due to the virtually unlimited and unpredictable operational impacts, environments and scenarios that can be experienced by systems deployed on military aircraft. Northrop claimed that this demonstrated that PWGSC introduced previously unstated criteria into the evaluation process, the results of which favoured Lockheed over the other bidders.

64. The Tribunal notes that the RFP does not state whether the term “error” means a maximum error value, as used by the evaluators, or an RMS error value, as used by Northrop. On the one hand, PWGSC claims that the RMS method provides for values that exceed the maximum allowable error value of 250 microrad/axis. Northrop claims, on the other hand, that the maximum error method is impossible to use, given the virtually unlimited circumstances under which the AMIRS pods are to operate. The Tribunal also notes that one of the evaluators initially assigned a score that matched Northrop’s self-score, presumably indicating that this evaluator did not consider that the maximum error method was required. In addition, although the ETL’s Collective Evaluation Synopsis¹⁴ indicates that “[t]he specification called for absolute error values . . .”, the discussion in the synopsis suggests that the evaluators’ position was not necessarily that 100 percent compliance (the maximum error method) was required, but merely that Northrop’s bid had too great a deviation from 100 percent compliance. The Tribunal believes that, had it been determined by the evaluation team that the maximum error method was required, the synopsis would have stated that the evaluator who initially adopted Northrop’s self-score had made an error in not applying the maximum error method and would not have allowed for the possibility of less than 100 percent compliance.

65. Given the foregoing indications that the technical evaluators did not consider the RFP to impose a requirement for the maximum error method, the Tribunal finds that the RFP can reasonably be interpreted in the manner in which Northrop did and that PWGSC applied a requirement not imposed by the RFP. Accordingly, the RFP failed to clearly identify the requirements of the procurement, as required by Article 506(6) of the AIT.

Rated Criterion R39

66. Rated criterion R39 of the RFP reads as follows:


Req #	REQUIREMENTS	Scoring Method
MLVS Interface Requirements
R39	The AMIRS software should be loadable via the MLVS [Memory Loader Verifier System] through the standard connection located in the CF-18 Nose Wheel Well (NWW) through the MC [mission computer] to the AMIRS.	If yes; (10 pts) If no; (0 pts)

67. PWGSC submitted that section 5.2.2.2 of the Evaluation Plan required that bidders “. . . prove compliance and demonstrate capability . . .” with regard to the rated requirements. In this case, it was a pass/fail criterion—if the bidder could prove that its AMIRS software was loadable as required, it received 10 points; if not, it received 0 points. According to PWGSC, Northrop’s compliance statement, as found in its self-evaluation, spoke of a future capability, as opposed to a current one, i.e. Northrop’s proposal stated that the “. . . software will be loadable . . . .” PWGSC submitted that the evaluators were consistent and awarded at least one other bidder 0 points for its commitment to integrate the MLVS at a later date.

68. Northrop submitted that, in its self-evaluation, it unequivocally stated the following about its product “. . . will be loadable via the MLVS via the NWW through the MC to AMIRS” and, for that, it should have received full points. It submitted that all the evaluators ignored this statement and appeared to search for some other statement of proof in other parts of its proposal. It submitted that the use of the verb tense of its response (“will be”) was dictated by the form of the requirement specified by PWGSC (the software “should be”).

69. Lockheed supported PWGSC’s submission regarding this particular element of the complaint. It also submitted that, if the Tribunal found that Northrop’s response to rated criterion R39 should be re-evaluated, its response to rated criterion R39 should also be re-evaluated.

70. The Tribunal notes that bidders were required by section 5.2.2.2 of the Evaluation Plan, “. . . to have provided sufficient data to prove compliance and demonstrate capability for all Rated Requirements . . . .” The bare compliance statement provided by Northrop does not indicate whether its software was loadable via the NWW at the time of bid closing, or if this would only be the case in the future. Northrop did not draw to the Tribunal’s attention any information in its bid that addressed these deficiencies. The Tribunal notes that, in conducting their evaluation, the evaluators could only locate information that appeared to indicate that the requirement would only be met in the future.¹⁵

71. Accordingly, the Tribunal finds that PWGSC did not act unreasonably in concluding that Northrop did not provide sufficient data to prove its compliance with this requirement. The Tribunal therefore finds this ground of complaint not to be valid.

Remedy

72. Having found the complaint to be valid in part, the Tribunal must now recommend a suitable means of redressing the harm that the deficiencies in the evaluation process caused to Northrop.

73. In this connection, the Tribunal is governed by subsection 30.15(3) of the CITT Act, which reads as follows:

(3) The Tribunal shall, in recommending an appropriate remedy under subsection (2), consider all the circumstances relevant to the procurement of the goods or services to which the designated contract relates, including

(a) the seriousness of any deficiency in the procurement process found by the Tribunal;

(b) the degree to which the complainant and all other interested parties were prejudiced;

(d) whether the parties acted in good faith; and

(e) the extent to which the contract was performed.

74. The Tribunal considers that there is a serious deficiency in the procurement process when a supplier’s proposal is not evaluated in accordance with the terms outlined in the RFP. In this case, the Tribunal finds that PWGSC’s actions potentially prejudiced all interested parties, to the extent that the contract may have been awarded to a company that was not the highest-ranked bidder, in accordance with the terms of the RFP upon which the bidders based their proposals. This type of behaviour also has the potential to make bidders reluctant to participate in future solicitations. The Tribunal finds that PWGSC’s behaviour has therefore compromised the integrity and efficiency of the procurement system as a whole. There is no evidence, however, that PWGSC was not acting in good faith, and the Tribunal notes that PWGSC applied or mis-applied itself, as the case may be, in a consistent manner during the evaluation, i.e. each bidder was affected in the same way. There is no indication that PWGSC favoured one bidder or evaluated one bidder differently from the others. Given that the contract has only recently been signed and that there is no evidence that deliveries have started, the Tribunal believes that re-evaluating the affected portions of all proposals offers the most reasonable remedy.

75. The Tribunal agrees with PWGSC’s submission in the GIR, in which it stated that “. . . the best recourse is to recommend a re-evaluation of competing proposals . . . .” The Tribunal notes that PWGSC applied a consistent approach in evaluating rated criteria R13 and R44 for all bidders and, consequently, its errors in evaluation may have affected all bidders’ scores. This, in turn, could have led to the award of the contract to the wrong bidder, and it is not clear which of the three bidders would have been the winning bidder if the three proposals had been evaluated correctly. The Tribunal therefore recommends that PWGSC, within 30 days of the publication of this determination, re-evaluate those portions of the proposals relating to rated criteria R13 and R44, for all three bidders, in accordance with the following directions:

• Rated criterion R13—that PWGSC allow the use of RMS and any other technically supportable definitions of the term “error”;

• Rated criterion R44—that PWGSC award points as specified in section 5.2.11.2 to only those stores that were cleared or certified prior to bid closing and for which adequate documentation was provided, in accordance with the wording of the Evaluation Plan.

76. The Tribunal awards Northrop its reasonable costs incurred in preparing and proceeding with the complaint. The Tribunal has considered its Guideline for Fixing Costs in Procurement Complaint Proceedings (the Guideline) and is of the view that this complaint case has a complexity level corresponding to the medium level of complexity referred to in Appendix A of the Guideline (Level 2). The Guideline contemplates classification of the level of complexity of complaint cases based on three criteria: the complexity of the procurement; the complexity of the complaint; and the complexity of the complaint proceedings. The complexity of the procurement was high in that it involved the procurement of a complex item and included elements of installation and maintenance. The complexity of the complaint was medium in that it involved ambiguous specifications. Finally, the complexity of the complaint proceedings was medium, as there was one motion, two interveners and no public hearing, a 135-day time frame was required, and the parties were given permission to file information beyond the normal scope of proceedings. Accordingly, as contemplated by the Guideline, the Tribunal’s preliminary indication of the amount of the cost award is $2,400.

DETERMINATION OF THE TRIBUNAL

77. Pursuant to subsection 30.14(2) of the CITT Act, the Tribunal determines that the complaint is valid in part.

78. Pursuant to subsections 30.15(2) and (3) of the CITT Act, the Tribunal recommends, as a remedy, that PWGSC, within 30 days of the publication of this determination, re-evaluate the proposals with respect to rated criteria R13 and R44 of the RFP for all three bidders. The Tribunal further recommends that, in accordance with the Evaluation Plan: (1) regarding rated criterion R13, PWGSC permit the use of RMS and any other technically supportable definitions of the term “error”; and (2) regarding rated criterion R44, PWGSC award points to only those stores that were cleared or certified prior to bid closing and for which adequate documentation was provided. If this re-evaluation results in the identification of a different winning bidder for the contract, the existing contract should be cancelled and awarded to that bidder.

79. Pursuant to section 30.16 of the CITT Act, the Tribunal awards Northrop its reasonable costs incurred in preparing and proceeding with the complaint, which costs are to be paid by PWGSC. The Tribunal’s preliminary indication of the level of complexity for this complaint case is Level 2, and its preliminary indication of the amount of the cost award is $2,400. If any party disagrees with the preliminary indication of the level of complexity or the preliminary indication of the amount of the cost award, it may make submissions to the Tribunal, as contemplated by the Guideline. The Tribunal reserves jurisdiction to establish the final amount of the award.

¹ . R.S.C. 1985 (4th Supp.), c. 47 [CITT Act].

² . S.O.R./93-602 [Regulations].

³ . Raytheon did not make any submissions during the inquiry.

⁴ . 18 July 1994, C. Gaz. 1995.I.1323, online: Internal Trade Secretariat <http://www.ait-aci.ca/index_en/ait.htm> [AIT].

⁵ . Canada’s electronic tendering service.

⁶ . GIR at 2.

⁷ . Rizzo & Rizzo Shoes Ltd. (Re), [1998] 1 S.C.R. 27 at 41.

⁸ . Sullivan, Ruth, 4th ed. (Markham, Ont.: Butterworths, 2002) at 64.

⁹ . Re Complaint Filed by Eurodata Support Services Inc. (30 July 2001), PR-2000-078 (CITT); Re Complaint Filed by Winchester Division—Olin Corporation (2 April 2004), PR-2003-064 (CITT); Re Complaint Filed by EFJohnson (26 April 2006), PR-2006-006 (CITT); Re Complaint Filed by Computer Label Worldwide Co. Ltd. (22 August 2006), PR-2006-023 (CITT); Re Complaint Filed by Europe Displays, Inc. (16 January 2007), PR-2006-039 (CITT).

¹⁰ . Re Complaint Filed by K-W Leather Products Ltd. (3 September 2002), PR-2002-012 (CITT).

¹¹ . The same wording was used for “Employment” and “Jettison” but with different scoring scales. For brevity, it has not been reproduced here.

¹² . As part of the evaluation scheme, bidders were provided with the Evaluation Plan and required to self-score their own proposals. For the formal evaluation, PWGSC submitted that three evaluators were used for each of the criteria in question. Individual evaluators’ scores, with substantiations, were submitted to an Evaluation Team Leader (ETL). If the individual evaluators’ scores were fewer than two points apart, the scores were averaged together to obtain a consensus score for that rated criterion. If the scores were more than two points apart, the ETL brought the evaluators together, allowed them to express their opinions and confirm that the scoring methodology was fair and consistent and then had them re-perform the evaluation. PWGSC also submitted that, in the case where there was either a significant score divergence and/or difficult initial or suspect scoring, the ETL carried out a due diligence review of that criterion’s scoring.

¹³ . The maximum error method involves placing an upper limit on the statistical variability of the measurements of the boresight error such that 100 percent of all measurement values do not exceed 250 microrad/axis.

¹⁴ . GIR, Confidential Exhibit 10.

¹⁵ . Ibid.

Collection	Procurement Inquiries
Publication Date	2007-10-25
Case Number(s)	PR-2007-008
Case Type	Determination
Notes	Archived Content Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.

Procurement Inquiries

Decision Information