RAYTHEON CANADA LIMITED
v.
DEPARTMENT OF PUBLIC WORKS AND GOVERNMENT SERVICES
File No. PR-2015-026

Determination and reasons issued
Tuesday, January 19, 2016

TABLE OF CONTENTS

• DETERMINATION
• STATEMENT OF REASONS
  • BACKGROUND
  • PROCUREMENT PROCESS
  • TRIBUNAL’S ANALYSIS
    • Ground 1: Unfair Evaluation—the Ground is Unfounded
    • Ground 2: Evaluation of System “availability”—not “performance”—Ground is Unfounded
    • Ground 3: Delays in Announcing Results are not Grounds for Review
    • Ground 4: Integrity Provisions Raised by Raytheon are not Applicable
  • COSTS
  • DETERMINATION OF THE TRIBUNAL

 

IN THE MATTER OF a complaint filed by Raytheon Canada Limited pursuant to subsection 30.11(1) of the Canadian International Trade Tribunal Act, R.S.C., 1985, c. 47 (4th Supp.);

AND FURTHER TO a decision to conduct an inquiry into the complaint pursuant to subsection 30.13(1) of the Canadian International Trade Tribunal Act.

BETWEEN

RAYTHEON CANADA LIMITED Complainant

AND

THE DEPARTMENT OF PUBLIC WORKS AND GOVERNMENT SERVICES Government Institution

DETERMINATION

Pursuant to subsection 30.14(2) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal determines that the complaint is not valid.

Pursuant to section 30.16 of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal awards the Department of Public Works and Government Services its reasonable costs incurred in responding to the complaint, which costs are to be paid by Raytheon Canada Limited. In accordance with the Procurement Costs Guideline, the Canadian International Trade Tribunal’s preliminary indication of the level of complexity for this complaint case is Level 3, and its preliminary indication of the amount of the cost award is $4,700. If any party disagrees with the preliminary indication of the level of complexity or the preliminary indication of the amount of the cost award, it may make submissions to the Canadian International Trade Tribunal, as contemplated by article 4.2 of the Procurement Costs Guideline. The Canadian International Trade Tribunal reserves jurisdiction to establish the final amount of the award.

Peter Burn
Peter Burn
Presiding Member

Tribunal Member: Peter Burn, Presiding Member

Counsel for the Tribunal: Eric Wildhaber
Rebecca Marshall-Pritchard

Complainant: Raytheon Canada Limited

Counsel for the Complainant: Brenda C. Swick
Michael J. Brzezinski
Nathan I. Lean

Government Institution: Department of Public Works and Government Services

Counsel for the Government Institution: Alexandre Kaufman
Susan D. Clarke
Ian McLeod
Roy Chamoun

Please address all communications to:

The Registrar
Canadian International Trade Tribunal Secretariat
333 Laurier Avenue West
15th Floor
Ottawa, Ontario  K1A 0G7

Telephone: 613-993-3595
Fax: 613-990-2439
E-mail: citt-tcce@tribunal.gc.ca

STATEMENT OF REASONS

BACKGROUND

1. On September 9, 2015, Raytheon Canada Limited (Raytheon) filed a complaint with the Canadian International Trade Tribunal (the Tribunal) pursuant to subsection 30.11(1) of the Canadian International Trade Tribunal Act^[1] concerning the procurement (Solicitation No. W8476-112965/B) by the Department of Public Works and Government Services (PWGSC) on behalf of the Department of Defence (DND) for the provision of an Integrated Soldier System Project (ISSP).
2. The ISSP is for the acquisition and support of “. . . over 4000 soldier-wearable communications suites, complete with required accessories, support equipment, contract management, training, logistic and engineering support . . . . The [ISSP] is a suite of military equipment that soldiers wear as part of their combat load. It includes weapon accessories and electronics that allow soldiers to stay connected with their teams after exiting vehicles on the battlefield. It also features a radio, a smartphone-like computer to run battle management software, a GPS, and a communications headset.”^[2]
3. Raytheon alleges that PWGSC:

• did not evaluate its bid fairly (ground 1);
• evaluated the “availability” of its ISSP suite instead of its “performance” (ground 2);
• did not conduct the procurement in a procedurally fair manner, citing the delay in announcing the results (ground 3); and
• failed to apply certain integrity provisions (ground 4).

4. Raytheon requested that the bids be re-evaluated and that the designated contract be terminated. In addition, Raytheon requested that it be compensated for lost profits and reimbursed its complaint costs and bid preparation costs.
5. On September 15, 2015, the Tribunal informed the parties that the complaint had been accepted for inquiry, as it met the requirements of subsection 30.13(1) of the CITT Act and the conditions set out in subsection 7(1) of the Canadian International Trade Tribunal Procurement Inquiry Regulations.^[3]
6. On September 30, 2015, Raytheon filed a motion for the production of focusing questionnaires completed by the evaluation soldiers in Stage 2 of the User Acceptance Performance Evaluation (UAPE). The UAPE was the evaluation process conducted by teams of Canadian Forces infantry soldiers with respect to the operational acceptability of the ISSP suites.
7. On October 2, 2015, the Tribunal instructed PWGSC to produce the focusing questionnaires by October 13, 2015, which it did.
8. On October 9 and 20, 2015, Raytheon requested that Mr. Jimmy J. Jackson be given access to the confidential record for the purposes of preparing an expert report.
9. On October 15, 2015, PWGSC objected to Mr. Jackson being given access to the confidential record.
10. On October 20, 2015, the Tribunal denied Raytheon’s request.
11. On October 23, 2015, PWGSC filed a Government Institution Report (GIR) in accordance with rule 103 of the Canadian International Trade Tribunal Rules.^[4] That same day, Raytheon filed a report from Mr. Jackson.
12. Raytheon filed comments on the GIR on November 4, 2015.
13. On November 16, 2015, PWGSC filed a response to Raytheon’s comments on the GIR, alleging that Raytheon included new arguments that merited a response.
14. On November 25, 2015, Raytheon provided further additional comments in response to PWGSC’s response to Raytheon’s comments on the GIR.
15. Given that there was sufficient information on the record to determine the validity of the complaint, the Tribunal decided that an oral hearing was not required and disposed of the complaint on the basis of the written information on the record.

PROCUREMENT PROCESS

16. The Request for Proposal (RFP) was issued on April 2, 2013.
17. On August 1, 2013, Raytheon submitted its proposal in response to the RFP.
18. On July 27, 2015, Raytheon was informed that its bid was not accepted and that a contract had been awarded to Rheinmetall Canada inc. (Rheinmetall).
19. On August 10, 2015, Raytheon filed its objection with PWGSC.
20. On September 4, 2015, PWGSC responded to Raytheon in writing.
21. On September 9, 2015, Raytheon filed its complaint with the Tribunal.

TRIBUNAL’S ANALYSIS

22. Subsection 30.14(1) of the CITT Act requires that, in conducting an inquiry, the Tribunal limit its considerations to the subject matter of the complaint. At the conclusion of the inquiry, the Tribunal must determine whether the complaint is valid on the basis of whether the procedures and other requirements prescribed in respect of the designated contract have been observed. Section 11 of the Regulations provides that the Tribunal is required to determine whether the procurement was conducted in accordance with the applicable trade agreements, which, in this instance, is the Agreement on Internal Trade.^[5]
23. For the reasons that follow, the Tribunal determines that there is no valid basis for any of the grounds of complaint raised by Raytheon.

Ground 1: Unfair Evaluation—the Ground is Unfounded

24. Raytheon bases its case that the evaluation is unreliable and unfair on a single piece of evidence that purports to demonstrate that the UAPE results do not stand the test of statistical analysis. A report that it brings forward claims that the UAPE results cannot be replicated.^[6] Raytheon essentially contends that something improper must necessarily have gone on, without tendering any evidence of what that something could be, other than theorizing that there must have been “latent defects” that compromised the evaluation. Raytheon admits that “. . . there is insufficient information available to pinpoint precisely what those latent defects are . . . .”^[7]
25. The list of the hypotheses that Raytheon advances as to what could possibly have gone wrong is lengthy: (i) the evaluation must have been arbitrary or imprecise; (ii) the conduct of the UAPE process must have been unfair; (iii) the evaluation soldiers must have lacked the necessary evaluation expertise; (iv) the evaluation must have been ambiguous and inconsistent; (v) the evaluation soldiers must have based their evaluation on undisclosed criteria; and (vi) the evaluation soldiers must have inadvertently applied personally held biases.^[8] PWGSC further broke down the preceding list of 6 arguments into one that climbs to 14 claims.^[9]
26. Before examining PWGSC’s answers to these hypotheses, the Tribunal remarks that Raytheon provides no answer to its lack of substantiating evidence except for the following statement: “The GIR touches on some of [hypotheses (i) to (vi)] . . . and attempts to discredit them on the basis that they are untimely and/or lacking in evidence . . . the unreliable results produced by the UAPE, the late stage at which Raytheon became aware [that] the results were unreliable, and the implications of those unreliable results on the fairness and overall integrity of the UAPE process, is a complete answer to PWGSC’s technical arguments.”^[10]
27. The Tribunal does not agree with Raytheon. Rather, the Tribunal finds that PWGSC provided cogent and complete answers to all of Raytheon’s arguments.^[11] More fundamentally, the Tribunal finds that Raytheon has substituted unsubstantiated allegations for evidence of impropriety and that, in any event, all of its allegations in respect of this ground of complaint are untimely.
28. As correctly underscored by PWGSC, all of Raytheon’s allegations under this ground of complaint refer to aspects of the evaluation that were set out in the solicitation documents and that were, in all instances, also the subject of further discussion or information at various stages leading up to the UAPE.^[12] These include:^[13] the manner in which soldiers would be selected for ISSP evaluation duty for the UAPE^[14] and trained (including for bias control by PWGSC^[15] or trained by bidders themselves for other subject matter);^[16] the manner in which teams would be assigned;^[17] the fact that they would be using surrogate Garmin Rino equipment;^[18] the manner in which the “Human Factor Observers” were to be used;^[19] the weighting of the UAPE mandatory point-rated criteria;^[20] and the mandatory requirement to meet a total UAPE score that was within 20 percent of the highest UAPE score.^[21]
29. It follows that Raytheon knew, or reasonably should have known, about any ground of complaint concerning these matters at some time during bid submission or, at the latest, 10 working days following completion of the UAPE exercises in 2013. Consequently, all of Raytheon’s objections pertaining to this ground of complaint are filed outside of the time frame provided for in section 6 of the Regulations.
30. Additionally, the Tribunal is satisfied with PWGSC’s response to Raytheon’s suggestion that the focusing questionnaires inappropriately affected the UAPE evaluation and, therefore, finds that they did not.^[22]
31. Similarly, the Tribunal affords no credence whatsoever to Raytheon’s allegation of an unfair advantage having been afforded to Rheinmetall by reason of it holding a contract for the provision of software unrelated to the ISSP.^[23] There is simply no demonstrated nexus between one and the other.

Raytheon’s Motion for Disclosure

32. The Tribunal rejects Raytheon’s allegation that it discovered what it termed to be latent defects that can only be fully unveiled through ex-post facto analysis of the UAPE results by an expert. For that reason, the Tribunal denied Raytheon’s request for access to the confidential record by Mr. Jackson. Raytheon’s proposed use of Mr. Jackson was unnecessary for three reasons.
33. First, it is well established that the Tribunal reviews procurement processes against the standard of reasonableness and typically affords a large amount of deference to evaluators.^[24] Raytheon’s proposed use of Mr. Jackson was tantamount to requesting that the Tribunal substitute his extrapolations for the Tribunal’s own judgment, or initiate a “battle of experts” between Raytheon and PWGSC where the Tribunal would have been asked to choose between competing extrapolations. As a general rule, the Tribunal sees no benefit whatsoever in being assisted by experts when ascertaining the reasonableness of an evaluation, particularly not in circumstances, like here, where the purported expertise is exclusively geared at examining the evaluation itself.
34. Second, Raytheon was essentially seeking to have Mr. Jackson substantiate all, or at least some, of the six hypotheses that it advances as the reasons for which it scored as it did on the UAPE. Based on it having failed to establish relevance, Raytheon’s motion, as formulated, did not convince the Tribunal that there was a reasonable possibility that Raytheon could establish any proximate cause or nexus between findings from any further analysis by Mr. Jackson and those hypotheses. In short, Raytheon’s proposed use of Mr. Jackson amounted to nothing other than a thinly veiled fishing expedition for evidence.
35. Third, and most importantly, the Tribunal finds that Raytheon’s proposed use of Mr. Jackson was an improper attempt at justifying a wait-and-see attitude.^[25] All of Raytheon’s allegations on this ground of complaint, and its proposed use of Mr. Jackson, converge on an attempt to discredit the results of the use of the UAPE methodology and, chiefly, its use of Likert scale questionnaires.
36. Mr. Jackson states that Likert “. . . questionnaire scoring [is inherently subjective and] must be evaluated to ascertain whether the results are statistically reliable and thus yield a fair and objective evaluation void of any latent defects.”^[26] However, Mr. Jackson also states that “[t]here is nothing objectionable about PWGSC’s UAPE evaluation design and planning including the use of a subjective Likert scale questionnaire. When properly executed, the overall results of a subjective Likert score evaluation can be objective.”^[27]
37. In this case, the use of the Likert scale questionnaires was announced in the solicitation documents. Those documents did not provide for an evaluation of the use of the Likert scoring results (essentially an evaluation of the evaluation). It was incumbent upon Raytheon to review the published evaluation methodology when it was published, perhaps seeking Mr. Jackson’s views at that time regarding its “proper execution”.
38. Had Raytheon wanted to object to the use of the published evaluation methodology, and request that the evaluation methodology comprise a statistical analysis of the Likert scoring results, it should have done so long ago.^[28] In short, Mr. Jackson’s views may have been germane at that time. They are no longer. Raytheon’s objection to the evaluation methodology was not timely pursuant to section 6 of the Regulations.
39. For the reasons given above, the motion was therefore denied.

Unsubstantiated Accusations of Bias Should not Have Been be Made

40. The Tribunal is deeply concerned with Raytheon’s accusations of bias in the absence of evidence.
41. The Tribunal is incapable of finding either the existence of actual bias or any reasonable apprehension of bias in the conduct of the UAPE. Accusations of this nature should not be brought lightly. As the British Columbia Court of Appeal (per Gibbs J.A.) stated in Adams v. British Columbia (Workers’ Compensation Board):^[29]

13 This case is an exemplification of what appears to have become general and common practice, that of accusing persons vested with the authority to decide rights of parties of bias or reasonable apprehension of it without any extrinsic evidence to support the allegation. It is a practice which, in my opinion, is to be discouraged. An accusation of that nature in an adverse imputation on the integrity of the person against whom it is made. The sting and the doubt about integrity lingers even when the allegation is rejected. It is the kind of allegation easily made but impossible to refute except by general denial. It ought not to be made unless supported by sufficient evidence to demonstrate that, to a reasonable person, there is a sound basis for apprehending that the person against whom it is made will not bring an impartial mind to bear upon the cause. As I have said earlier, and on other occasions, suspicion is not enough.

42. In the same manner as Gibbs J.A. very pointedly remarked, even though the accusations of bias made by Raytheon against some 39 soldiers and various other personnel involved in the UAPE were entirely unfounded, they most certainly cast aspersions on their work and personal integrity. We do not know how this has individually affected them, but it is shameful that they were targeted in the first place.
43. Raytheon is represented in this case by a team of lawyers that is led by a senior member of the Procurement Bar. The Tribunal finds it regrettable that she could not have persuaded her client to formulate this aspect of its case differently, if at all, given the absence of extrinsic evidence supporting the allegation.
44. Good faith must be presumed, and unfounded accusations discouraged. Because of the nature of the work of public servants and members of the Canadian Forces, and their commitment to Canada, until there are serious and real grounds to believe otherwise, the Tribunal would hope that they are all treated with respect.

Ground 2: Evaluation of System “availability”—not “performance”—Ground is Unfounded

45. Clause 3.4.3 of Appendix 2 to Annex CB to Volume 1 in the RFP provides as follows:

3.4.3 Phase 3 – UAPE

. . . 

3.4.3.5 The Objective of the UAPE is for users to evaluate the performance of each system and not the availability of the bid systems. As such the protocols listed below will be used to compensate for bid systems down time; and

3.4.3.6 Bidders will be required to have a technician readily available to repair their [ISSP suites], as required, during scheduled UAPE stands. The Crown is under no obligation to telephone or search for technicians.

[Emphasis added]

46. Phase 3 of the UAPE comprised the following three stages: Stage 1—Bidder Led Training Stand; Stage 2—Bidder Assisted Test Stands; and Stage 3—Dynamic Test Stand. There is no evaluation or testing at Stage 1, only training.
47. At Stage 3, the evaluation soldiers completed the final scoring questionnaire. Some evaluation soldiers completed additional comment sheets that contained appraisals of the so-called “availability” of Raytheon’s ISSP suite as follows:██████████████████████████████████████████████████████████████████
    █████████████████████████████.^[30]
48. Raytheon argues that clause 3.4.3.5 required the evaluation of the performance of its ISSP suite only when it was available. Raytheon thereby equates “available” with “operational” (i.e. not when on down time). The Tribunal disagrees.
49. Raytheon’s position on this ground of complaint does not consider the terms of the RFP in their entirety. Its position begins by failing to consider the second sentence of clause 3.4.3.5, which refers the reader to “. . . the protocols listed below . . .” which are to “. . . be used to compensate for bid systems down time . . . .” The “protocols listed below” are found in clause 3.4.3.6, which require a bidder to have its “. . . technician readily available to repair [its ISSP suite], as required, during scheduled UAPE stands.”^[31] Only Stages 1 and 2 of the UAPE have “scheduled UAPE stands” that provide for the availability of bidder technicians to repair ISSP suites that may encounter down time.
50. Stage 3 does not provide for the presence of bidder technicians whatsoever. In fact, the entirety of the testing done at Stage 3 is to be conducted “. . . without support from the Bidders (this includes all logistics, mentoring and training)”^[32] [underlining in original] and in a context where the evaluation soldiers “. . . will have to troubleshoot and adapt to issues that arise with the [ISSP suites].”^[33]
51. As such, clauses 3.4.3.5 and 3.4.3.6, read together, provide for protocols to overcome down time during the evaluations at Stage 2, but not during Stage 3.
52. Fundamentally, Raytheon is stating that its ISSP suite could not be evaluated until any and all shortcomings dealing with “availability” had been ironed out. Raytheon’s position on this ground must be rejected, as that is not an evaluation, but testing and development for market readiness.
53. The RFP did provide that the evaluations made at the “Bidder Assisted” test stand of Stage 2 would allow for a bidder’s technician to intervene to attempt to remedy any availability issue. However, Stage 2 was also to take place during a specified period of time. To adopt Raytheon’s position would be to agree that, despite the time restrictions on the duration of the UAPE,^[34] PWGSC could not evaluate (or eliminate) a given ISSP suite until the suite was able to pass a Stage 2 evaluation without encountering any “availability issue”. This would mean “never” should a problem prove to be unfixable—an absurd reading of the RFP that the Tribunal cannot adopt.
54. Additionally, as indicated above, the RFP provided for no bidder intervention whatsoever at Stage 3. The evaluation, at this stage, is a 24-hour simulated combat mission geared at ascertaining whether ISSP suites would “work in an operational scenario that incorporates common infantry battle tasks in order to assist the soldiers in making an informed assessment on the level of acceptability of the [ISSP suites] for use on operations.”^[35] Stage 3 provides for no protocol to compensate for ISSP suite down time.
55. Time is of the essence in combat—simulated or real. During combat, the only protocol or course of action in the event of system availability-related down time would be to ditch a non-operational ISSP suite, and do whatever soldiers are trained to otherwise do when equipment does not perform as expected on the battlefield.
56. The RFP provided for a professional assessment of bidders’ ISSP suites by front-line evaluation soldiers.^[36] Evaluation soldiers knew that they were choosing new gear for the missions of tomorrow, where lives would inevitably be on the line – perhaps their own. The Tribunal trusts that they discharged their duty with nothing other than the utmost attentiveness and that they made an honest, straightforward and professional assessment evaluation. The Tribunal was presented with no evidence whatsoever to the contrary and, therefore, believes that the evaluation soldiers committed no reviewable error.
57. This ground of complaint is not valid.

Ground 3: Delays in Announcing Results are not Grounds for Review

58. Raytheon claims that the solicitation was conducted in a procedurally unfair manner because the results of the procurement process were not announced for some 18 months following bid evaluation and that this caused it a financial prejudice.
59. The Tribunal finds that there was nothing procedurally unfair so as to attract the discipline of the trade agreements because of the time that PWGSC took to announce the results of the solicitation. No time frame for award was specified in the RFP, which also provided that PWGSC could have cancelled the solicitation altogether.^[37]
60. This ground of complaint is not valid. If Raytheon has recourse on this ground, it is not before the Tribunal.

Ground 4: Integrity Provisions Raised by Raytheon are not Applicable

61. Raytheon claims that PWGSC failed to apply certain integrity provisions.^[38]
62. The Tribunal notes that federal integrity provisions have changed several times in the last few years as a matter of evolving public policy^[39] but that they remain outside any legislative or regulatory framework. Accordingly, the Tribunal is therefore limited to considering only the applicable law of Contract A^[40] between the parties.
63. By amendment dated July 16, 2013, PWGSC set out the contractually applicable integrity provisions of the solicitation.^[41] Those provisions are not the ones referenced by Raytheon in its complaint.^[42] The latter are therefore not the applicable law of Contract A in this matter.
64. The integrity provisions set out in the amendment of July 16, 2013, that are the applicable contractual law between the bidders and PWGSC in regard to the solicitation do not contain the disciplines that Raytheon is seeking to have applied to Rheinmettal. In particular, the scope of the applicable integrity provisions is restricted to various situations of discipline arising out of Canadian legislation only.^[43]
65. This ground of complaint is not valid.

COSTS

66. The Tribunal awards PWGSC its reasonable costs incurred in responding to the complaint.
67. In determining the amount of the cost award for this complaint case, the Tribunal considered its Procurement Costs Guideline (the Guideline), which contemplates classification of the level of complexity of cases on the basis of three criteria: the complexity of the procurement, the complexity of the complaint and the complexity of the complaint proceedings.
68. The Tribunal’s preliminary indication is that this complaint case has a complexity level corresponding to the highest level of complexity referred to in Annex A of the Guideline (Level 3). The complexity of the procurement was high, as it involved the provision of a specific highly technical piece of equipment. The Tribunal finds that the complexity of the complaint was high, as the issues were not straightforward and dealt with whether PWGSC properly evaluated Raytheon’s proposed system against a sophisticated UAPE process. Finally, the complexity of the proceedings was high, although the issues were resolved by the parties through documentary evidence and written representations, and a hearing was not necessary, there were a number of procedural matters that arose in advance of the Tribunal’s determination.
69. Accordingly, as contemplated by the Guideline, the Tribunal’s preliminary indication of the amount of the cost award is $4,700.

DETERMINATION OF THE TRIBUNAL

70. Pursuant to subsection 30.14(2) of the CITT Act, the Tribunal determines that the complaint is not valid.
71. Pursuant to section 30.16 of the CITT Act, the Tribunal awards PWGSC its reasonable costs incurred in responding to the complaint, which costs are to be paid by Raytheon. In accordance with the Guideline, the Tribunal’s preliminary indication of the level of complexity for this complaint case is Level 3, and its preliminary indication of the amount of the cost award is $4,700. If any party disagrees with the preliminary indication of the level of complexity or the preliminary indication of the amount of the cost award, it may make submissions to the Tribunal, as contemplated by article 4.2 of the Guideline. The Tribunal reserves jurisdiction to establish the final amount of the award.