VAISALA OYJ
v.
DEPARTMENT OF PUBLIC WORKS AND GOVERNMENT SERVICES
File No. PR-2017-022

Determination and reasons issued
Friday, December 29, 2017

TABLE OF CONTENTS

• DETERMINATION
• STATEMENT OF REASONS
  • INTRODUCTION
  • SUMMARY OF COMPLAINT
  • PROCUREMENT PROCESS
  • PROCEEDINGS OF THE COMPLAINT
  • RELEVANT PROVISIONS OF THE RFP
  • ANALYSIS
    • Ground 1: Technical Specifications of the RFP
    • Ground 2: Conditions for Participation in the RFP
    • Ground 3: Bid Evaluation in Compliance with the Mandatory Criteria of the RFP
  • COSTS
  • DETERMINATION
• APPENDIX: RELEVANT RFP PROVISIONS

IN THE MATTER OF a complaint filed by Vaisala Oyj pursuant to subsection 30.11(1) of the Canadian International Trade Tribunal Act, R.S.C., 1985, c. 47 (4th Supp.);

AND FURTHER TO a decision to conduct an inquiry into the complaint pursuant to subsection 30.13(1) of the Canadian International Trade Tribunal Act.

BETWEEN

VAISALA OYJ - Complainant

AND

THE DEPARTMENT OF PUBLIC WORKS AND GOVERNMENT SERVICES  - Government Institution

DETERMINATION

Pursuant to subsection 30.14(2) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal determines that the complaint is not valid.

Pursuant to section 30.16 of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal awards the Department of Public Works and Government Services its reasonable costs incurred in responding to the complaint, which costs are to be paid by Vaisala Oyj. In accordance with the Procurement Costs Guideline, the Canadian International Trade Tribunal’s preliminary indication of the level of complexity for this complaint case is Level 2, and its preliminary indication of the amount of the cost award is $2,750. If any party disagrees with the preliminary indication of the level of complexity or the preliminary indication of the amount of the cost award, it may make submissions to the Canadian International Trade Tribunal, as contemplated by article 4.2 of the Procurement Costs Guideline. The Canadian International Trade Tribunal reserves jurisdiction to establish the final amount of the award.

Peter Burn -
Peter Burn
Presiding Member

Tribunal Panel: - Peter Burn, Presiding Member

Support Staff: - Laura Little, Counsel

Michael Carfagnini, Student-at-law

Complainant: - Vaisala Oyj

Counsel for the Complainant: - Graham Ragan

Phuong T.V. Ngo

Julie Chung

Marc Crandall

Government Institution: - Department of Public Works and Government Services

Counsel for the Government Institution: - Ian McLeod

Kathryn Hamill

Roy Chamoun

Please address all communications to:

The Registrar
Secretariat to the Canadian International Trade Tribunal
333 Laurier Avenue West
15th Floor
Ottawa, Ontario  K1A 0G7

Telephone: 613-993-3595
Fax: 613-990-2439
E-mail: citt-tcce@tribunal.gc.ca

STATEMENT OF REASONS

INTRODUCTION

1. On August 16, 2017, Vaisala Oyj (Vaisala) filed a complaint with the Canadian International Trade Tribunal (the Tribunal), under subsection 30.11(1) of the Canadian International Trade Tribunal Act,^[1] concerning a Request for Proposals (RFP) (Solicitation No. K3D33-170264/A) issued by the Department of Public Works and Government Services (PWGSC)^[2] for the provision of radiosondes^[3] systems on behalf of the Department of the Environment (ECCC).
2. On August 17, 2017, the Tribunal decided to conduct an inquiry into the complaint, having determined that it met the requirements of subsection 30.13(1) of the CITT Act and the conditions set out in subsection 7(1) of the Canadian International Trade Tribunal Procurement Inquiry Regulations.^[4]
3. For the following reasons, the Tribunal finds that the complaint is not valid.

SUMMARY OF COMPLAINT

4. Vaisala alleged that PWGSC erred in awarding the contract in the above-noted solicitation to a competing bidder, GRAW Radiosondes GmbH & Co. (GRAW). In particular, Vaisala claimed that the ECCC Technical Authority for this solicitation conducted an unreasonable, incomplete and inadequate evaluation of GRAW’s bid on the following grounds: (1) by implementing technical specifications and a proof of performance (POP) test that were not based on existing international standards; (2) by failing to assess whether GRAW satisfied the conditions for participation under the RFP; and (3) by awarding the contract to a tender that at the time of opening could not have satisfied certain technical mandatory and point-rated criteria of the RFP.^[5] In addition, Vaisala submitted that the POP testing was not properly conducted by the ECCC Technical Authority. Accordingly, Vaisala submitted that GRAW’s bid should not have been found to be the highest-ranked responsive bid, which resulted in PWGSC’s decision to award the contract to GRAW.
5. As a remedy, Vaisala requested that the contract awarded to GRAW be terminated and that all compliant bids be re-evaluated in accordance with the terms of the RFP. In the event that such re-evaluation results in Vaisala’s bid receiving the highest score, Vaisala asked that it be awarded the contract and compensated for lost opportunity in the amount of the profit that it would reasonably have made during the time that GRAW held the current contract.

PROCUREMENT PROCESS

6. The RFP was published on December 19, 2016, with a bid closing date of January 30, 2017.
7. On January 18, 2017, PWGSC issued Amendment No. 001 to the RFP, extending the bid closing date to February 6, 2017. Amendment No. 001 also included 24 answers to questions from suppliers, certain of which (including Nos. 6, 12, 13, 15, 20 and 21) touched upon provisions of the RFP relevant to this complaint, namely, Annex D, Section 1, mandatory criteria M7, M10, M13, and Annex D, Section 3, Proof of Performance (POP) Testing.
8. On January 20, 2017, PWGSC issued Amendment No. 002 to correct certain clerical errors in Amendment No. 001. Amendment No. 002 is not relevant to this complaint.
9. On February 6, 2017, the solicitation closed. Two bids were received, one from Vaisala and one from GRAW. The PWGSC Contracting Authority separated and retained the Financial Bid portion of the bids and sent the Technical Bid documents to the ECCC Technical Authority for evaluation.
10. In order to be declared responsive to the RFP, a bid had to comply with all requirements of the bid solicitation, meet all mandatory criteria and pass the POP testing.^[6] A contract was to be awarded to the responsive bid with the highest combined rating of technical merit and price.
11. As both Vaisala’s and GRAW’s bids were found to be compliant with the technical mandatory criteria of the RFP, they were next evaluated against the point-rated criteria. Vaisala’s bid received a technical score of 120 out of 120 points.^[7]
12. Although GRAW’s bid received a lower technical score, it obtained the highest combined rating of technical merit and price.^[8] Accordingly, on March 21, 2017, the PWGSC Contracting Authority requested that GRAW deliver and install equipment for POP testing.
13. The ECCC Technical Authority conducted the POP testing between April 25, 2017, and May 4, 2017. It then submitted a summary report to the PWGSC Contracting Authority, with a recommendation that a contract be awarded to GRAW.
14. On May 18, 2017, PWGSC notified Vaisala that the contract was awarded to GRAW and that Vaisala’s bid, while responsive to the RFP requirements, did not achieve the highest ranking.
15. On May 29, 2017, a debrief meeting occurred, following which Vaisala made an objection to PWGSC. Its objection letter, dated June 5, 2017, requested the cancellation of the contract awarded to GRAW and a re-evaluation of the technical proposals, including POP retesting.
16. On August 2, 2017, PWGSC replied to Vaisala’s objection and denied the relief sought.

PROCEEDINGS OF THE COMPLAINT

17. On August 16, 2017, Vaisala filed its complaint with the Tribunal. The complaint was accompanied by the affidavit of Mr. Larry Miloshevich, whom Vaisala proposed as an independent expert witness with expertise in the fields of meteorological instrumentation, measurement of atmospheric parameters and industry practice in the area of these instruments.^[9]
18. The complaint was accepted for inquiry on August 17, 2017. The Tribunal later granted a request by PWGSC for an extension of time to file the Government Institution Report (GIR). By doing so, the proceedings were moved to a 135-day calendar pursuant to section 12(c) of the Regulations.
19. On September 29, 2017, PWGSC filed public and protected versions of its GIR with the Tribunal in accordance with section 103 of the Canadian International Trade Tribunal Rules.^[10] The GIR was accompanied by the affidavit (confidential) of Mr. Jeffery Anderson, who led the evaluation team for the ECCC Technical Authority.
20. On October 6, 2017, Vaisala requested additional time to file its comments on the GIR. In addition, it requested the disclosure to Mr. Miloshevich of certain confidential information contained in the GIR (particularly Mr. Anderson’s affidavit). Mr. Miloshevich is not a resident of Canada. Vaisala submitted that it required the assistance and input of Mr. Miloshevich in order to be able to comment on the confidential portion of the GIR because it contained highly technical material.
21. On October 12, 2017, the Tribunal granted Vaisala’s request for an extension of time to file its comments on the GIR (to which PWGSC did not object), pursuant to subsection 104(3) of the Rules.
22. On October 13, 2017, PWGSC indicated that it did not object in principle to Mr. Miloshevich having access to the confidential version of the GIR, subject to the use of appropriate procedures for granting such access to a non-resident. On October 18, 2017, the Tribunal informed counsel for the parties that, in its view, it was appropriate for them to make arrangements between themselves with respect to the disclosure matter, noting that the confidential information in question was filed with the GIR and relied on by PWGSC. The Tribunal further indicated that “[i]f the party filing the evidence or arguments refuses to provide such disclosure then, in accordance with the principles of procedural fairness, this refusal may affect the weight that the Tribunal will give to those submissions and evidence.”^[11]
23. On October 23, 2017, Vaisala notified the Tribunal that the parties had reached an agreement regarding the disclosure matter.
24. On November 1, 2017, in accordance with section 104 of the Rules, Vaisala filed public and protected versions of its comments on the GIR with the Tribunal.
25. By that stage of the proceedings, the Tribunal was satisfied that the information on the record was sufficient for it to make its determination on the merits of the complaint. In particular, the Tribunal saw no need for expert opinion to assist it in ascertaining whether the evaluation of GRAW’s technical proposal was reasonable, which is the applicable standard of review in these proceedings.^[12] As such, the Tribunal considered it unnecessary to qualify Mr. Miloshevich as an expert for the purpose of this inquiry.
26. Nevertheless, the Tribunal found Mr. Miloshevich’s affidavit relevant and helpful insofar as it provided factual (as opposed to opinion) evidence on the technical nature of the radiosondes being procured, as well as information on the international radiosonde market, related international studies, and standard testing methodologies for radiosondes systems.
27. The Tribunal further notes that while both parties filed affidavit evidence in support of their arguments, neither side sought to cross-examine the witnesses on the contents of their respective affidavits.
28. In light of the above, the Tribunal decided that an oral hearing was not necessary and ruled on the complaint on the basis of the written record.

RELEVANT PROVISIONS OF THE RFP^[13]

29. Part 4 of the RFP (as amended) is titled “Evaluation Procedures and Basis of Selection”. Part 4, Article 4.1.2 is titled “Technical Evaluation”, and provides that the mandatory and point-rated technical evaluation criteria are set out in Annex D of the RFP.
30. Annex D, Article 1 sets out the 27 mandatory criteria for bid evaluation, five of which (M7, M10, M11, M12 and M13) are the subject of this complaint. Article 1 provides that bidders who fail to meet all mandatory requirements will be considered non-responsive and will not be evaluated further, and that bids meeting all mandatory criteria will go on to be evaluated for point-rated technical criteria.
31. Annex D sets out the above-noted mandatory criteria, as well as 12 point-rated criteria, each of which was worth 10 possible points for a total possible score of 120 points. Four of these point-rated criteria are the subject of this complaint, namely, R2, R3, R6 and R7.
32. Article 4.1.4 of the RFP is titled “Proof of Performance Testing”. Article 4.1.4 provides that the responsive bid with the highest combined rating of technical merit and price will be required to deliver and install equipment for POP testing. Article 4.1.4 further provides that, should this bid not pass the POP testing, the bid with the next highest combined rating will be contacted to submit equipment for POP testing, and so on until a bid meeting the above requirements also passes POP testing.
33. Article 4.2 of the RFP is titled “Basis of Selection – Highest Combined Rating of Technical Merit and Price”. Paragraph 1 provides that, in order to be declared responsive, a bid must comply with all the requirements of the bid solicitation, meet all mandatory criteria and pass the POP testing. It further provides that bids not meeting those requirements will be declared non-responsive. Article 4.2 sets out the method of calculating the total combined rating of technical merit and price, with technical merit worth 60% and price worth 40%. Finally, Article 4.2 paragraph 7 states that the responsive bid with the highest combined rating will be recommended for award of the contract.

ANALYSIS

34. Subsection 30.14(1) of the CITT Act requires that, in conducting an inquiry, the Tribunal limit its considerations to the subject matter of the complaint. At the conclusion of the inquiry, the Tribunal must determine whether the complaint is valid on the basis of whether the procedures and other requirements prescribed in respect of the designated contract have been observed.
35. Section 11 of the Regulations provides that the Tribunal is required to determine whether the procurement was conducted in accordance with the applicable trade agreements, which in this case are the World Trade Organization Revised Agreement on Government Procurement,^[14] the North American Free Trade Agreement^[15] and the Agreement on Internal Trade.^[16] As Vaisala is a Finnish company, and Finland is not a party to either the AIT or NAFTA, Vaisala rightly based its complaint only on the provisions of the AGP.
36. Vaisala alleged that PWGSC/ECCC breached the terms of the AGP in three ways, namely:
    1. by implementing technical specifications and a POP test that were not based on international standards, where such standards exist, contrary to Article X(2)(b) of the AGP (the “Technical Specifications of the RFP” ground of complaint);
    2. by failing to assess whether GRAW satisfied the conditions for participation in the solicitation by evaluating GRAW’s commercial and technical abilities on the basis of their business activities both inside and outside Canada, and by failing to base the evaluation of GRAW’s ability to participate on the conditions specified in the notices or tender documentation, contrary to Articles VIII(3)(a) and (b) of the AGP (the “Conditions for Participation in the RFP” ground of complaint); and
    3. by failing to award the contract to a tender that at the time of opening complied with the essential requirements set out in the notices and tender documentation and was from a supplier that satisfied the conditions for participation, contrary to Article XV(4) of the AGP (the “Bid Evaluation in Compliance with the Mandatory Criteria of the RFP” ground of complaint).
37. The Tribunal will address each of the above allegations, in turn.

Ground 1: Technical Specifications of the RFP

38. Article X(2) of the AGP deals with how technical specifications are set out in the tender documentation. It provides as follows:

2. In prescribing the technical specifications for the goods or services being procured, a procuring entity shall, where appropriate:

1. set out the technical specification in terms of performance and functional requirements, rather than design or descriptive characteristics; and
2. base the technical specification on international standards, where such exist; otherwise, on national technical regulations, recognized national standards or building codes.

[Emphasis added]

39. While the complaint referred to a breach of Article X(2)(b), Vaisala did not challenge the manner in which the technical specifications were set out in the RFP, but rather how they were “implemented”^[17] and “interpreted” in the evaluation of GRAW’s bid. Specifically, Vaisala alleged that “the technical specifications as outlined in the RFP were interpreted in a manner that allowed for the arbitrary and unfair evaluation of the GRAW bid, resulting in PWGSC accepting a non-compliant bid” [emphasis added].^[18] In the Tribunal’s view, the crux of this allegation relates to the conduct of the evaluation of GRAW’s technical proposal, which is captured by Vaisala’s third ground of complaint and Article XV(4), as opposed to Article X(2)(b).
40. Even if Vaisala had sought to challenge the manner in which the technical specifications were set out in the RFP (for example, to argue that they should have been based on international standards), the Tribunal notes that this ground of complaint would be time-barred. Pursuant to section 6 of the Regulations, Vaisala had 10 working days from when it knew or reasonably should have known about this ground of complaint to make an objection to PWGSC (or file a complaint directly with the Tribunal). Given that Vaisala’s objection letter to PWGSC of June 5, 2017, did not raise the manner in which technical specifications were set out in the RFP (which was last amended on January 20, 2017, and closed on February 6, 2017), it cannot be considered to have filed a timely objection in this regard.^[19]
41. The Tribunal therefore finds this ground of complaint not valid.

Ground 2: Conditions for Participation in the RFP

42. Vaisala alleged that PWGSC’s evaluation of GRAW’s bid failed to respect Article VIII(3) of the AGP. Article VIII(3) requires that:

3. In assessing whether a supplier satisfies the conditions for participation, a procuring entity:

1. shall evaluate the financial capacity and the commercial and technical abilities of a supplier on the basis of that supplier’s business activities both inside and outside the territory of the Party of the procuring entity; and
2. shall base its evaluation on the conditions that the procuring entity has specified in advance in notices or tender documentation.

43. Article VIII sets out rules on the use of “conditions for participation” by a procuring entity in a procurement process. This provision allows for a procuring entity to establish preconditions that a supplier must satisfy in order to participate in a procurement. Only conditions which are essential to ensure the legal and financial capacities and commercial and technical abilities of the supplier, and which are required to undertake the procurement, are permitted.
44. After raising this ground of complaint, Vaisala did not provide any further explanation or evidence to support it in these proceedings. The complaint itself does not explain the basis for Vaisala’s allegation that conditions for participation in the RFP were improperly assessed by PWGSC, nor was this addressed in its comments on the GIR.
45. The Tribunal also notes that Vaisala’s objection letter to PWGSC of June 5, 2017, did not include any mention of PWGSC’s assessment of the conditions for participation.^[20]
46. A “condition for participation” is a distinct concept from, and should not be conflated with, the mandatory technical evaluation criteria set out in the RFP. Once again, the crux of Vaisala’s allegation is that the ECCC Technical Authority improperly evaluated GRAW’s bid against the mandatory technical criteria in the RFP, which is covered by the third ground of complaint, as discussed below.
47. The Tribunal therefore finds the second ground of complaint not valid.

Ground 3: Bid Evaluation in Compliance with the Mandatory Criteria of the RFP

48. As indicated above, the RFP expressly provided that only bids that met all technical mandatory criteria would proceed to an evaluation against the point-rated technical criteria. The bidder with the highest combined rating of technical merit and price would proceed to POP testing, which it had to pass in order to be declared responsive and recommended for award of the contract.
49. Vaisala alleged that the ECCC Technical Authority erred in determining that GRAW’s bid demonstrated that its radiosonde product, the “GRAW DFM-09”, met the technical mandatory criteria of the RFP. In particular, Vaisala submitted that the GRAW DFM-09 did not meet M7, M10, M11, M12 and M13. Next, Vaisala argued that even if GRAW’s bid had met the mandatory criteria, it should not have received full points in the evaluation of the GRAW DFM-09 against technical rated criteria R2, R3, R6 and R7. Lastly, Vaisala submitted that the POP testing conducted by the ECCC Technical Authority did not follow the provisions set out in the RFP. As a result, Vaisala submitted that PWGSC erred in concluding that GRAW’s bid was the highest-ranked responsive bid and, on this basis, awarding it the contract.
50. According to Vaisala, the above allegations constituted a breach of Article XV(4) of the AGP, which provides as follows:

4. To be considered for an award, a tender shall be submitted in writing and shall, at the time of opening, comply with the essential requirements set out in the notices and tender documentation and be from a supplier that satisfies the conditions for participation.

[Emphasis added]

51. The Tribunal also considered Article XV(5) relevant to Vaisala’s allegation that PWGSC erred in its determination that GRAW’s bid was the highest-ranked responsive bid and, therefore, awarding GRAW the contract. Article XV(5) provides as follows:

5. Unless a procuring entity determines that it is not in the public interest to award a contract, the entity shall award the contract to the supplier that the entity has determined to be capable of fulfilling the terms of the contract, and that, based solely on the evaluation criteria specified in the notices and tender documentation, has submitted:
   1. The most advantageous tender; or
   2. Where price is the sole criterion, the lowest price.

[Emphasis added]

52. In assessing whether PWGSC/ECCC erred in its evaluation of GRAW’s bid, the Tribunal is guided by certain key principles. The Tribunal typically accords a large measure of deference to evaluators in their evaluation of proposals. It is well established that the Tribunal will interfere only with an evaluation that is unreasonable^[21] and will substitute its judgment for that of the evaluators only when the evaluators have not applied themselves in evaluating a bidder’s proposal, have ignored vital information provided in a bid, have wrongly interpreted the scope of a requirement under the RFP, have based their evaluation on undisclosed criteria or have otherwise not conducted the evaluation in a procedurally fair way.^[22]
53. Accordingly, the main issue to be determined in respect of the third ground of complaint is whether the evaluation of GRAW’s bid against the mandatory and point-rated technical criteria of the RFP, as well as the conduct of POP testing, was reasonable.
54. Before turning to its examination of each of the technical requirements in issue, the Tribunal will briefly address the nature of the evidence presented in this case. Both parties referred extensively to the information from the 2011 World Meteorological Organization Radiosonde Intercomparison (WMORI), a published study.^[23] It was undisputed that the WMORI is a widely recognized source of information on the performance and accuracy of radiosonde products, including the GRAW DFM-09 and Vaisala’s own radiosonde product. In fact, in Amendment No. 001 to the RFP, PWGSC confirmed that bidders could submit analysis of test reports using data from the WMORI to demonstrate compliance with M7, M13, M14 and R2.^[24]
55. According to Vaisala, the small number of radiosonde manufacturers globally means that producers are familiar with the products available in that market and their capabilities. This knowledge is in turn largely based on “stringent and standardized international testing”, such as that which is reported in the WMORI.^[25]
56. The publicly available information on the GRAW DFM-09 radiosonde in the 2011 WMORI initially provided the evidentiary basis for Vaisala’s complaint. The essence of its argument was that GRAW’s bid could not have demonstrated compliance with several technical mandatory and rated requirements of the RFP because, according to Vaisala, the WMORI indicated that the GRAW DFM-09 radiosonde was unable to meet those requirements.
57. Following the Tribunal’s acceptance of the complaint for inquiry, the GIR was filed, which included a confidential version of GRAW’s bid and Mr. Anderson’s evidence explaining the evaluation process and the information in GRAW’s bid that was relied on by the ECCC Technical Authority in determining that the technical requirements in issue were met.
58. In its comments on the GIR, Vaisala challenged the evidence provided by Mr. Anderson regarding the evaluation of GRAW’s bid against some, but not all, of those technical requirements. In particular, Vaisala did not dispute Mr. Anderson’s evidence regarding M11, M12 or R2. Moreover, its reply submissions in respect of R6 and R7 were not relevant to the requirements stated in those criteria, for the reasons set out below.
59. The Tribunal will address the evaluation of GRAW’s bid against each of the evaluation criteria at issue, beginning with the mandatory criteria followed by the point-rated criteria. Lastly, the Tribunal will examine Vaisala’s allegation of improper POP testing.

Mandatory Criterion M7

60. Mandatory criterion M7 required temperature sensor accuracy to be less than or equal to 0.5°C from surface to 10 hectopascals (hPa), calculated at two standard deviation (k=2). To satisfy this requirement the bidder had to provide an analysis of test reports under operational conditions of a minimum of 20 daytime flights and 10 nighttime flights.^[26]
61. At the initial complaint stage, Vaisala relied on publicly available information discussed in Mr. Miloshevich’s affidavit, including technical data published by GRAW and the WMORI study, to purport that the GRAW DFM-09 temperature measurements had substantial systematic error and, therefore, were incapable of fulfilling the M7 temperature accuracy requirement.^[27]
62. The GIR provided confidential evidence, including a copy of GRAW’s bid and Mr. Anderson’s affidavit that explained in detail how the evaluation of GRAW’s bid was conducted in relation to the M7 requirement. GRAW in fact submitted portions of the WMORI study, including two graphs showing test data to demonstrate compliance with M7. According to Mr. Anderson, the evaluators ████ ███████ █████ ██ █████ ██████ ████ ███ ████ ██████ ███████ █████ ██████ ███ ███████████ ███ ███ ███ ███ ███████ ██████ ██ ███ ████████.^[28]
63. The evaluators then looked to other data submitted as part of GRAW’s bid. In particular, they considered other data provided from the WMORI study that compared the temperature sensor accuracy of the GRAW DFM-09 to a “traceable flight reference radiosonde . . . calibrated to international standards on the ground”, which was, in fact, Vaisala’s radiosonde (the Vaisala RS92). Such a comparison is known as a “consistency test”. According to Mr. Anderson, the data from this portion of the WMORI study is a summary of all flights conducted in the study, satisfying the requirement set out in M7 for testing data based on a minimum of 20 daytime and 10 nighttime flights.^[29] This data, together with other data submitted by GRAW, was sufficient to satisfy the evaluators that its bid met the requirements of M7.
64. The Tribunal notes that ████ ███████ ██ ███ ███████ ███ ███████ ██ ███████ ███████ ███████████ ██ ██ ███████ ███████ ███████ ██ ███████ ███████████ ███████ ███ ███ ███████ ██ ██ ███████████ ███████  ███ ███████████ ███ ███████ ███████████ ███ ███ ███████  ███████ ██ ██ ███████ ██ ██ ██ ██ ███████ ██ ██ ███ ██ ███ ███.^[30] In the Tribunal’s view, this reflects a thorough and conscientious evaluation of GRAW’s bid.
65. In its comments on the GIR, Vaisala argued that the data from GRAW’s bid that was relied upon by the evaluators could not have demonstrated compliance with M7 for two reasons.
66. First, the data from the WMORI study submitted by GRAW in support of M7, as well as the ███████████ ███████████ ███████████ ██ ███████████ ██████ ██████ ███████ ███████ (███████████)████.^[31] On the basis of Mr. Miloshevich’s evidence, Vaisala argued that M7 required a measurement of total temperature measurement accuracy, meaning the sum of “random” error plus “systematic” or “bias” error, systematic error being other types of error caused by calibration bias, time-lag error or conditions such as sensor icing.^[32]
67. Vaisala argued that the data from the WMORI study, submitted by GRAW, was incomplete as it omitted the data from the WMORI study regarding systematic error, and could never have satisfied mandatory criterion M7. The evidence demonstrates that, in fact, the evaluation team did not accept that this data satisfied the requirement, and only determined compliance with mandatory criterion M7 by considering this data in combination with other information provided in GRAW’s bid.^[33] It was open to the evaluation team to look to information elsewhere in GRAW’s bid, and in the Tribunal’s view this was a reasonable approach to take.
68. Second, Vaisala argued on the basis of Mr. Miloshevich’s evidence that “consistency test” data “only indicates that there is agreement between two test sensors” and does not accurately measure temperature sensor accuracy.^[34] According to Vaisala, “[i]f a bidder provides consistency test results only, they are not sufficient to ‘demonstrate compliance’ to the ‘temperature sensor accuracy’ requirement.”^[35]
69. Without necessarily accepting this standard for demonstrating compliance, as proposed by Vaisala, the Tribunal notes that GRAW’s bid did not provide consistency test results only, but rather in addition to other data. Mr. Anderson’s affidavit demonstrates that, taken together, these test results allowed the evaluation team to conclude that GRAW’s submission satisfied mandatory criterion M7.
70. The Tribunal finds that the above evidence demonstrates that the evaluation team applied themselves in evaluating GRAW’s bid against the M7 requirement. There is no indication that the evaluators ignored any vital information, improperly interpreted the scope of the requirement, relied on any undisclosed criteria or acted in a procedurally unfair manner.
71. On the basis of the foregoing, the Tribunal is satisfied that the ECCC Technical Authority’s evaluation of GRAW’s bid with regard to R7 was reasonable.

Mandatory Criterion M10 and Point-rated Criterion R3

72. The Tribunal will address the evaluation of GRAW’s bid against mandatory criterion M13 and rated criterion R3 together, as they both related to radiosonde humidity measurement accuracy. In order to meet mandatory criterion M10 bidders had to demonstrate radiosonde humidity accuracy (less than or equal to 5% for temperatures between +50°C and -60°C, calculated at k=2) by providing laboratory or operational test reports showing that the requirement is satisfied at five specific test points.^[36]
73. Point-rated criterion R3 allowed bidders to receive 10 points for demonstrating a more stringent standard for humidity accuracy (less than or equal to 4% for temperatures between +50^oC and -60^oC, calculated at k=2).
74. Vaisala alleged, on the basis of Mr. Miloshevich’s assessment of the WMORI study results and other publicly available data, that the GRAW DFM-09 exceeded the acceptable ranges of humidity accuracy required to meet M10 and rated criterion R3, respectively.^[37]
75. According to Mr. Anderson’s affidavit, the evaluation team concluded that test data submitted with GRAW’s bid demonstrated compliance with M10 and R3. Mr. Anderson further explained that some of the data in GRAW’s bid, ████ ████ ██ ████ ███████████ ███████████ ████ the acceptable ranges for M10 and R3, respectively, ████ ███████████ ███████████ ██ ██ ████ ████ ███████████ ██ ██ ████ ██ ████ ██ ████ ████ ████ ██ ██ ███████████.^[38]
76. Mr. Anderson also indicated that the evaluation team referred to ████ ███████████ ████ ████ ███████████ ███████████ (hereinafter referred to as the “humidity sensor datasheet”) that were provided in GRAW’s bid as demonstrating that the humidity sensor was accurate to 5% at K=2.^[39]
77. In response, Vaisala questioned the basis on which the evaluators accepted the humidity sensor datasheet as sufficient to demonstrate compliance with the humidity measurement accuracy requirements ████ ██ ██ ██ ████ ██ ████ ███████████ ██ ███████████ ███████████ ████ ████ ██ ██ ███████████ ███████████ ███████████ (████). According to Vaisala, it was “inconsistent, and scientifically incorrect, to allow a component datasheet to meet the requirement for one measurement and not another.”^[40]
78. The Tribunal cannot agree with this characterization. PWGSC has not asserted, nor does Mr. Anderson’s evidence indicate, that the evaluators concluded the humidity sensor datasheet in question was sufficient to demonstrate compliance of GRAW’s bid with M10 or R3.
79. The GIR refers to other sources of test result data included in GRAW’s bid, on which the evaluation team based its evaluation of M10 and R3.^[41] In fact, the remainder of Vaisala’s arguments relating to the evaluation of GRAW’s bid against the requirements of M10 and R3 pertain to the evaluation team’s reliance on these other sources. Vaisala’s position, that the evaluation team inappropriately relied solely on information from the humidity sensor datasheet, while also inappropriately relying on information from sources other than the humidity sensor datasheet, is untenable.
80. Vaisala further argued that the evaluation team selectively considered the test data submitted with GRAW’s bid, by ignoring the data characterised by hysteresis (i.e. the residual humidity remaining in a sensor as humidity is brought back down in the laboratory). It submitted that humidity measurements would be subjected to a range of conditions in the field and that to ignore hysteresis in testing was “simply wrong”.^[42]
81. In his affidavit, Mr. Anderson stated the evaluation team would have accepted ██ ████ ████ ██ ████ ████ ██ ██ ████ ██████ ███████████ ████ ██ ███████████ ██ ███████████ as there was no requirement for hysteresis defined in the technical evaluation criteria set out in the RFP.^[43] The Tribunal finds that explanation to be tenable.
82. As stated above, the applicable standard of review in a procurement inquiry is reasonableness, not correctness. The Tribunal is not tasked with determining whether the evaluation conducted conforms to that which Vaisala or Mr. Miloshevich would recommend or consider to be appropriate standards by, for example, taking into account the effect of hysteresis. Instead, the Tribunal is tasked with determining whether the ECCC Technical Authority’s evaluation of the compliance of GRAW’s bid with the terms of the RFP was reasonable. Furthermore, the Tribunal recognizes that the procuring entity is entitled to structure the RFP, including technical evaluation criteria,^[44] in a manner which fulfills its legitimate operational requirements.^[45]
83. In light of the above, the Tribunal is satisfied that the evaluators applied themselves in evaluating GRAW’s bid with respect to the M7 requirement and rated criterion R3. There is no indication that the evaluators ignored any vital information, improperly interpreted the scope of the requirement, relied on any undisclosed criteria or acted in a procedurally unfair manner. The Tribunal finds that it was reasonable for the evaluation team to conclude that the laboratory test reports submitted by GRAW demonstrated humidity measurement accuracy within the acceptable ranges stipulated in mandatory criterion M10 and point-rated criterion R3, respectively.

Mandatory Criterion M11

84. Technical mandatory criterion M11 required bidders to provide laboratory or operational testing reports to demonstrate humidity sensor response time of the radiosonde.^[46] The RFP set out the following requirements which had to be met for humidity sensor response time, defined as sensor reaching 63.2% of a step humidity change, at two temperatures:

    1. response time must be less than or equal to 1 second at surface and +20°C; and
    2. response time must be less than or equal to 20 seconds at surface and -40°C.

85. In its complaint, Vaisala argued that the GRAW DFM-09 was incapable of meeting requirement M11 on the basis of publicly available information from the WMORI study regarding the humidity sensor response time of the GRAW DFM-09 at -70°C. While Vaisala admitted that M11 did not specify a sensor response time requirement below -40°C, it submitted, on the basis of Mr. Miloshevich’s evidence, that the humidity sensor response time of the GRAW DFM-09 at -70°C demonstrated that “it is an extremely slow-responding sensor which can lead to large measurement errors at low temperatures . . . .”^[47]
86. The evidence shows, however, that GRAW’s bid did not submit the WMORI study data referred to by Vaisala to demonstrate compliance with M11. Rather, GRAW provided other laboratory test results, which the evaluators concluded satisfied the requirements of M11.^[48]
87. The Tribunal sees no basis to interfere with the evaluation of GRAW’s bid in response to this criterion. The information provided in the GIR and Mr. Anderson’s affidavit demonstrates that the evaluators applied themselves in evaluating GRAW’s bid against the M11 requirement. That evidence was not disputed by Vaisala in its comments on the GIR. Moreover, there is no indication that the evaluators ignored any vital information, improperly interpreted the scope of the requirement, relied on any undisclosed criteria or acted in a procedurally unfair manner.
88. The Tribunal is therefore satisfied that the evaluation of GRAW’s bid with regard to M11 was reasonable.

Mandatory criterion M12

89. Mandatory criterion M12 required the radiosonde to be equipped with a pressure sensor that measured over the complete range from 1050 hPa to 3 hPa. To demonstrate compliance, bidders had to provide laboratory or operational test reports or published technical specifications.^[49]
90. Vaisala submitted that, as confirmed by Mr. Miloshevich’s evidence, publicly available technical data on the GRAW DFM-09 indicated that it does not include a dedicated pressure sensor, although there is an option to add one.^[50] In addition, Vaisala submitted that POP testing would have been required for the evaluators to verify that any such added sensor met the M12 criterion.
91. The evidence filed with the GIR, including GRAW’s bid and Mr. Anderson’s affidavit, clearly demonstrated that, contrary to Vaisala’s allegation, the GRAW DFM-09 configuration proposed in GRAW’s bid had a pressure sensor.^[51] Furthermore, GRAW’s bid included supporting documentation in the form of laboratory test results which the evaluators concluded satisfied the requirements of M12.^[52] Vaisala did not dispute that evidence in its comments on the GIR.
92. The information provided in the GIR and Mr. Anderson’s affidavit demonstrates that the evaluators applied themselves in evaluating GRAW’s bid against the M12 requirement. There is no indication that the evaluators ignored any vital information, improperly interpreted the scope of the requirement, relied on any undisclosed criteria or acted in a procedurally unfair manner.
93. The Tribunal is therefore satisfied that the evaluation of GRAW’s bid with regard to M12 was reasonable.

Mandatory Criterion M13

94. Technical mandatory criterion M13 required pressure measurement accuracy calculated at two standard deviation (k=2) with the following specifications (after the application of surface pressure correction is applied by the ground system, if applicable):

Accuracy must be less than or equal to 1 hPa from surface to 100 hPa.

Accuracy must be less than or equal to 0.6 hPa from 100 hPa to 10 hPa.

95. To demonstrate compliance with M13, bidders had to provide an analysis of test reports demonstrating pressure measurement accuracy under operational conditions of a minimum of 20 daytime flights and 10 nighttime flights.^[53]
96. Vaisala submitted that the evaluators erred in concluding that GRAW’s bid met M13 because ████████████ ██████ ██ ██████ ██ ███████████ ██████ ██████ ███████████ ██████ ██████ ██ ██ ██████.^[54] According to Vaisala, the evaluators improperly accepted derived pressure accuracy measurements for GRAW’s product that were provided in its bid.
97. Mr. Anderson stated in his affidavit that the RFP allowed bidders to demonstrate compliance with the pressure measurement accuracy requirement set out in M13 through one of two methods: (1) by using an on-board pressure sensor or (2) using altitude measurements from GPS location data to derive pressure accuracy.^[55] His evidence further indicated that the evaluation of GRAW’s bid was consistent with this interpretation.^[56]
98. The original version of mandatory criterion M13 referred to “Pressure Sensor Accuracy” and was subsequently revised in Amendment No. 001 to the RFP to refer to “Pressure Measurement Accuracy” [emphasis added].^[57] According to Mr. Anderson, this constituted ██ ███████████ ██ ██████ ██ ██ ███████████ ███████████ ███████████ ███████████ ██████ ██ ██████ ██████ ██████ ███████████ ██ ██████ ██ ███████████ ██████ ██████ ███████████ ██ ███████████ ███████████ ███████████ ██████ ██ ██████ ██████ ██████ ██ ██ ██████ ██ ██ ███████████ ███████████ ██████ ██ ███████████ ███████████ ██ ██ ███████████ ██████ ██ ███████████ ██ ██ ██ ███████████ ██████ ██ ██████ ██ ██████ ███████████ ██ ██ ██████ ███████████ ██ ██ ██████.”^[58] Mr. Anderson stated that the ECCC Technical Authority, in setting out the technical evaluation requirements, did not want to constrain the vendors to use a specific type of methodology to meet the requirement.^[59]
99. In response, Vaisala submitted that nothing in the RFP or the related amendments supports Mr. Anderson’s interpretation of M13. On the contrary, Vaisala argued that since the preceding mandatory requirement, M12, required bidders to demonstrate that their radiosonde “is equipped with a pressure sensor” (as confirmed by PWGSC in its answers to supplier questions No. 8 and 22 of Amendment No. 001), M13 should be read as requiring pressure accuracy measurements from that pressure sensor. Vaisala also pointed to Annex A of the RFP, in which section 4(q) required that “[t]he radiosondes must be equipped with pressure sensors to be used as primary source of pressure data.”^[60]
100. The amendment of M13 to remove the word “sensor” resulted from a supplier question during the procurement process (namely, Q22/A22 in Amendment No. 001). The supplier suggested changes to the M13 requirement on the basis of a World Meteorological Organization (WMO) recommendation that “most radiosondes nowadays use (GPS based) calculated geopotential height to derive pressure.”^[61] As a result, the wording of M13 was amended from “pressure sensor accuracy” to “pressure measurement accuracy”. In the Tribunal’s view, this change clearly indicated that bidders were not limited to providing test results based on actual pressure sensor data for the purposes of demonstrating compliance with the M13 requirement; they could provide GPS-derived pressure accuracy measurements instead.
101. The fact that A22 also confirmed that bidders had to demonstrate that their radiosonde “is equipped with a pressure sensor” for the purposes of the M12 requirement is irrelevant, as these were altogether separate requirements. The Tribunal finds that it was reasonable, on the basis of the evaluation criteria as set out in the RFP, for the evaluators to assess bid compliance with M12 and M13 independently.
102. With respect to the requirement set out in Annex A of the RFP, that is a separate section of the RFP that described the contractual requirements for the goods and services being procured. It did not form part of the technical evaluation criteria set out in Annex D. Thus, while section 4(q) of Annex A indicated that on‑board pressure sensors would be used to obtain pressure data, the Tribunal sees no basis on which to read in a more stringent product requirement for the purposes of interpreting the technical evaluation criteria of the RFP.^[62]
103. The Tribunal also considers Q23/A23 in Amendment No. 001 to the RFP to be relevant to the interpretation of M13. The supplier requested “clarification if the geopotential height specification may be met with GPS-based measurements, or relaxed to be consistent with the pressure-based measurements documented in WMO Guide 8.”^[63] While Q23 only referred to the product specifications in Annex A with respect to both pressure accuracy and geopotential height accuracy,^[64] PWGSC’s response in A23 also addressed the technical evaluation criteria related to pressure accuracy and geopotential height accuracy measurements, namely, M13 and M14. In particular, the last sentence of A23, which Vaisala did not mention in its submission,^[65] reads “[s]ee Amendment regarding pressure measurement accuracy.”^[66] In the Tribunal’s view, this is clearly a reference to the amendment of M13 that resulted from Q22/A22,^[67] i.e. to replace the phrase “pressure sensor accuracy” with “pressure measurement accuracy”.
104. Finally, Vaisala’s argument, that the amendment of M13 was for the purpose of clarifying that the pressure accuracy measurement reflected the accuracy of the radiosonde as a whole rather than simply the sensor itself, is inconsistent with the fact that PWGSC did not similarly amend other requirements specifying sensor performance specifically, for example M7 and M11.
105. On the basis of the foregoing, the Tribunal is satisfied that the evaluators properly interpreted the scope of the M13 requirement in their evaluation of GRAW’s bid against this criterion and did not ignore any vital information or rely on any undisclosed criteria. The Tribunal finds that the evaluation of GRAW’s bid with regard to M13 was reasonable.

Point-rated Criterion R2

106. In the GIR, PWGSC provided evidence demonstrating that point-rated criterion R2, which related to radiosonde temperature accuracy, is not relevant to Ground 3 of the complaint because ██████ ██ ██ ██ ██████ ██ ██████ ██ ██.^[68] Vaisala did not dispute, in its comments on the GIR, that rated criterion R2 was no longer at issue.
107. The Tribunal is therefore satisfied that the evaluation of GRAW’s bid with regard to R2 was reasonable.

Point-rated Criterion R6

108. Point-rated criterion R6 allowed a bidder to earn up to five points if it provided documentation that demonstrated how the sounding system of the radiosonde maintains performance under conditions of water accumulation, ice formation and wet-bulb effect on the temperature sensor.^[69]
109. Vaisala alleged that, according to Mr. Miloshevich’s examination of information in the WMORI, the GRAW DFM-09 does not have protection from evaporative cooling (“wet bulb”) errors and, therefore, its bid should not have been awarded points for this portion of the R6 criterion.^[70]
110. According to evidence filed with the GIR, GRAW submitted internal documentation in its bid that was determined by the evaluators to satisfy the portion of R6 in issue.^[71]
111. Vaisala did not dispute in its comments on the GIR that the documentation provided in GRAW’s bid satisfied the requirements of R6; rather, it alleged that the evaluators failed to test GRAW’s product to verify the performance of its temperature sensor. The Tribunal considers Vaisala’s allegation irrelevant to the evaluation of criterion R6, which required demonstration by way of documentation, and not testing or verification.
112. In light of the above, the Tribunal finds that the evidence filed with the GIR demonstrates that the evaluation team applied themselves in evaluating GRAW’s bid against the R6 criterion. There is no indication that the evaluators ignored any vital information, improperly interpreted the scope of the criterion, relied on any undisclosed criteria or acted in a procedurally unfair manner.
113. The Tribunal is therefore satisfied that the ECCC evaluation of GRAW’s bid with regard to R6 was reasonable.

Point-rated Criterion R7

114. Point-rated criterion R7 allowed a bidder to earn up to five points for providing documentation that demonstrated how the sounding system of the radiosonde maintained performance in conditions of water accumulation and ice formation (moisture contamination) on the humidity sensor and upon exit of cloud.^[72]
115. To support its allegation that GRAW’s bid should not have been awarded full points for the R7 criterion, Vaisala referred to several publicly available sources of information discussed in Mr. Miloshevich’s affidavit, which indicated that the GRAW DFM-09 was “susceptible” to sensor icing and, therefore, would have been incapable of meeting the performance requirement under R7.^[73]
116. According to evidence filed with the GIR, GRAW submitted documentation in its bid that was determined by the evaluators to satisfy the portion of R7 in issue.^[74]
117. Vaisala did not dispute that the documentation provided in GRAW’s bid satisfied the requirements of R7; rather, it alleged that the evaluators failed to test GRAW’s product to verify the performance of its humidity sensor on the basis of other publicly available test results of GRAW’s product that were not included in GRAW’s bid. The Tribunal considers Vaisala’s allegation irrelevant to the evaluation of criterion R7, which required demonstration by way of documentation, and not testing or verification.
118. The information provided in the GIR and Mr. Anderson’s affidavit demonstrates that the evaluators applied themselves in evaluating GRAW’s bid against the R7 criterion. There is no indication that the evaluators ignored any vital information, improperly interpreted the scope of the criterion, relied on any undisclosed criteria or acted in a procedurally unfair manner.
119. The Tribunal is therefore satisfied that the ECCC evaluation of GRAW’s bid with regard to R7 was reasonable.

Proof of Performance Testing

120. Article 4.1.4 of the RFP provided that the responsive bid with the highest combined rating of technical merit and price must deliver and install equipment to be subjected to POP testing. Where such a bid did not pass the POP testing, it would be deemed non-compliant. Annex D contained the full breakdown of the POP testing requirements.^[75]
121. Vaisala’s argument that the POP testing of GRAW’s radiosonde was deficient rested primarily on its simultaneous claims with regard to the evaluation of the technical point-rated and mandatory criteria outlined above. In essence, its position was that, since GRAW’s bid was not compliant with these requirements, it should not have passed the POP testing outlined in the RFP, had such testing been applied “rigorously”.
122. In support of this assertion, Vaisala relied on Mr. Miloshevich’s opinion that “the Proof of Performance Testing described in the RFP, that would demonstrate radiosonde performance that meets the accuracy requirements set out in the RFP, would, at a minimum entail” several testing conditions and parameters that are set out in his affidavit, including, for example, that a minimum of 2.5 months (and more realistically 5 months) would be needed to conduct such POP testing.^[76]
123. Furthermore, Vaisala argued that POP testing conducted for GRAW’s product was inconsistent with the POP testing requirements in the RFP. First, Vaisala alleged that the POP testing failed to test the GRAW DFM-09 under difficult weather conditions as it claimed was required by Annex D, Section 3, subsection q) of the RFP. Second, it alleged that the duration of the POP testing was insufficient to provide accurate results, given that the GIR indicated the POP testing of GRAW’s radiosonde was conducted over the course of 10 days.^[77] In this respect, Vaisala pointed out that the testing period was 30 days in a nearly identical RFP in 2006, which only required bidders to provide 120 radiosondes in comparison with 300 radiosondes under the current RFP.^[78] Vaisala submitted that, on the basis of the description of POP testing in the RFP, as well as the requirement for bidders to provide up to 300 radiosondes, it “reasonably expected that the POP Testing would be extensive.”^[79]
124. The Tribunal reiterates that in respect of Ground 3, as indicated above, the issue to be determined is whether the POP testing was conducted in conformity with the relevant RFP provisions and that the applicable standard of review is reasonableness. Given its conclusions above with respect to the technical mandatory and point-rated criteria in issue, the Tribunal does not accept as a general premise that GRAW’s bid was incapable of meeting the POP testing requirements. In addition, Vaisala’s argument that additional or more stringent requirements and conditions would need to be implemented in order for the POP testing described in the RFP to provide accurate results is misplaced. Regardless of industry standards or practices, or Mr. Miloshevich’s opinion evidence, the Tribunal is of the view that had Vaisala wished to challenge the terms of the RFP itself, including the POP testing requirements stated in Section 3 of Annex D, it should have filed a timely complaint on that basis, as discussed above.
125. Accordingly, the Tribunal will focus its analysis on Vaisala’s claim that POP testing conducted for GRAW’s product was inconsistent with the POP testing requirements as they were set out in the RFP.
126. In his affidavit, Mr. Anderson explained that the POP testing requirements in the RFP were designed to test general conformity with the performance presented on paper, but intentionally did not define specific conditions required to pass, in order to maintain flexibility in determining at what point compliance had been demonstrated.^[80] Mr. Anderson further submitted that PWGSC/ECCC determined the POP process would not use a “full radiosonde system verification process”, like that used in the previous 2006 RFP, due to the significant time and expense involved in such a project.^[81] Annex D, Section 3, subsection q) of the RFP provides for an “operational assessment”. Paragraph 1 of subsection q) provides that this assessment includes “[v]erifying that the radiosondes perform reliably under operational conditions; this will include the capacity of the radiosondes to withstand the shock associated with releases in difficult weather conditions.”^[82] The Proof of Performance Summary Report filed with the GIR indicates that difficult weather conditions were not encountered during the testing of GRAW’s product.^[83] The Report also indicates that GRAW’s radiosonde ██ ██ ███████████ ██████ ██ ██ ██████ ██ ██ ██ ███████████ ██████ ██ ██ ██ ███████████ ██████ ██ ██████ ███████████ ██ ██ ██ ██████ ██████ ███████████ ██████ ██ ██████ ██ ██ ██ ██████ ███████████ ██ ███████████ ██████ ██ ██ ███████████ ██ ██████ ██████ ██ ███████████.^[84]
127. Nonetheless, the Tribunal finds that paragraph q(1) of Annex D, Section 3 did not require the evaluators, for the purposes of the POP testing requirements, to assess conformity with the mandatory and point-rated criteria under difficult weather conditions, as argued by Vaisala (for example, with regard to point-rated criteria R6 and R7).^[85] A plain reading of the POP testing requirements under that provision indicates only that the evaluators would verify reliable performance under operational conditions, including the capacity of the radiosondes to withstand the shock associated with releases in difficult weather conditions.
128. Verifying such capacity could reasonably take place by observing the ability of the radiosonde to withstand shocks encountered during POP testing under conditions other than difficult weather, or inferred from other indicators of durability observed during testing. This is an area in which the Tribunal believes significant deference to the expertise of technical evaluators is appropriate, and the POP Summary Report on the record indicates that the evaluators turned their minds to the assessment of such capacity in the course of POP testing.
129. Furthermore, according to Mr. Anderson, the POP testing was not intended to verify compliance with each mandatory and point-rated criteria listed in the RFP, but instead it was to be conducted for the purpose of evaluating conformity to the general performance requirements stated under Section 3 of Annex D on POP testing. The ECCC Technical Authority intentionally omitted specific performance criteria at this stage of the technical evaluation, including cloud-type performance or exiting specific clouds, as this would require the evaluation process to be at the mercy of the weather.^[86]
130. In this regard, and absent language indicating that POP testing was to be conducted specifically under difficult weather conditions, the Tribunal finds PWGSC’s explanation to be tenable, and the conclusions of the ECCC Technical Authority’s evaluation team to be reasonable.
131. Similarly, with respect to Vaisala’s claim that POP testing was conducted over an insufficient length of time, the Tribunal finds that the relevant provisions of the RFP did not specify a time line for POP testing. Mr. Anderson stated that the POP testing requirements were determined on the basis of ECCC’s experience in the use of radiosondes, as well as its acceptance of risk, in deciding not to use a full radiosonde system verification process similar to the previous 2006 RFP.^[87]
132. Mr. Anderson further indicated that ECCC requested such a large number of radiosondes in order to ensure a likely representation of the manufacturer’s population of radiosondes.^[88] He submitted that the ECCC Technical Authority omitted specific testing requirements and time lines from the POP testing provisions of the RFP in order to maintain flexibility to decide at what point compliance had been demonstrated, reflecting a conscious undertaking of risk on the part of ECCC.^[89] The Tribunal considers this explanation to be tenable and sees no basis on which it could find that the evaluators failed to conduct the POP testing in accordance with the process set out in the RFP. That this deviated from the extent or length of POP testing inferred by Vaisala, or recommended after the fact by Mr. Miloshevich, does not translate into a breach of the AGP.
133. The Tribunal therefore finds that the ECCC Technical Authority acted reasonably in evaluating GRAW’s bid against the technical mandatory and rated criteria discussed above, and in its conduct of the POP testing. As a result, the Tribunal finds that it was reasonable for PWGSC to award the contract to GRAW.
134. In light of the foregoing, the Tribunal finds that the complaint is not valid.

COSTS

135. Both parties requested costs in relation to the proceedings. Given its success, pursuant to section 30.16 of the CITT Act, the Tribunal awards PWGSC its reasonable costs in responding to this complaint. In accordance with the Procurement Costs Guideline (the Guideline), the Tribunal’s preliminary indication of the level of complexity of the complaint is Level 2 and its preliminary indication of the amount of the cost award is $2,750. If any party disagrees with the cost decision, it may make submissions to the Tribunal, as contemplated by article 4.2 of the Guideline. The Tribunal reserves jurisdiction to establish the final amount of the costs award.

DETERMINATION

136. Pursuant to subsection 30.14(2) of the CITT Act, the Tribunal determines that the complaint is not valid.
137. Pursuant to section 30.16 of the CITT Act, the Tribunal awards PWGSC its reasonable costs incurred in responding to the complaint, which costs are to be paid by Vaisala. In accordance with the Guideline, the Tribunal’s preliminary indication of the level of complexity for this complaint case is Level 2, and its preliminary indication of the amount of the cost award is $2,750. If any party disagrees with the preliminary indication of the level of complexity or the preliminary indication of the amount of the cost award, it may make submissions to the Tribunal, as contemplated by article 4.2 of the Guideline. The Tribunal reserves jurisdiction to establish the final amount of the award.

APPENDIX: RELEVANT RFP PROVISIONS^_[90]

PART 4 – EVALUATION PROCEDURES AND BASIS OF SELECTION

4.1 Evaluation Procedures

Bids will be assessed in accordance with the entire requirement of the bid solicitation including the technical and financial evaluation criteria. For the purposes of this solicitation, Canada will utilize a two-step evaluation process as more fully described herein.

An evaluation team composed of representatives of Canada will evaluate the bids.

Not all members of the evaluation team will necessarily participate in all aspects of the evaluation.

4.1.1 Two Step Bid Evaluation Process

Step 1: Initial Review of Mandatory Requirements

Canada will conduct an initial review of the Bidder’s bid to determine if all mandatory requirements (as outlined in Annex D) have been addressed and met as required. This will be a paper evaluation only; it will not apply to the physical evaluation of equipment (Proof of Performance Testing). After the initial review, if any bid is determined to be non-responsive, the Contracting Authority will provide each Bidder with a “Preliminary Evaluation Report” listing the non-compliant mandatory requirements evaluated to date. This will include only a list of RFP references.

Bidders whose bids are considered to be non-responsive will be invited by the Contracting Authority to submit additional or different information to demonstrate to Canada, in accordance with the solicitation, that the bid is compliant with the solicitation requirements. This information must be submitted on or before the date and closing time specified in the invitation.

lf all bids are determined to be responsive, no Preliminary Evaluation Reports will be issued and the Contracting Authority will complete the full bid evaluation, including the financial bid evaluation, using the original bid documents submitted.

The new information submitted by the Bidder must be based on the system it proposed at bid closing. A bidder responding to a request for information will not be allowed to do a hardware or software substitution to correct a non-compliance issue.

Step 2: Final Review of Mandatory Requirements

Canada will conduct a final review of the Bidder's bid, taking into account the new information submitted, to determine if all mandatory requirements have been met as required in the RFP. Should one or more of the initially evaluated non-compliant mandatory requirements continue to be evaluated as non-compliant after receipt of the new information, or a new non-compliancy be created as a result of the new information, the bid will be evaluated as non-responsive and will not be given any further consideration.

The new information submitted will take precedence over that submitted with the original bid at bid closing. Bids that have demonstrated compliance with mandatory requirements at the conclusion of the final review will continue to be evaluated.

Point Rated Technical Criteria, Proof of Performance Testing and Financial Evaluation will not be included in the Two Step Bid Evaluation Process, and therefore additional information will not be accepted after bid closing.

4.1.2 Technical Evaluation

4.1.2.1 Mandatory Technical Criteria

See Annex D

4.1.2.2 Point Rated Technical Criteria

See Annex D

. . .

4.1.4 Proof of Performance Testing

The responsive bid with the highest combined rating of technical merit and price must deliver and install equipment to Stony Plain, Alberta to allow ECCC to carry out Proof of Performance testing.

lf the responsive bid with the highest combined rating of technical merit and price does not pass the proof of performance testing they will be deemed non-compliant and the next responsive bid with the highest combined rating of technical merit and price will be contacted and must submit equipment to perform Proof of Performance Testing, and so on until the highest combined rating of technical merit and price passes the proof of performance testing.

See Annex D for a full breakdown of the Proof of Performance Testing requirement.

4.2 Basis of Selection - Highest Combined Rating of Technical Merit and Price

1. To be declared responsive, a bid must:

a. comply with all the requirements of the bid solicitation; and

b. meet all mandatory criteria; and

c. Pass the proof of performance testing

2. Bids not meeting (a) or (b) or (c) will be declared non-responsive.

3. The selection will be based on the highest responsive combined rating of technical merit and price. The ratio will be 60% for the technical merit and 40o/o for the price.

4. To establish the technical merit score, the overall technical score for each responsive bid will be determined as follows: total number of points obtained / maximum number of points available multiplied by the ratio of 60%

5. To establish the pricing score, each responsive bid will be prorated against the lowest evaluated price and the ratio of 40%.

6. For each responsive bid, the technical merit score and the pricing score will be added to determine its combined rating.

7. Neither the responsive bid obtaining the highest technical score nor the one with the lowest evaluated price will necessarily be accepted. The responsive bid with the highest combined rating of technical merit and price will be recommended for award of a contract.

. . .

ANNEX D

EVALUATION CRITERIA

1. Mandatory Criteria

The bidder must demonstrate that they meet all of the mandatory requirements described below. Bidders who fail to meet all mandatory requirements will be considered non-responsive and will not be evaluated further. Bids meeting the mandatory criteria will go on to be evaluated for Point-Rated Technical Criteria

Mandatory Criteria

. . .

M7

The bidder must provide an analysis of test reports under Operational conditions of minimum 20 Daytime flights & Minimum 10 Night time flights to demonstrate compliance to the following requirements. The reference used in the flight should be traceable to international standards

Temperature Sensor Accuracy

The temperature sensor accuracy must be less than or equal to 0.5°C from Surface to 10 hPa. The temperature accuracy must be calculated at two standard deviation (k=2).

. . .

M10

To demonstrate compliance to this requirement the bidder must provide laboratory or operational test reports to show that the requirement is satisfied at the following test points

+20°C and 90%RH at surface

+20°C and 50%RH at surface

+20°C and 20%RH at surface

-40°C and 50%RH at surface

-40°C and 20%RH at surface

Humidity Accuracy

The radiosonde humidity accuracy must be less than or equal to 5% for temperatures between +50.0°C and  -60°C. The radiosonde humidity accuracy must be calculated at two standard deviation (k=2).

M11

The bidder must provide Laboratory or operational test reports to demonstrate compliance to the following requirements.

Humidity Response Time

The radiosonde humidity sensor response time is defined as sensor reaching 63.2% of a step humidity change. The humidity response time must have the following specifications:

Response time must be less than or equal to 1 second at Surface and +20.0°C

Response time must be less than or equal to 20 seconds at Surface and -40.0°C

M12

The bidder must provide laboratory or operational test report with data or graphs or published technical specifications to demonstrate compliance to the following requirements.

Pressure Sensor Measurement Range

The radiosonde must be equipped with a pressure sensor that measures over the complete range from 1050 hPa to 3 hPa.

M13^[91]

The bidder must provide an analysis of test reports under Operational conditions of minimum 20 Daytime flights & Minimum 10 Night time flights to demonstrate compliance to the following requirements. The reference used in the flight should be traceable to international standards on the ground.

Pressure Measurement Accuracy

The pressure measurement accuracy must be calculated at two standard deviation (k=2).The pressure measurement must have the following specifications after surface pressure correction is applied by the ground system (if applicable):

Accuracy must be less than or equal to 1h Pa from Surface to 100 hPa.

Accuracy must be less than or equal to 0.6 hPa from 100 hPa to 10 hPa.

. . .

2. Point Rated Criteria

Bids will be evaluated and scored in accordance with the point rated criteria outlined below. Bidders should outline in their bid how they meet each of the point rated criteria.

. . .

R2

The bidder should provide a minimum of 20 Daytime and 10 Night time test reports under operational conditions showing compliance to this requirement. The reference used in the flight should be traceable to international standards on the ground. The bidder must show that the reference sensor used is traceable to Sl units in order to receive points.

Temperature Accuracy must be less than or equal to 0.4°C from Surface to 10hPa. The radiosonde temperature accuracy must be calculated at two standard deviation (k=2) = 10 Points

R3

To demonstrate compliance to this requirement the bidder must provide laboratory or operational test reports to show that the requirement is satisfied at the following test points

1. +20°C and 90%RH at surface
2. +20°C and 50%RH at surface
3. +20°C and 20%RH at surface
4. -40°C and 50%RH at surface
5. -40%°C and 20%RH at surface

Humidity Accuracy must be less than or equal to 4% for temperatures between +50.0°C and -60°C. The radiosonde humidity accuracy must be calculated at two standard deviation (k=2).

= 10 points

. . .

R6

The bidder should provide documentation that demonstrates how the sounding system maintains performance in the following conditions

• Water accumulation, ice formation and wet bulb effect on the temperature sensor.
• Solar and infrared radiation effect on the temperature sensor at different phases of sounding.

Documentation for the water accumulation, ice formation and wet bulb effect on the temperature sensor = 5 points

Documentation for the solar and infrared radiation at different phases of sounding = 5 points

R7

The bidder should provide documentation that demonstrates how the sounding system maintains performance in the following conditions

Water accumulation and ice formation (moisture contamination) on the humidity sensor and upon exit of cloud.

Solar and infrared radiation effect on the humidity sensor at different phases of sounding.

Documentation for the Water accumulation and ice formation (moisture contamination) on the humidity sensor and upon exit of cloud. = 5 points

Documentation for Solar and infrared radiation effect on the humidity sensor at different phases of sounding. = 5 points

. . .

3. Proof of Performance Testing

The responsive bid with the highest combined rating of technical merit and price must deliver and install equipment to Stony Plain, Alberta to allow ECCC to carry out Proof of Performance testing.

The requested bidder will be given 30 days from the date of the request to delivery and install the equipment as required and outlined below.

lf the responsive bid with the highest combined rating of technical merit and price does not pass the proof of performance testing they will be deemed non-compliant and the next responsive bid with the highest combined rating of technical merit and price will be contacted and must submit equipment to perform Proof of Performance Testing, and so on until the highest combined rating of technical merit and price passes the proof of performance testing.

. . .

q) - Operational assessment: Two or more experienced operational staff will assess the system from an operational users perspective. This will be an ongoing process over the duration of the Proof of Performance Test, and will include feedback received from staff carrying out the actual test flights. This assessment has the following objectives:

. . .

iii. The Site Lead will obtain reports from observers about the performance of the radiosondes. Using these reports and from other sources, including personal observations during the test, the Site Lead will provide a written assessment. This assessment will address whether:

1. Verifying that the radiosondes perform reliably under operational conditions; this will include the capacity of the radiosondes to withstand the shock associated with releases in difficult weather conditions.
2. Hardware or software deficiencies.
3. Verifying the accuracy and completeness of system documentation supplied by the Bidder.
4. Verify the functionality of the workstation computer and user interface, against the technical requirements.

. . .