OSHKOSH DEFENSE CANADA INC.
v.
DEPARTMENT OF PUBLIC WORKS AND GOVERNMENT SERVICES
File Nos. PR-2015-051 and PR‑2015-067

Determination issued
Friday, May 20, 2016

Reasons issued
Wednesday, July 20, 2016

TABLE OF CONTENTS

• DETERMINATION
• STATEMENT OF REASONS
  • BACKGROUND
  • PRELIMINARY COMMENTS
  • PROCUREMENT PROCESS
  • REQUIREMENTS AT ISSUE
  • APPLICATION OF THE TRADE AGREEMENTS
    • Positions of Parties
    • Analysis
  • ANALYSIS
    • I.       Failure to Consider Information in Bid
    • II.     Failure to Notify Oshkosh of Anomalies Observed During Testing
    • III.         Failure to Properly Configure CTIS Settings
    • IV.        Failure to Use a Proper Mobile Dynamometer
    • V.         Failure to Award Points for Performance Achieved
    • VI.        Failure to Consult Vehicle Checklist
    • VII.       Failure to Properly Perform the Vehicle Payload Test
    • VIII.          Failure to Conduct a Proper Debriefing
    • IX.        Failure to Properly Document Evaluation Process
    • Conclusion
  • DETERMINATION
  • COSTS

IN THE MATTER OF two complaints filed by Oshkosh Defense Canada Inc. pursuant to subsection 30.11(1) of the Canadian International Trade Tribunal Act, R.S.C., 1985, c. 47 (4th Supp.);

AND FURTHER TO decisions to conduct an inquiry into the complaints pursuant to subsection 30.13(1) of the Canadian International Trade Tribunal Act;

AND FURTHER TO a decision to combine the complaint cases as a single proceeding pursuant to rule 6.1 of the Canadian International Trade Tribunal Rules.

BETWEEN

OSHKOSH DEFENSE CANADA INC. Complainant

AND

THE DEPARTMENT OF PUBLIC WORKS AND GOVERNMENT SERVICES Government Institution

DETERMINATION

Pursuant to subsection 30.14(2) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal determines that the complaints are valid in part.

Pursuant to subsections 30.15(2) and (3) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal recommends that the Department of Public Works and Government Services conduct a re-evaluation of the bid submitted by Oshkosh Defense Canada Inc., including a physical re-evaluation of the technical compliance program test protocols performed by the Nevada Automotive Test Center. If the re-evaluation demonstrates that Oshkosh Defence Canada Inc. would have been the winning bidder but for the breaches of the applicable trade agreements, the Canadian International Trade Tribunal recommends that the Department of Public Works and Government Services compensate Oshkosh Defence Canada Inc. for the profits it would have received had it been properly awarded the contract. The Canadian International Trade Tribunal will not recommend that the contract awarded to Mack Defense LLC be cancelled.

The Canadian International Trade Tribunal recommends that the Department of Public Works and Government Services and Oshkosh Defense Canada Inc. determine the feasibility of a physical re-evaluation, and report back to the Canadian International Trade Tribunal within 15 days of the issuance of the statement of reasons for this determination.

In the event that a physical re-evaluation is no longer feasible at this time, the Canadian International Trade Tribunal recommends that the Department of Public Works and Government Services compensate Oshkosh Defence Canada Inc. for its lost opportunity. In that circumstance, the Canadian International Trade Tribunal recommends that the parties negotiate the amount of compensation to be paid for lost opportunity, and report the outcome of the negotiations to the Canadian International Trade Tribunal within 30 days of the issuance of the statement of reasons for this determination.

Should the parties be unable to agree on the amount of compensation, Oshkosh Defense Canada Inc. shall file with the Canadian International Trade Tribunal, within 40 days of the issuance of the statement of reasons for this determination, a submission on the issue of compensation. The Department of Public Works and Government Services will then have seven working days after the receipt of Oshkosh Defense Canada Inc.’s submission to file a response. Oshkosh Defense Canada Inc. will then have five working days after the receipt of the Department of Public Works and Government Services’ reply submission to file any additional comments. Counsel are required to serve each other and file with the Canadian International Trade Tribunal simultaneously.

Pursuant to section 30.16 of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal awards Oshkosh Defense Canada Inc. its reasonable costs incurred in preparing and proceeding with the complaint, which costs are to be paid by the Department of Public Works and Government Services. Given the extraordinary level of complexity of this complaint, the Canadian International Trade Tribunal exercises its discretion to deviate from its Guideline for Fixing Costs in Procurement Complaint Proceedings. The Canadian International Trade Tribunal will inform the parties of the schedule for submissions on costs at a later date.

The Canadian International Trade Tribunal reserves jurisdiction to establish the final recommended remedy and the amount of the costs award.

Serge Fréchette
Serge Fréchette
Presiding Member

The statement of reasons will be issued at a later date.

Tribunal Member: Serge Fréchette, Presiding Member

Counsel for the Tribunal: Eric Wildhaber
Alexandra Pietrzak
Jessica Spina (student-at-law)

Registrar Officer: Julie Lescom

Complainant: Oshkosh Defense Canada Inc.

Counsel for the Complainant: Gerry Stobo
Vincent DeRose
Olivier V. Nguyen
Mandy Aylen

Government Institution: Department of Public Works and Government Services

Counsel for the Government Institution: Susan Clarke
Ian G. McLeod
Roy Chamoun
David D’Angela

Intervener: Mack Defense LLC

Counsel for the Intervener: Paul Conlin
R. Benjamin Mills
M. Drew Tyler

Please address all communications to:

The Registrar
Secretariat to the Canadian International Trade Tribunal
333 Laurier Avenue West
15th Floor
Ottawa, Ontario  K1A 0G7

Telephone: 613-993-3595
Fax: 613-990-2439
E-mail: citt-tcce@tribunal.gc.ca

STATEMENT OF REASONS

BACKGROUND

1. On January 6, 2016, Oshkosh Defense Canada Inc. (Oshkosh) filed a complaint with the Canadian International Trade Tribunal (the Tribunal) pursuant to subsection 30.11(1) of the Canadian International Trade Tribunal Act,^[1] concerning a Request for Proposal (RFP) (Solicitation No. W8476‑06MSMP/L) issued by the Department of Public Works and Government Services (PWGSC) on behalf of the Department of National Defence (DND) for the provision of Standard Military Pattern (SMP) vehicles for the Medium Support Vehicle System (MSVS) project in five variants: a Cargo variant, a Cargo with Material Handling Crane variant, a Load Handling System (LHS) variant, a Cargo Mobile Repair Truck (MRT) variant and a Gun Tractor variant. The requirement also includes trailers, Armour Protection Systems (APSs), trailers, associated equipment and In-service Support for the SMP vehicles, APSs, trailers and associated equipment.
2. Oshkosh alleged that PWGSC ignored information in its bid, failed to follow the evaluation provisions within the RFP, improperly conducted the evaluation and failed to conduct a proper debriefing.
3. As a remedy, Oshkosh requested that its bid be re-evaluated and that, upon the evaluation determining that Oshkosh should have been awarded the contract, the Tribunal recommend that the contract awarded to Mack Defense LLC (Mack Defense) be terminated and awarded to Oshkosh. In the alternative, Oshkosh asked that it be awarded the value of the profit that it reasonably could have expected had it been awarded the contract.
4. Oshkosh also requested that it be awarded its bid preparation costs and be compensated to reflect the prejudice inflicted by PWGSC on Oshkosh and on the integrity and efficiency of the procurement process.
5. On January 13, 2016, the Tribunal informed the parties that the complaint had been accepted for inquiry, having met the requirements of subsection 30.13(1) of the CITT Act and the conditions set out in subsection 7(1) of the Canadian International Trade Tribunal Procurement Inquiry Regulations.^[2]
6. On January 26, 2016, the Tribunal granted intervener status to Mack Defense. The Tribunal also granted PWGSC’s request for an extension of time to file its Government Institution Report (GIR) and informed the parties that, as a result of that request, the deadline for the issuance of its decision had been extended to 135 days after the filing of the complaint pursuant to paragraph 12(c) of the Regulations.
7. On February 19, 2016, in response to a motion filed by Oshkosh in its complaint and after considering the submissions of the parties, the Tribunal ordered PWGSC to produce all test reports, evaluations, analyses, memoranda, communications and contemporaneous records, including test data (such as digital video and photographic data), which together formed the full test record for each of the test profiles at issue in the complaint.
8. On February 23, 2016, PWGSC filed its GIR in accordance with rule 103 of the Canadian International Trade Tribunal Rules.^[3]
9. On February 24, 2016, PWGSC filed documents in response to the Tribunal’s order of February 19, 2016.
10. On March 3, 2016, in response to a letter from Oshkosh, the Tribunal wrote to PWGSC to ask that it confirm that all relevant documentation required by its order of February 19, 2016, had been produced. PWGSC wrote to the Tribunal on March 4, 2016, to confirm that it had produced all relevant documents.
11. On March 8, 2016, Mack Defense submitted its comments on the complaint and the GIR.
12. On March 10, 2016, Oshkosh filed a second complaint (File No. PR-2015-067) related to the RFP in which it alleged that PWGSC failed to maintain complete documentation regarding the conduct of the procurement in violation of Article 1017(1)(p) of the North American Free Trade Agreement.^[4]
13. On March 14, 2016, the Tribunal joined the two complaints, as requested by Oshkosh. However, as the inquiry process for File No. PR-2015-051 was well advanced, the Tribunal directed that the issues in File No. PR-2015-067 would be addressed through a separate submissions process.
14. On March 16, 2016, Oshkosh filed its comments on the GIR and Mack Defense’s reply in File No. PR-2015-051.
15. On March 22, 2016, PWGSC requested that it be permitted to make additional written submissions on alleged new arguments raised by Oshkosh in its comments on the GIR in File No. PR-2015-051.
16. On March 23, 2016, the Tribunal directed the parties to provide submissions on a list of specific questions (the Tribunal questions). It instructed parties to file their responses on April 7, 2016, and subsequently file replies to those responses on April 15, 2016. In addition, the Tribunal denied PWGSC’s request to make additional written submissions on alleged new arguments raised by Oshkosh in its comments on the GIR in File No. PR-2015-051.^[5]
17. On April 4, 2016, PWGSC provided the Tribunal with additional documents which were subject to the Tribunal’s production order of February 9, 2016, but which had “. . . only recently been brought to the attention of PWGSC . . . .”^[6] As Oshkosh had not received these documents in time to comment on them in its comments on the GIR in File No. PR-2015-051, the Tribunal granted Oshkosh an opportunity to file submissions on these additional documents.
18. On April 7, 2016, all parties filed their responses to the Tribunal questions. The parties then filed replies to these responses on April 15, 2016.
19. On April 11, 2016, PWGSC filed its GIR in File No. PR-2015-067. Mack Defense filed its comments on this GIR on April 19, 2016, and Oshkosh filed its comments on April 27, 2016.
20. On April 20, 2016, Oshkosh wrote to the Tribunal to request that it be permitted to file additional comments on the letter by Mr. Steve Walkiewicz filed by PWGSC in its submissions of April 15, 2016.
21. On April 22, 2016, both Mack Defense and PWGSC wrote to argue that Mr. Walkiewicz’s letter did not contain new information and to request that they be permitted to make additional submissions on arguments made by Oshkosh in its April 15, 2016, submissions regarding videos of the “Sand Dune Maximum Gradeability” tests (the videos) conducted on Oshkosh’s vehicles.
22. While the Tribunal initially determined that no further submissions were warranted on any grounds, it overlooked the fact that the videos had not been produced by PWGSC until after both Mack Defense and Oshkosh had filed their comments on the GIR. In the interest of procedural fairness, the Tribunal therefore decided to allow one final round of submissions on the videos only.
23. On April 29, 2016, PWGSC and Mack Defense filed their submissions on the videos. Oshkosh then filed its submission on May 3, 2016.
24. Given that there was sufficient information on the record to determine the validity of the complaints, including two rounds of submissions with respect to the questions posed by the Tribunal and the additional round of submissions regarding the videos, the Tribunal decided that an oral hearing was not required and disposed of the complaints on the basis of the written information on the record.

PRELIMINARY COMMENTS

25. Before embarking on its analysis, the Tribunal notes that these complaints involved an extremely complex procurement process for the acquisition of highly specialized goods and services. The solicitation called for the provision of SMP vehicles for the MSVS project in five variants and associated services. The RFP for this procurement was almost 1,500 pages in length and contained over 500 mandatory and point‑rated technical requirements. The evaluation consisted of both a “paper” process which evaluated Oshkosh’s bid and physical vehicle testing at the Nevada Automotive Test Center (NATC) facilities which took approximately five months to complete.
26. Similarly, the complaints themselves were complex. In total, there were 9 grounds of complaint, involving 10 point-rated technical requirements, in addition to Oshkosh’s issues regarding PWGSC’s debriefing and record-keeping obligations. Moreover, the methods of evaluation involved multifaceted assessments and often involved exceedingly technical nuances.
27. The confidentiality considerations surrounding these complaints were also unique. As noted in its Confidentiality Guidelines, the Tribunal appreciates that some information in its proceedings may be commercially sensitive and could, if disclosed to a business rival, have adverse financial consequences for the parties. Accordingly, the Tribunal affords substantial protection to information designated as confidential. Nonetheless, the Tribunal recognizes that complete and well-documented public reasons are essential to the transparency of the Tribunal’s decision-making process.
28. In the current situation, the submissions and evidence filed by the parties included information which was not only designated as confidential by the parties but also protected under the Defence Production Act.^[7] Due to the exceptionally technical nature of the goods and services being procured and the resulting RFP, the Tribunal’s analysis would not be complete without a discussion and analysis of certain confidential elements of the evaluation process.
29. The Tribunal has made every effort to document the public reasons for its decision to the greatest extent possible. However, in order to ensure the transparency of its reasons, while maintaining the appropriate level of confidentiality, the Tribunal will issue both a public version of its reasons with certain confidential information omitted and a confidential version which is available only to parties with proper access to the confidential record of these complaints.

PROCUREMENT PROCESS

30. The RFP was issued on July 13, 2013, and the solicitation closed on January 14, 2014.
31. The RFP contained over 500 mandatory and point-rated technical requirements that were to be evaluated. Appendix BA to the RFP listed various requirements which would be evaluated. For the point-rated technical requirements, the evaluation was conducted either on the basis of the Proof of Compliance (POC), the Technical Compliance Program (TCP) test protocol, or a combination of both the POC and the TCP.^[8]
32. The POC evaluation was conducted in accordance with section 2.3.3 of the RFP, which provides as follows:

    Mandatory requirements identified as “POC” in the Proposal Compliance Methods column of Appendix BA and its corresponding attachments will be deemed compliant if the Bidder provides the specific information (when applicable) and any other supporting documentation as part of its response which proves to the satisfaction of the evaluation team that the requirement is met and (as applicable) meets the pass/fail criteria defined in Part 4, Attachment 5, Section 2, Schedule 5-2, Appendix 1 for the associated Test Profiles.^[9]

33. For the TCP portion of the evaluation, three vehicles (the Cargo variant, the LHS variant with APS and the LHS trailer) were supplied for physical testing and taken to the NATC site for testing.^[10] PWGSC contracted the NATC to perform the actual vehicle testing.^[11] The testing was conducted according to 12 test profiles, each of which described the physical testing procedures and conditions, the requirements being evaluated and the scoring criteria to be applied. The relevant test profiles will be discussed in the analysis below.
34. The testing was then conducted in accordance with the TCP details, test profiles and testing matrix set out in Appendix 1 to Schedule 5-2, Section 2, Attachment 5 to Part 4 of the RFP, which provides as follows:

    2.6 Technical Compliancy Program (TCP)

    2.6.1 The Bidders will be evaluated on their test articles’ results in the TCP.

    2.6.1.1 Canada will conduct the TCP to verify specific performance requirements. The TCP will consist of a configuration audit^[[12]] and, performance testing and human factors evaluation . . . .

    2.6.1.2 The TCP results will be documented and assessed in accordance with the Test Profiles contained in Part 4, Attachment 5, Section 2, Schedule 5-2, Appendix 1 . . . .^[13]

    Of the elements of the TCP listed in section 2.6.1.1, only the performance test is at issue (i.e. not the configuration audit or human factors evaluation).

35. On January 21, 2014, after the solicitation had closed, PWGSC held a TCP pre-evaluation briefing with bidders which provided an overview of the following:
    • SMP project
    • Canadian procurement
    • Roles and responsibilities
    • Key personnel
    • Communication roadmap
    • Rules of engagement
    • NATC overview^[14]

    The pre-evaluation briefing also included a question and answer period.

36. In the section titled “Conduct of TCP”, the RFP listed a “Testing Outline”, which included the following proposed timeline for the tests:

    3.3.1 TESTING OUTLINE

    . . . 

    a. Weeks 1-3. Performance Tests Preparation Activities

    (1) Initial Briefing

    (2) Initial Preparation of Test Articles (to bring the Test Articles to test readiness)

    (3) Operator/User Training

    (4) Configuration Audit as per Article 4 herein.^[15]

37. Bids were first assessed to determine whether they met all the mandatory requirements of the RFP. After TCP testing was concluded, each bid was then evaluated on the point-rated technical requirements in order to provide an overall score for each compliant proposal.^[16] As set out in the RFP, the responsive bid with the highest overall score, which included both the technical and financial evaluation, was then recommended for contract award.^[17]
38. On July 16, 2015, PWGSC informed Oshkosh that it was not the highest bidder and that the contract would be awarded to Mack Defense.
39. On July 21, 2015, Oshkosh contacted PWGSC to request a debriefing of the evaluation results. PWGSC acknowledged receipt of Oshkosh’s request that same day. However, PWGSC did not contact Oshkosh to schedule a debriefing until August 12, 2015. PWGSC attributed this delay to its need to coordinate the debriefing requests of other bidders.
40. Oshkosh asked PWGSC to provide it, in advance, with copies of the materials that would be presented at the debriefing. PWGSC declined this request, but informed Oshkosh that it could submit written questions after the debriefing.
41. On August 27, 2015, PWGSC conducted a debriefing with Oshkosh.
42. On August 31, 2015, Oshkosh provided a list of written questions to PWGSC.
43. On September 10, 2015, Oshkosh sent a notice of objection to PWGSC, in which it contested the results of the evaluation.
44. On November 9, 2015, PWGSC provided responses to the questions sent by Oshkosh on August 31, 2015.
45. On November 26, 2015, Oshkosh filed a complaint with the Tribunal; however, as PWGSC had not yet provided a response to Oshkosh’s notice of objection, the complaint was premature.^[18]
46. On December 21, 2015, PWGSC provided a response to Oshkosh’s notice of objection.
47. Oshkosh filed its complaint in File No. PR-2015-051 with the Tribunal on January 6, 2016. As indicated above, Oshkosh’s complaint in File No. PR-2015-067 was filed on March 10, 2016.

REQUIREMENTS AT ISSUE

48. Oshkosh alleged that ████ the following point-rated technical requirements were improperly evaluated by PWGSC:

49. Specifically, Oshkosh alleged that the following errors were made in evaluating its bid:

APPLICATION OF THE TRADE AGREEMENTS

Positions of Parties

50. Oshkosh stated that the procurement was subject to the Agreement on Internal Trade,^[20] NAFTA and the World Trade Organization Agreement on Government Procurement.^[21]
51. In its submissions, Mack Defense contended that NAFTA and the AGP did not apply to the RFP. In particular, Mack Defense pointed to Annex 1001.1b-1 and Annex 1001.1b-2 of NAFTA, which provide that military trucks and trailers in Federal Supply Classification (FSC) codes 2320 and 2330, and the provision of services related to such goods, are not covered by NAFTA.^[22] Similarly, Mack Defense indicated that Annex 4 of the AGP excludes military trucks and trailers in FSC codes 2320 and 2330 and Annex 5 excludes all services related to the purchase of such goods.^[23]
52. In its reply, Oshkosh maintained that, in addition to the AIT, which the RFP specifically acknowledged as being applicable, the RFP is in fact subject to both NAFTA and the AGP.^[24] Oshkosh pointed to Chapter Ten of NAFTA, which lists PWGSC and DND as entities of Canada whose procurements are covered by NAFTA. While Section B of Annex 1001.1b-1 of NAFTA excludes “. . . military trucks and trailers in 2320 . . .” purchased by DND, Oshkosh contended that the goods were actually purchased by PWGSC, as the contracting authority. Since the exclusion for military trucks in Section B of Annex 1001.1b-1 of NAFTA applies only to goods purchased by DND and not to goods purchased by PWGSC, Oshkosh submitted that NAFTA is applicable.
53. Oshkosh contended that the RFP is also subject to the AGP for the same reasons as those explained in relation to NAFTA.^[25]
54. Finally, even if neither NAFTA nor the AGP applied, Oshkosh noted that the RFP would still be covered by the AIT, which also imposes an obligation to maintain a complete record of the procurement process and evaluation.

Analysis

55. All the parties agree, and the Tribunal is satisfied, that the solicitation is covered under the provisions of the AIT. While it is therefore not strictly necessary for the Tribunal to determine whether NAFTA and the AGP apply to this solicitation,^[26] the Tribunal will nonetheless comment on Oshkosh’s contention in relation to the applicability of NAFTA and the AGP.
56. The Tribunal has repeatedly held that, when considering the application of the relevant trade agreements, the government institution acting as the contracting authority may simply be acting as an agent for another government institution. In such cases, the determination of the relevant government institution for the purposes of the application of the trade agreements will depend on factors such as the department that requires the actual goods or services, that drafted the specifications, that conducted the evaluation and that will pay for the work and other statements in the solicitation document indicating that the contracting authority is acting on behalf of another government institution.^[27]
57. In reviewing the RFP, the Tribunal finds that there are multiple provisions which make it clear that PWGSC is acting as an agent for DND. For example:
    • Part I, Section 1.2 indicates that the bid solicitation is being issued on behalf of DND;
    • Part I, Section 2.1 states that DND requires the vehicles and related services;
    • Attachment 2 to Part 4, Section 1.1.1 notes that DND will participate in evaluating parts of the bids received; and
    • Part 8, Section 3.12 states that DND will be responsible for prearranging remission on importation or for paying customs duties on importation.

    On balance, these provisions suggest that this RFP is for the provision of vehicles and related services for and on behalf of DND. Thus, it is DND, not PWGSC, which is the relevant government institution when deciding the applicability of the trade agreements. As such, the exclusions in both NAFTA and the AGP apply, and the solicitation is not subject to either of those trade agreements.

58. Nevertheless, as the AIT does apply to this solicitation. The Tribunal will therefore proceed to determine whether each of the grounds of complaint raised by Oshkosh is valid under the applicable provisions of the AIT.

ANALYSIS

59. Subsection 30.14(1) of the CITT Act requires that, in conducting an inquiry, the Tribunal limit its considerations to the subject matter of the complaint. At the conclusion of the inquiry, the Tribunal must determine whether the complaint is valid on the basis of whether the procedures and other requirements prescribed in respect of the designated contract have been observed. Section 11 of the Regulations provides that the Tribunal is required to determine whether the procurement was conducted in accordance with the applicable trade agreements, which, in this case, is the AIT.
60. Considering the nature of the allegations raised by the complaints, the obligations contained under Article 506 of the AIT are those that are relevant to the Tribunal’s inquiry. Article 506(6) provides as follows:

    In evaluating tenders, a Party may take into account not only the submitted price but also quality, quantity, delivery, servicing, the capacity of the supplier to meet the requirements of the procurement and any other criteria directly related to the procurement that are consistent with Article 504. The tender documents shall clearly identify the requirements of the procurement, the criteria that will be used in the evaluation of bids and the methods of weighting and evaluating the criteria.

61. In cases where the application of Article 506(6) of the AIT is at issue, the Tribunal does not generally substitute its judgment for that of the evaluators, unless the evaluators have not applied themselves in evaluating a bidder’s proposal, have ignored vital information provided in a proposal, have based their information on undisclosed criteria or have otherwise not conducted the evaluation in a procedurally fair way.^[28] In addition, the Tribunal is of the view that the responsibility for ensuring that a proposal is compliant with all essential criteria of a solicitation ultimately resides with the bidder.^[29]

I.             Failure to Consider Information in Bid

Requirement BA-9-5

–           Positions of Parties

62. Requirement BA-9-5 was evaluated on the basis of the POC only and provided as follows:

    The crane should be capable of lifting a 3700 kg load to a height of 6.0 m from the ground level at a reach of 3.5 m from the rear edges of the Vehicle.^[30]

    However, in accordance with a formula set out in the RFP, a bidder could only achieve full marks if its proposal contained sufficient information to demonstrate that its crane could lift a weight of more than 3,700 kg, at a height of 6 m and at a reach of 3.5 m.^[31] The formula in the RFP is as follows:

    Let W = Weight (kg)

    . . . 

    If W > 3700 kg; then weight allotted points = 100%

    Otherwise the weight allotted points will be calculated as follows:

    Weight points = -3000/12 + (1/12) x W^[32]

    When the formula was applied to Oshkosh’s proposal, it received  ████ points out of a maximum 0.7630 points.

63. Oshkosh argued that its bid demonstrated that the proposed Cargo with Material Handling Crane variant was capable of exceeding the requirements necessary to receive full points for requirement BA-9-5. It pointed to three separate figures and a performance chart included in its bid which, it contended, demonstrated this capability.^[33] It argued that PWGSC either applied undisclosed criteria or ignored information contained in Oshkosh’s bid when this requirement was evaluated.
64. For its part, PWGSC contended that Oshkosh’s response to requirement BA-9-5 contained conflicting information, which led the evaluators to conclude that the proposed crane did not exceed the 3,700 kg weight necessary to be awarded full points. In support of this position, PWGSC pointed to Figures 2.2.7-2, 2.2.7.2.2.2.1-5 and 2.2.7.3.2-1 of Oshkosh’s proposal, which, it argued, demonstrated that the crane did not demonstrate that it could exceed the 3,700 kg weight.^[34] As such, PWGSC argued that Oshkosh could not have been awarded full points for this requirement.
65. In response, Oshkosh asserted that such a reading of its bid ignores essential information or fails to consider the information in context. For instance, in Figures 2.1-15 and 2.2.7.2.2.2.1-1 of its bid, Oshkosh argued that it lists the crane lifting capacity for its vehicle as , while Figure 2.2.7.2 shows a crane lifting capacity in excess of the capacity in requirement BA-9-5.^[35] When read in conjunction with the bid on which PWGSC relied, Oshkosh asserted that its bid demonstrated that its vehicle’s crane lifting capacity exceeded 3,700 kg and that it therefore should have received full points for this requirement.

–           Analysis

66. At issue in the evaluation of requirement BA-9-5 is whether PWGSC reasonably concluded that Oshkosh’s bid demonstrated that its proposed vehicle was capable of lifting a weight of more than 3,700 kg. In conducting its analysis, the Tribunal has repeatedly held that it will not substitute its judgment for that of the evaluators unless it is established that the evaluators have not applied themselves in evaluating a bidder’s proposal or have ignored vital information provided in a proposal.^[36] This includes an obligation that the evaluation of bids be carried out in a transparent, non-arbitrary fashion.^[37] The Tribunal notes that, while there is no general obligation of fairness per se, unfairness may be triggered by a breach of any of the trade agreement obligations noted above.
67. The Tribunal notes that PWGSC conceded that, in both the individual and consensus evaluations for requirement BA-9-5, the evaluators initially concluded that Oshkosh with this requirement.^[38] However, the Technical Score Summary then goes on to show that Oshkosh was only awarded points out of a maximum 0.7630 points.^[39] No explanation was provided in the evaluation materials as to why the evaluators reversed their decision for this requirement.
68. It is well established that evaluators must consider all relevant information provided in a bidder’s proposal and that a breach of the trade agreements will occur when evaluators ignore vital information in a bid.^[40] It is clear from the evidence submitted that Oshkosh’s bid did contain information that could reasonably be interpreted as demonstrating that its vehicle’s crane lifting capacity exceeded 3,700 kg.^[41] For instance, both Figures 2.2.7.2.2.2.1-4 and 2.2.7.2.2.2.1-5 showed the targeted load points of Oshkosh’s proposed vehicle from both the rear and side views, with plotted points representing the weight, height and reach requirements for requirement BA-9-5. As demonstrated below, the graphed lines depicting the capabilities of Oshkosh’s proposed vehicle clearly exceed those points.

    Figure 2.2.7.2.2.2.1-4^[42]

    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████

    Figure 2.2.7.2.2.2.1-5^[43]

    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████

69. Similarly, Figure 2.2.7-2 shows the capacity of Oshkosh’s proposed vehicle on a grid where the “y” axis represents weight and the “x” axis represents reach.

    Figure 2.2.7-2^[44]

    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████
    ████████████████████████████████████

    Once again, the line illustrating the capacity of Oshkosh’s proposed vehicle exceeds the plotted point representing requirement BA-9-5.

70. In addition, the Tribunal has previously found that, in order for a procurement to have been fairly conducted, the evaluation must be transparent and not carried out in an arbitrary manner.^[45] The individual evaluators’ notes show that the evaluators initially found that Oshkosh’s bid with requirement BA-9-5.^[46] Yet, the evaluators subsequently reversed this decision, without any apparent rationale being provided in the evaluation report.^[47] The Tribunal can find no explanation in the evidence as to how the evaluators could have reached such diametrically opposed conclusions based on the same set of information. Absent any explanation as to why the evaluators reversed their decision, the Tribunal agrees with Oshkosh’s contention that the evaluators appear to have ignored information in Oshkosh’s bid which demonstrated that the crane lifting capacity of its vehicle could exceed 3,700 kg.
71. The Tribunal notes that the evaluators could have sought clarification from Oshkosh if they believed the information to be contradictory or unclear, as they had done in previous instances during the evaluation process.^[48] That they chose not to do so makes the reversal of the evaluators’ decision, without any apparent justification, all the more perplexing. The Tribunal has previously held that an evaluator’s determination will be considered reasonable if it is supported by a tenable explanation, regardless of whether or not the Tribunal itself finds that explanation compelling.^[49] This is consistent with Article 501 of the AIT, which outlines the importance of developing “. . . a strong economy in a context of transparency and efficiency.”
72. To this end, the Tribunal has previously held as follows:

    . . . transparency lies at the heart of the procurement regulatory regime, and transparency is equally essential at the stage of publishing the requirements applicable to a solicitation, ensuring that evaluators are provided with guidelines fully consistent with the published criteria, as well as at the stage of providing bidders, and the Tribunal, with meaningful explanations regarding the evaluation.^[50]f

    [Emphasis added, footnote omitted]

73. In the present circumstances, the Tribunal finds that the evaluators failed to provide an explanation for the reversal of their original assessment, beyond a post-hoc rationalization in the GIR, which is not supported by any evidence in the evaluation notes or test reports. PWGSC’s behaviour in evaluating requirement BA-9-5 raises serious concerns about both the manner in which the evaluation process was carried out and the reasonableness of the conclusion. In particular, the Tribunal finds that PWGSC either failed to consider information in Oshkosh’s bid which demonstrated the proposed vehicle could lift in excess of 3,700 kg or failed to provide a transparent evaluation process by not offering any explanation as to why the initial evaluation was reversed.
74. The Tribunal finds this ground of complaint to be valid.

Requirement BA-528

–           Positions of Parties

75. For requirement BA-528, Oshkosh received points out of a maximum 0.5030 points, which was evaluated on the basis of the POC only. Oshkosh did not contend that it should have received full points for this requirement, but it nonetheless argued that its bid established that the vehicle “. . . confirmed a fuller compliance with this requirement . . . .”^[51] Oshkosh argued that its bid demonstrated that its vehicle could achieve a maximum MMP of kPa, as opposed to the kPa on which PWGSC based its evaluation.
76. For its part, PWGSC noted that the RFP provided as follows:

    2.3.6 Should a variant-dependent response be proposed to a rated criterion, the Bidder’s mark will be the one obtained by the lowest scoring solution for that criterion.^[52]

    [Emphasis added]

    In its response to requirement BA-528, Oshkosh’s bid stated that the MMP ranged “. . . from kPa for the Cargo variant to kPa for the MRT variant.”^[53] PWGSC acknowledged that “variant-dependent” should be interpreted as “. . . a response that included different variants of the vehicle that achieved different scoring . . . .”^[54] PWGSC noted that Oshkosh’s bid stated as follows:

    All variants at GVW, with a payload of  █████, have an MMP ranging from  ███ kPA for the Cargo variant to  ███ kPa for the MRT variant.^[55]

    As such, PWGSC maintained that it awarded points for this requirement for the lowest scoring solution proposed by Oshkosh, as required by the RFP (i.e. by awarding points in accordance with an MMP of  ███ kPa, which represented a higher ground pressure than  ███ kPa).^[56] Therefore, it argued that the evaluators awarded the proper number of points for this requirement.

77. Oshkosh stated that it provided data for two CTIS settings ( and) in order to give PWGSC an understanding of the capabilities of its vehicle.^[57] Oshkosh contended that PWGSC misinterpreted article 2.3.6 of the RFP by applying the term “variant” to alleged variants of CTIS settings, when in fact the terms of the RFP made it clear that the provision applied to variants of vehicles. Oshkosh acknowledged, as follows, that the RFP did not contain an express definition of the term “variant”:

    . . . the RFP is clear through its repeated use of the term variant that it refers to the Cargo variants of the MSVS vehicle and the Load Handling System (LHS) variants of the MSVS vehicle.^[58]

    Moreover, Oshkosh submitted that the RFP did not contain any references to CTIS settings being considered variants.

78. Oshkosh stated that, since the RFP did not specify which CTIS was required for this requirement, it was entitled to select the setting that it deemed most appropriate and that PWGSC was required to consider its response on this basis.^[59] By not awarding points on the basis of the response for the setting (i.e. points for kPa instead of points for kPa), Oshkosh argued that PWGSC improperly ignored information within its bid.

–           Analysis

79. As discussed above, the Tribunal will only substitute its judgment for that of the evaluators if it finds that those evaluators have not applied themselves in evaluating a bidder’s proposal, have ignored vital information provided in a proposal, have based their information on undisclosed criteria or have otherwise not conducted the evaluation in a procedurally fair way. However, the Tribunal has also noted that it is incumbent upon a bidder to exercise due diligence in the preparation of its proposal to make sure that it demonstrates compliance with the requirements of a solicitation.^[60]
80. There is no dispute amongst the parties that requirement BA-528 was designed to reward vehicles which achieved lower ground pressure:

    Simply put, the lower the ground pressure exhibited by the vehicle the more the vehicle would score well (on the basis that low ground pressure is indicative of higher mobility).^[61]

    As such, whether Oshkosh was awarded points on the higher or lower kPa level listed in its proposal impacted the number of points that it received.

81. Both parties agree, and a review of the terms of the RFP confirms, that a “variant-dependent response” in article 2.3.6 must reasonably refer to vehicle variants rather than CTIS variants. Thus, for different vehicle variants (LHS, MRT, etc.), the lowest-scoring variant proposed would be the one used to calculate the points awarded. However, the RFP is silent on which CTIS settings should be used or how PWGSC should award points if multiple CTIS settings are included in a bidder’s proposal.
82. Oshkosh argued that PWGSC improperly applied the “lowest variant” provision in article 2.3.6 of the RFP to CTIS settings, as opposed to across vehicle variants. However, in its response to requirement BA-528, Oshkosh included a summary statement which provided as follows:

    POC – Ground Pressure [BA-528]

    All variants at GVW, with a payload of  █████, have an MMP ranging from  ███ kPA for the Cargo variant to  ███ kPa for the MRT variant. The data for our vehicle pressure, as defined in MMP, IAW UK Def Standard 23-6, Issue 4, Clause 20, are provided in Figure 2.2.1.3.4-4.^[62]

    [Emphasis added]

    The above response is variant dependent. That is, the response lists the MMP values achieved for two different vehicle variants (the Cargo variant and the MRT variant). There is no mention of CTIS settings. Moreover, in Figure 2.2.1.3.4-1, titled “MSVS SMP Rated Requirements Performance”, Oshkosh’s bid lists its performance for requirement BA-528 as  ███████████ kPa.^[63] On the basis of these responses alone, the Tribunal finds that PWGSC acted reasonably by awarding points in accordance with the lowest‑scoring vehicle variant ( ███ kPa for the MRT variant).

83. Nevertheless, the Tribunal noted that, at other points in Oshkosh’s bid, it included values for both the and CTIS settings. The data provided in Figure 2.2.1.3.4-4 are listed as “Ground Pressure MMP Table at ” and list MMPs ranging from kPa kPa depending on the vehicle variant.^[64] This is consistent with what it stated in both the summary statement and Figure 2.2.1.3.4-1 discussed above. Its bid then goes on to state that the driver could enhance performance by .^[65] The data provided in Figure 2.2.1.3.4-5 then list the MMP table at setting and list values ranging from kPa to kPa, depending on the vehicle variant selected.^[66]
84. As Oshkosh noted, no CTIS setting was specified for bidders’ responses to requirement BA-528. Since Oshkosh was not required to use the CTIS setting in calculating the MMP for its vehicle, it is not clear why Oshkosh used that data to summarize the capability of its vehicle at two separate points in its response to requirement BA-528 (the summary statement and Figure 2.2.1.3.4-1), if it in fact expected PWGSC to instead use the CTIS setting data to calculate its score. Put another way, since Oshkosh bore the onus of demonstrating how its proposal met the point-rated requirements, it ought to have plainly indicated which information was relevant for the scoring of its bid. The Tribunal finds that PWGSC was entitled to rely on Oshkosh’s own statements that the MMP for its vehicles ranged from kPA to kPa and to award Oshkosh points accordingly.
85. Oshkosh argued that by failing to consider the setting, PWGSC “. . . improperly failed to consider the full functionality of the Oshkosh test vehicles.”^[67] However, Oshkosh provided no explanation as to why PWGSC was not entitled to consider the results of the setting as part of the “full functionality” of the vehicles proposed. Oshkosh’s bid clearly set out a range of kPa levels which its vehicles were capable of achieving, depending on the variant and setting selected. Had Oshkosh considered that it was not necessary for PWGSC to consider the kPa levels achieved at the setting in evaluating this requirement, it need not have included them in its bid.
86. The Tribunal has repeatedly held that it is not required to determine whether PWGSC’s decision was correct in all circumstances, but must simply determine whether or not its evaluation was reasonable.^[68] In the current situation, Oshkosh’s bid stated that the MMP ranged from kPA to kPa.^[69] As its submissions indicate, Oshkosh was aware that there was no required CTIS setting for this requirement and that the higher the kPa, the lower the bidder’s score. Oshkosh cannot now demand that that part of its bid be ignored in order to maximize the points that it could achieve.
87. In light of Oshkosh’s responsibility to exercise due diligence in the preparation of its proposal to make sure that it demonstrates compliance with the requirements of a solicitation, the Tribunal finds that PWGSC’s decision to consider all the information contained in response to requirement BA-528, rather than to arbitrarily confine its evaluation to one of two proposed CTIS settings, was reasonable. The Tribunal finds that this ground of complaint is not valid.

II.           Failure to Notify Oshkosh of Anomalies Observed During Testing

Positions of Parties

88. Oshkosh alleged that, contrary to the provisions of the RFP, PWGSC failed to inform it that its vehicle experienced anomalies during testing and that its vehicles did not perform as expected. Oshkosh stated that it provided a summary of the performance attributes of its SMP vehicles for the MSVS project, which, it argued, demonstrated that it was capable of achieving a higher performance than it attained during TCP testing.^[70] Oshkosh maintained that the failure of its vehicle to achieve these standards during TCP testing indicated that it did not perform “as expected” or experienced “. . . anomalies or observed conditions . . . .”^[71] As such, Oshkosh argued that it should have been informed of this result in accordance with the provisions of the RFP relating to TCP testing and “Aborted runs and retests”.^[72]
89. For its part, PWGSC contended that Oshkosh’s position is based upon a misreading of articles 3.3.1 and 4.4E of the RFP. PWGSC argued that, in order for Oshkosh’s argument to be successful, the term “performance” would have to be read into the provisions of the RFP.^[73] PWGSC submitted that the re-test provisions of the RFP do not contemplate re-tests based on the performance of a vehicle. Rather, PWGSC maintained that re-tests are limited to situations where the vehicle suffered an identifiable, unexpected behaviour, breakdown or fault during the conduct of the test, not to circumstances where the vehicle simply performed poorly.^[74]
90. In addition, PWGSC submitted that the only way in which to “correct” the poor performance of Oshkosh’s vehicle would have been to conduct a re-test of every vehicle that did not achieve maximum performance during a test.^[75] Moreover, Mack Defense argued that, if Oshkosh had been notified of each instance in which its vehicle did not perform up to the expected standards, PWGSC would have been required to violate the provisions of the RFP, which clearly stated that bidders would not be briefed on the performance results of their vehicles.^[76]
91. For its part, Oshkosh argued that, when viewed contextually, the phrase “anomalies or observed conditions” in article 3.3.1 of the RFP has the same meaning as “behave as expected” contained in article 4.4 E(c) of each of the test profile provisions. In particular, Oshkosh alleged that the anomaly or observed condition of the test vehicle in question was that the vehicle did not behave as expected.^[77] As such, Oshkosh submitted that, when read together, both provisions obligated the tester to notify the bidder, allow the correction of any anomalies and undertake a re-test if an anomaly was observed and the vehicle did not behave as expected.^[78] Such an interpretation seems to be premised on the notion that the failure of Oshkosh’s vehicle to achieve the test results that Oshkosh anticipated was a failure to behave “as expected” and, although not explicitly stated, an anomaly.
92. In addition, Oshkosh submitted that re-tests should not have been limited to tests that were aborted; rather, re-tests should have been conducted even if testing was completed, if testers observed a material departure from projected performance capabilities.^[79]
93. In response to the Tribunal questions on this issue, parties provided additional submissions on their interpretation of the terms “anomaly” and “observed conditions”. Oshkosh contended that the RFP included a category of unanticipated behaviour that did not require an observable mechanical fault or breakdown of the test vehicle to trigger PWGSC’s notification and re-test obligation.^[80] In contrast, Mack Defense submitted that an “anomaly” relates to something abnormal when assessed against the usual operation of the vehicle.^[81]

Analysis

94. In order to properly analyze Oshkosh’s contention that PWGSC ignored provisions of the RFP, the Tribunal must properly interpret the meaning of the applicable provisions of the solicitation documents. As the Tribunal has previously stated, the terms of the RFP should be interpreted contextually, in accordance with their ordinary meaning and with a view to the underlying rationale and objectives of the clause in question and the RFP as a whole. Interpretations must remain consistent with the objectives of the procurement mechanism and the broader commercial context.^[82]
95. Two distinct provisions of the RFP require interpretation. Article 3.3.1, “Testing Outline”, provides as follows:

    Conduct of Testing

    . . . 

    Bidders will be advised of any anomalies or observed conditions of their respective Test Articles that were noted during testing.

    Bidders will not be briefed on the results of testing.

96. In addition, article 4.4 E of the RFP provides as follows:

    E. Aborted runs and retests

    The following outlines the different potential situations that will cause the test to be aborted.

    . . . 

    c. Vehicle does not behave as expected or incurs fault, malfunction, or mechanical failure. If the issue is correctable using procedures established within the Bidder’s operator manuals, NATC will correct the issue and the testing will be repeated. If the issue is not correctable by NATC using procedures established within the Bidder’s operator manuals, the Bidder will be notified in writing with a description of the test vehicle behaviour, fault, malfunction, or mechanical failure. The Bidder will be required to confirm to the [contracting authority] representative, within the prescribed timeframe, that: the test vehicle behaviour, fault, malfunction, or mechanical failure will not affect the testing and the Bidder recommends that the test resume; or the vehicle should be returned to the Bidder for maintenance. Under no circumstances will Canada or NATC conduct maintenance on the test vehicle, nor conduct any procedures not established within the Bidder’s operator manuals. Should the Bidder be unavailable for contact or not provide a response within the prescribed timeframe, then the test vehicle will be returned to the Bidder for maintenance. If the test vehicle is return to the Bidder for maintenance, the test will be rescheduled for up to one (1) additional attempt.

    [Emphasis added]

97. As noted above, in its arguments, Oshkosh stated that the phrase “anomalies or observed conditions” in article 3.3.1 of the RFP should be read as having the same meaning as “behave as expected” in article 4.4 E(c) of each of the test profile provisions. When read together, Oshkosh argued that both provisions obligated the tester to notify the bidder if testers observed a material departure from projected performance capabilities, allow the correction of any anomalies and undertake a re-test.^[83] Thus, Oshkosh submitted that both article 3.3.1 and article 4.4E(c) required bidders to be notified when its test vehicle did not perform up to Oshkosh’s own projected performance capabilities for its vehicles.
98. While Oshkosh appears to be conflating the meaning of these two distinct provisions, the Tribunal believes that, in order to properly interpret the meaning of the RFP, the terms of both articles must be interpreted separately before being understood in context and in light of their object and purpose.
99. The parties did not take issue with the proper interpretation of the phrase “incurs fault, malfunction, or mechanical failure”. As such, the Tribunal believes that it is not necessary to analyze the interpretation of those terms.

–           Article 3.3.1 of the RFP

100. To begin, the Tribunal finds that the phrase “observed conditions” in article 3.3.1 of the RFP relates to the physical status of the test vehicle. The Oxford Dictionary defines “condition” as “. . . [t]he state of something with regard to its appearance, quality, or working order . . . .”^[84] Taken in its ordinary sense, whether or not a vehicle may be deemed to be in “working order” is a low threshold. As long as a vehicle is capable of functioning as intended, regardless of whether it achieves optimal performance results, it may be deemed as being in working order. As such, the Tribunal finds that the phrase “observed conditions” refers to a malfunction or other issue which prevents a vehicle from operating, not to the degree of performance achieved by that vehicle. Thus, this provision in article 3.3.1 does not support Oshkosh’s contention that PWGSC was obligated to inform it when one of its test vehicles did not perform as well as Oshkosh anticipated.
101. With respect to “anomaly”, the Oxford Dictionary defines it as “. . . [s]omething that deviates from what is standard, normal, or expected . . . .”^[85] Thus, a plain reading of article 3.3.1 of the RFP suggests that bidders are to be notified of any deviations or abnormal or unexpected conditions noted by the evaluators during testing. As such, nothing in this definition would appear to exclude the possibility that it includes the deviation from standard, normal or expected performance. However, the Tribunal finds that the actual results of the performance tests, regardless of whether those results were in excess of or below the bidders’ expectations, could not have reasonably been interpreted as “anomalies”, considering that the object and purpose of the testing was to evaluate each test vehicle and produce measurable results in terms of performance with the unquestionable intent to compare those results with those of the vehicle of other successful bidders. In that sense, performance results, good or bad, cannot be “anomalies” in circumstances where the tests are designed to compare the relative success of the vehicles proposed by the different bidders in meeting the test.
102. Moreover, article 3.3.1 of the RFP states that bidders will not be briefed on the results of those tests. This sentence provides important textual context for the interpretation of the term “anomalies”. The prohibition against communicating the result of the test indicates the extreme importance of maintaining the objectivity and impartiality of the test. Oshkosh’s position—that article 3.3.1 required bidders to be briefed whenever a test vehicle performed poorly or below the bidders’ projected performance capabilities—would violate the express provisions of article 3.3.1.
103. In support of its position, Oshkosh argued that PWGSC did communicate with Oshkosh “. . . concerning some anomalies, observed conditions or unexpected vehicle behaviours that arose during testing . . . .”^[86] However, an examination of the evidence reveals that these notifications related to requests for maintenance and repair.^[87] Therefore, the Tribunal finds that they do not support Oshkosh’s position, but rather reinforce the conclusion that an “anomaly” or “observed condition” constituted an incapability to function or carry out the testing in question.
104. As such, when the terms of article 3.3.1 of the RFP are given their ordinary meaning and read in context and in light of their object and purpose, the Tribunal finds that they did not require PWGSC to notify Oshkosh of the poor, or below expected, performance of its vehicles during testing.

–           Article 4.4E(c) of the RFP

105. Article 4.4E(c) of the RFP requires a test to be aborted if the vehicle does not “. . . behave as expected or incurs fault, malfunction, or mechanical failure.” The subsequent sentences state that, in the event of such an issue (either the vehicle not behaving as expected or a malfunction), the NATC official was required to consult the bidder’s operator manual to determine if the issue was correctable. The provision makes it clear that it is only if the “issue” is not correctable using procedures outlined in the bidder’s operator manual that the bidder should be contacted.^[88]
106. The Oxford Dictionary defines the term “behave” as follows:

     1 . . . Act or conduct oneself in a specified way, especially towards others:

     . . . 

     1.1 (Of a machine or natural phenomenon) work or function in a specified way.^[89]

     [Emphasis added]

     For its part, “expected” is defined as to “. . . [r]egard (something) as likely to happen . . . .”^[90] While poor performance or performing below the bidders’ projected performance capabilities could possibly constitute something unexpected, when the phrase “behave as expected” is interpreted as a whole, in the context of the applicable provision, it does not support Oshkosh’s position that it ought to include poor performance.

107. The term “behave” refers to working or functioning “in a specified way”. As noted above, the object and purpose of the testing was to evaluate each test vehicle and produce measurable results. Read in this context, the Tribunal finds that the phrase “behave as expected” in this instance refers to the ability of the test vehicles to function as specified in order to carry out the testing in question. The object and purpose of the testing was to evaluate each test vehicle and produce measurable results. Therefore performance results, good or bad, cannot be considered to be unexpected behaviour in circumstances where the tests are designed to compare the relative success of the vehicles proposed by the different bidders in meeting the test.
108. This interpretation is supported by the information given to bidders by PWGSC during the January 21, 2014, pre-evaluation briefing. During that briefing, bidders were reminded of the re-test provisions of the RFP and, in particular, of the issues observable during the conduct of a test for which a re‑test may have been permitted. These situations were described as those where, during a test, a “. . . readily observable electrical or mechanical issue . . .” was observed or where “. . . obviously observable vehicle issues . . .” arose. They included such things as (1) engine warning light – brake warning light, (2) engine oil pressure low voltage, (3) high coolant temperature, (4) shock failure - inoperable ABS, (5) fluid leaks – power loss, (6) cracked suspension components and (7) abnormal mechanical noise.^[91] The examples provided to bidders at the TCP pre-evaluation briefing indicate that the re-test provision was to be triggered only if observable malfunctions were noted and do not support an interpretation of the provision that would equate “behave as expected” with poor performance. As such, this does not support Oshkosh’s position that it was to be informed of the poor performance results of its vehicles during testing.
109. In its complaint, Oshkosh argued that, by providing such information in a presentation after bid closing, PWGSC was attempting to modify the terms of the RFP without a formal amendment.^[92] However, Oshkosh cannot ignore that it was made aware of this issue on January 21, 2014, when the information was presented to all bidders. If Oshkosh felt that the information presented was an “unauthorized” attempt to modify the terms of the RFP, or if the examples given did not correspond with how it believed the provision should be interpreted, it was incumbent on Oshkosh to raise those concerns when it became aware of them. Given that Oshkosh did not raise this potential ground of complaint until its comments on the GIR, the Tribunal finds this argument to be time-barred.
110. The Tribunal is of the opinion that Oshkosh’s interpretation with respect to article 4.4E(c) of the RFP, if accepted, would lead to an absurd result which, in terms of interpretation, must be avoided. Firstly, article 3.3.1, which sets out the conduct of testing and, therefore, must be read in context with article 4.4E(c), explicitly states that bidders will not be briefed on the results of testing. Oshkosh’s contention that it was required to be notified when its vehicles did not perform as well as Oshkosh anticipated would clearly violate the prohibition against bidders being informed of the results of testing.
111. Secondly, Oshkosh’s interpretation would have required NATC officials to have a detailed description of the expected performance capabilities of Oshkosh’s vehicle for each test profile. They would have been required to immediately check the results of each testing run against Oshkosh’s expected performance. This would have been extremely burdensome and, for some test profiles, it may not have been possible. For instance, the testing procedures for the speed on grade test included the following:

     The results will be plotted on a speed versus grade plot and a linear trend line applied to the data. The speed on a 2 percent grade and the maximum grade at 80 km/h will be determined based on the trend line. If the results [of] either evaluation are outside the minimum or maximum speed or grade load identified above additional runs will be conducted to verify the vehicle capability.

     Maximum speed will be plotted as a function of the drawbar load for each vehicle. A regression line will be applied to the points and used to interpolate the maximum speed at exactly two percent (2%) representative drawbar as well as the longitudinal grade at which the maximum vehicle speed is exactly 80 km/h.^[93]

112. The Tribunals notes that it is unclear how NATC officials could have processed the data collected, done the necessary calculations, produced the results and then compared those results to the information in Oshkosh’s bid after each and every test run. Such a process would have been highly impractical, would likely have brought the evaluations to a virtual standstill and jeopardized the ability of NATC officials to complete the physical testing in the time allotted. In addition, the wording of the above-noted test procedures also calls Oshkosh’s interpretation of the re-testing provisions into question. The speed on grade test procedures explicitly state that re-tests will be conducted on the basis of performance results for specific situations (i.e. if the results are outside the minimum or maximum speed or grade load identified). If, as implied by Oshkosh’s argument, re-tests were to be conducted for each instance in which the vehicle did not perform up to the bidder’s expectations, the re-testing provision in the speed on grade test procedures would have been unnecessary and redundant.
113. Finally, the Tribunal notes that Oshkosh gives no explanation as to how a failure to achieve maximum points could be “corrected” by consulting the bidder’s operator manual. Instead, Oshkosh’s suggested interpretation would have required testers to inform the bidder that a vehicle did not meet the bidder’s own performance expectations and to have permitted the vehicles to be re-tested. The Tribunal finds that such an interpretation cannot stand, as it runs contrary to article 3.3.1 of the RFP, which provides that bidders would not be informed of the results of testing.
114. Having examined the terms of article 4.4E(c) of the RFP contextually, in accordance with their ordinary meaning and with a view to the underlying rationale and objectives of the clause in question and the RFP as a whole, the Tribunal finds that they do not support Oshkosh’s contention that PWGSC had an obligation to inform it that its vehicles did not achieve the performance capabilities predicted by Oshkosh during the vehicle testing.

–           Conclusion

115. In light of the foregoing, the Tribunal finds that PWGSC did not breach the provisions of articles 3.3.1 and 4.4E(c) of the RFP. The Tribunal finds that this ground of complaint is not valid.

III.          Failure to Properly Configure CTIS Settings

Overview

116. Oshkosh received out of 2.1067 points for both requirement BA-644 and requirement BA-668. Both requirements were evaluated on the basis of TCP. The drawbar pull requirement was tested through test profile 4.7 “Fine Grained Soil Tractive Effort”, while the sand gradeability requirement was tested through test profile 4.8 “Sand Dune Maximum Gradeability”. Oshkosh alleged that PWGSC used the incorrect CTIS settings when testing its vehicle for these requirements.
117. In support of its argument, Oshkosh noted the following provisions of the RFP:

     5.3.3 Additional Information

     The Bidder should also provide the following information with the Bid, or upon request by the [contracting authority]:

     . . . 

     c. a settings checklist (e.g. ride height setting, CTIS setting), with associated step-by-step procedures, for vehicle settings that are applicable to each TCP Test. The settings and procedures must be consistent with settings that would be employed during operation of the vehicle and must be able to be completed from CREW positions.^[94]

     . . . 

     A. Vehicle Configurations

     This test will be conducted with the following vehicle configuration:

     1. LHS variant at GVW

     Tire pressures and adjustable configuration such as transfer case setting, ride height, differential locks will be set as required. Any other adjustable settings will be set as recommended by the Bidder on his submitted settings checklist IAW Part 3, Attachment 3, Section 2, paragraph 5.3.3(c).^[95]

     [Italics in original, underlining added for emphasis]

     With respect to the proper CTIS setting for the drawbar pull and sand gradeability requirements, Oshkosh confirmed that only the sentence stating “[t]ire pressures and adjustable configuration such as transfer case setting, ride height, differential locks will be set as required” was in issue.^[96] As such, the question for both requirements was whether the CTIS configurations were set “as required”.^[97]

118. In its bid, Oshkosh provided the following information for both requirements:^[98]

     Vehicle Configuration: LHS variant at GVW

     8,000 kg payload

     No Trailer

     Settings:  ██████████

      ██████████████

      ████████████████

     Oshkosh argued that PWGSC used the setting rather than the setting described in its bid and that the performance of its vehicle was therefore negatively impacted. In doing so, Oshkosh maintained that PWGSC used undisclosed criteria (namely, a configuration setting that was different from the one specified by Oshkosh) to evaluate its bid.

119. In response, PWGSC argued that the following points were in issue:
     • Oshkosh’s objection on this ground was not timely;
     • PWGSC had the discretion to determine the proper CTIS configurations for both the drawbar pull and sand gradeability requirements; and,
     • Oshkosh had instructed NATC officials to use the  ███ configuration during bidder-led training.

     Each of these points is analyzed separately below.

Timeliness of Objection

–           Positions of Parties

120. PWGSC stated that, in its pre-evaluation briefing to bidders on January 21, 2014, officials specifically informed bidders that the “Fine Grained Soil Tractive Effort Test” for the drawbar pull requirement would be conducted with bidders’ vehicles set to the MSS setting.^[99] Given that Oshkosh did not object to this information, or the use of the MSS setting, until its complaint to the Tribunal in File No. PR-2015-051, PWGSC argued that this ground of complaint was not only without merit but also not timely.^[100]
121. While Oshkosh did not deny that the pre-evaluation briefing did contain a slide stating that the MSS setting would be used for the drawbar pull requirement, it noted that this was a “. . . single line item in a 75‑page slide, which was orally presented to bidders after bid closing.”^[101] Oshkosh argued that this information amounted to an attempt to modify the conditions of the RFP “. . . without a formal amendment to the solicitation documents.”^[102] Moreover, even if this slide could be used to explain the MSS setting for the drawbar pull requirement, Oshkosh maintained that it could not be applied to the settings used for the sand gradeability requirement.

–           Analysis

122. During the TCP pre-evaluation briefing to bidders on January 21, 2014, bidders were given information on configurations and settings which was different from the provisions contained in the RFP. Whereas the TCP provisions for the “Fine Grained Soil Tractive Effort” stated that CTIS settings would be set “as required”,^[103] during the briefing, a slide labelled “Fine Grained Soil Tractive Effort” stated as follows:

     Procedure Reference

     • MTP 2-2-619
     • TOP 2-2-604
     • Mud/Sand/Snow inflation pressure only^[104]

     [Emphasis added]

     Thus, in at least one instance, bidders were notified that the MSS setting would be used for the drawbar pull requirement (as noted above, the “Fine Grained Soil Tractive Effort” test was used to evaluate the drawbar pull requirement).

123. At the very least, these two responses, when considered together, may have created an ambiguity. The Tribunal has repeatedly held that, where a patent ambiguity is apparent on the face of the tender or related documents, a complainant must either object to the government institution or bring a complaint to the Tribunal.^[105] As the slide in question clearly stated that the MSS setting was to be used, by January 21, 2014, Oshkosh was aware of PWGSC’s intention to use this setting for the drawbar pull requirement.
124. While it is not entirely clear, Oshkosh appears to argue that the information in question, being a “. . . single line item in a 75-page slide . . .”,^[106] was too obscure or understated to constitute proper notice to bidders. When combined with the response given during the question and answer period on January 21, 2014,^[107] the information does seem to be confusing. As set out in its submissions, this information clearly contradicted Oshkosh’s own understanding of the RFP and test processes; however, Oshkosh did not object or seek clarification until over a year after the presentation was made.^[108] The Tribunal has made it clear that, where uncertainty exists, it is incumbent upon the bidder to seek clarification.^[109] While Oshkosh may have overlooked the information on the slide at the time of the presentation, this does not constitute a sufficient rationale for failing to object or seek clarification in a timely manner.
125. In addition, Oshkosh argued that, by providing such information in a presentation after bid closing, PWGSC was attempting to modify the terms of the RFP without a formal amendment.^[110] This argument is subject to the same timeliness concerns discussed above. Even if the Tribunal were to accept that the information provided in PWGSC’s slide was an attempt to modify the RFP without an amendment, Oshkosh was made aware of this issue on January 21, 2014. Given that Oshkosh did not raise this potential ground of complaint until its comments on the GIR, it is not possible to consider it as timely.
126. In light of the foregoing, the Tribunal finds that Oshkosh’s ground of complaint regarding the CTIS setting used for the drawbar pull requirement is not timely.
127. However, this finding does not impact Oshkosh’s contention that the incorrect CTIS setting was also used for the sand gradeability requirement. PWGSC argued that there was a “clear implication” that the MSS setting would also be used for the sand gradeability requirement.^[111] However, it provided no evidence or rationale for this contention. In fact, unlike the slide describing the “Fine Grained Soil Tractive Effort” test, the “Procedure Reference” listed on the slide titled “Sand Dune Maximum Gradeability” does not list a CTIS setting.^[112]
128. As no statement was made during the January 21, 2014, TCP pre-evaluation briefing to bidders regarding the CITS setting to be used for the sand gradeability requirement, it is not clear why PWGSC believes that parties should have known that the settings listed for an entirely different test profile would also be applied to the sand gradeability requirement. Therefore, the Tribunal finds that PWGSC’s arguments regarding timeliness do not apply to the sand gradeability requirement.

Configurations to be set “as required”

–           Positions of Parties

129. Oshkosh submitted that the RFP must be interpreted to mean that it, as the bidder, had the responsibility of determining the appropriate CTIS setting for the sand gradeability requirement. In particular, Oshkosh pointed to the fact that PWGSC chose to pre-determine the settings to be used for certain tests (i.e. 4.2 “Mission Profile” or 4.4 “Ride Quality”)^[113] in the RFP but not for the test for the sand gradeability requirement. Oshkosh contended that this implied that PWGSC would not be solely responsible for determining the settings to be used, which would be consistent with the contra proferentem rule.^[114] Oshkosh submitted that the settings were required to be set by Oshkosh, as detailed in the settings checklist submitted with its bid.^[115]
130. In response, both PWGSC and Mack Defense argued that the evaluators had the discretion to configure the CTIS in accordance with the normal operation expected during a testing scenario.^[116] They argued that, when read in context, the vehicle configuration provision in question could not have meant that bidders were responsible for establishing the required setting in their bidder checklists.
131. In particular, Mack Defense argued that the language for “Vehicle Configuration” for the test profile made it clear that there were two categories of adjustable settings. The first category (CTIS, ride height, etc.) was to be set “as required”, while the second category (“any other adjustable settings”) would be set in accordance with the bidder’s recommendations in its checklist. Mack Defense maintained that Oshkosh’s proposed interpretation is unreasonable, as it would fail to give different meanings to the use of different terms in adjacent sentences.^[117]
132. Moreover, PWGSC argued that it was the Crown and NATC engineers who had the responsibility of determining which setting was required for the test. PWGSC pointed out that every other test profile clearly set out which CTIS setting would be used. This, in PWGSC’s contention, was indicative that all TCP testing would be carried out using the CTIS setting determined by the Crown and NATC engineers.^[118] PWGSC maintained that this was further demonstrated by the January 21, 2014, TCP pre-evaluation briefing to bidders, which contained a slide specifically stating that the MSS setting would be used for the “Fine Grained Tractive Effort” test.^[119]
133. Finally, PWGSC maintained that, during the bidder-led driver training, the Oshkosh trainer did not attempt to prescribe a particular CTIS setting that had to be used for any type of terrain.^[120] In fact, PWGSC asserted that the trainer’s evidence confirmed PWGSC’s position that the Crown and NATC engineers had the discretion to determine which CTIS setting was required for each test.^[121]
134. In reply, Oshkosh noted that the RFP did not contain a definition or explanation of what was meant by the phrase “as required”.^[122] However, it pointed to an exchange during the pre-evaluation briefing on January 21, 2014, as follows:

     10) How are vehicle configurations getting set for each test? (i.e. CTIS, ride height, etc.)

     That will be included during the bidder training period; the trainers should let the test vehicle operator know the specifics of the vehicle including CTIS, ride height, etc.^[123]

     Oshkosh contended that this response shows that bidders would be given discretion to choose the proper setting and to communicate this information to NATC officials during bidder-led training.

135. Furthermore, Oshkosh argued that the specific nature of the RFP required most bidders “. . . to reconfigure their standard vehicles to present a unique vehicle for consideration under the RFP.”^[124] Oshkosh pointed to the fact that bidders were required to instruct NATC officials on the proper use of the vehicles, as well as to the fact that bidders were asked to provide checklists (though not required) as further evidence that PWGSC in fact sought bidder input on proper vehicle configuration.^[125] In Oshkosh’s contention, by not pre-determining the settings, PWGSC “. . . engaged the requirement to consult the bidder settings checklist to determine the required setting as dictated by the bidder.”^[126]
136. Finally, Oshkosh submitted that PWGSC’s own arguments in its GIR confirmed Oshkosh’s position. In particular, Oshkosh highlighted that PWGSC relied on the alleged instructions given by Oshkosh officials to NATC officials to use the setting as evidence that the correct setting was used. In Oshkosh’s view, this argument is tantamount to an acknowledgement by PWGSC that it was the bidder, not PWGSC, that had discretion to determine the correct configuration to be used.

–           Analysis

137. As previously stated, the terms of the RFP should be interpreted contextually, in accordance with their ordinary meaning and with a view to the underlying rationale and objectives of the clause in question and the RFP as a whole. They must not be read in isolation, but rather should be understood in harmony with the rest of the RFP, its purpose and objectives, and the broader commercial context.^[127]
138. In applying such an interpretation, the Tribunal finds that the provision cannot be read to mean that the bidder has sole responsibility for determining the appropriate CTIS setting. As noted by Mack Defense, the two sentences forming the provision must be read in context of one another. In using the phrase “[a]ny other adjustable settings will be set as recommended by the Bidder on his submitted checklist . . .”, it is clear that the second sentence in the provision is intended to distinguish those settings from the CTIS settings listed in the first sentence. The provision distinguishes between settings that are to be set by the bidder (the second sentence) from those in the first sentence which are to be set differently. By making such a distinction, it is clear that the drafters did not intend for the bidder alone to determine which CTIS settings would be used. Doing so would improperly conflate the meaning of the two separate sentences and essentially render the first sentence meaningless.
139. In order to understand the meaning of “will be set as required’,’ the broader context of the provision must be considered in light of the object and purpose of the evaluation scheme as a whole. The provision here is part of a complex mechanism that is contained in the TCP section of the RFP. The provision deals with the configuration of vehicles for the purpose of conducting evaluation tests. Given that the provisions for other TCP tests clearly identified the CTIS setting to be used, the use of the language “as required” strongly suggests that PWGSC could not determine the CTIS setting to be used for the sand gradeability requirement when the RFP was issued. Rather, the language implies that other factors or considerations had to be taken into account when determining the proper CTIS setting.
140. What those factors or considerations may be is not clearly identified in the provision. However, the Tribunal recognizes PWGSC’s ability to determine the specific technical requirements necessary to procure highly sophisticated military equipment and, in that respect, finds it reasonable to conclude that the determination of what is “required” under the first sentence is ultimately made by PWGSC. Nonetheless, when considered in the context of the RFP, the Tribunal finds that PWGSC’s contention that the bidder has no role to play in determining the precise nature of these requirements is unreasonable.
141. As indicated above, the RFP suggests that the determination of the required CTIS setting is subject to factors or considerations that were undetermined when the RFP was issued. In that respect, article 5.3.3 of the RFP invited bidders to submit certain information with their bids, including a settings checklist, which would address such things as ride height setting and CTIS setting, as well as “. . . step-by-step procedures, for vehicle settings that are applicable to each TCP Test.”^[128]
142. The RFP specifically contemplated that the checklist referenced above would include CTIS settings to be used during TCP testing, which were undetermined when the RFP was issued. It seems reasonable to consider that the invitation to submit additional information, in the form a settings checklist, was intended to assist evaluators determine the appropriate settings to be used in light of the specific conditions in which tests were to be conducted. Consequently, the Tribunal finds it reasonable for bidders to have concluded that the settings checklist would be used by the evaluators as a means of determining the “required” CTIS setting.
143. The bidder checklist was not the only source of information that PWGSC was required to consider when determining the appropriate CTIS setting. PWGSC’s response to question 10 during the pre‑evaluation briefing on January 21, 2014, demonstrates that information given during bidder-led training would also be considered. In particular, trainers were encouraged to inform test vehicle operators, during the bidder training period, of the specifics of the vehicle, including such things as CTIS and ride height.^[129] Given that the RFP requested that bidders provide a checklist and that this checklist specifically included CTIS settings, the Tribunal finds that it was also relevant to determining which CTIS setting was required.
144. Having received the checklist in response to the request in the RFP, it was incumbent on PWGSC to take the information contained therein into consideration. If PWGSC did not intend for the information provided in response to article 5.3.3 of the RFP to play a role in the determination of which CTIS setting was required, it should clearly have indicated this in the RFP. Instead, it left the matter open for interpretation by the bidder. Oshkosh’s expectation that the checklist would be considered in determining the requirement for the CTIS setting was therefore entirely reasonable in light of the terms of the RFP.
145. The evidence submitted makes it clear that PWGSC did not consider the information provided by Oshkosh in its bidder checklist, nor did it ensure that NATC officials took the information provided therein into account when determining the CTIS setting to be used for the sand gradeability requirement. Instead, testers focused on the information provided to NATC officials during bidder training in order to determine the appropriate settings for those that fell within the “as recommended” category.^[130] While the information provided during bidder training was relevant, it should have been considered together with the information provided in the settings checklist. As the Tribunal has previously held, evaluators have a duty to thoroughly and diligently review proposals and to consider all relevant information contained therein.^[131]
146. As such, the Tribunal finds that PWGSC did not carry out the evaluation for the sand gradeability requirement in accordance with the terms of the RFP, since it did not consider information in the bidder checklist provided by Oshkosh. Therefore, the Tribunal finds that this ground of complaint is valid.

Information Given During Bidder-led Training

–           Positions of Parties

147. The following terms of the RFP called on bidders to conduct training sessions in order to instruct NATC officials on the proper usage of their vehicles:

     3.4.1. Performance Test Operator/User Training:

     NATC Test Drivers will operate the Test Articles for the Performance Tests.

     . . . 

     Each bidder will have the opportunity to provide up to 16 hours of Operator/User Training to a group of 6-8 NATC Drivers. Bidders are to ensure that their proposed Operator/User training includes the following:

     Training Outline:

     Classroom (cover the following subjects) – 1 hour

     • Vehicle specification
     • Vehicle features and controls
     • Capabilities review
     • Vehicle limitations
     • Payloading
     • Vehicle configurations for terrain types– Highway, Gravel, Trails, Cross Country, Sand, Mud, Cold and Hot weather.^[132]

     [Emphasis added]

148. PWGSC argued that the CTIS setting for the vehicle was set to .”^[133] In support of this argument, PWGSC pointed to the NATC testing report and submitted a letter written by a representative of the NATC approximately two years after the training occurred, in which the NATC representative stated that he had been told to use the setting for terrain sites with sand and soft soil.^[134]
149. By contrast, Oshkosh stated that it instructed NATC officials to use the setting during bidder training. In support of this contention, Oshkosh submitted an affidavit sworn by one of the Oshkosh employees who conducted the bidder-led training of the NATC officials, in which he explained that bidders were .^[135] He therefore explained as follows:

     25. During the classroom training, we instructed the NATC representatives to use the  █████ CTIS setting when driving in the mud, sand or snow. However, we also instructed the NATC representatives that if they are operating the vehicle in the  █████ CTIS setting and they need more traction,  ███████████████████████████████████

     26. We further instructed the NATC representatives that in situations where they anticipate driving in mud, sand or snow and maximum vehicle traction will be required due to the conditions of the terrain, they should operate the vehicle from the very start in the  ████ CTIS setting and  ████████████████████████████████████^[136]

     Thus, Oshkosh argued that the statements made by the NATC representative in the letter attached to the GIR were inaccurate and that the  ████████ setting should have been used.

–           Analysis

150. While the Tribunal has already concluded that the evaluation of the sand gradeability requirement was not carried out in accordance with the terms of the RFP, an examination of the information given during bidder-led training serves to underscore this finding. When considering the evidence presented by both parties, the Tribunal notes that, while PWGSC filed an unsworn letter containing the thoughts of the NATC official who was present for the bidder-led training, Oshkosh filed a sworn affidavit from its representative who conducted that training. While the evidence presented is not contradictory, the Tribunal is inclined to give more weight to the sworn evidence presented by Oshkosh.
151. As noted by Oshkosh, during bidder-led training, it was not permitted to , but instead could only .^[137] Nonetheless, the evidence presented by Oshkosh demonstrates that it had given instructions to NATC representatives to use the CTIS setting when driving in mud, sand or snow, but to operate the vehicle in when superior traction was required, while driving in mud, sand or snow.^[138] The Tribunal finds that these instructions were consistent with the contents of the bidder checklist submitted by Oshkosh, which listed the appropriate setting for the sand gradeability requirement as .^[139]

Conclusion

152. The Tribunal finds that Oshkosh’s complaint with respect to the drawbar pull requirement is not timely and, therefore, not valid. However, the Tribunal finds that PWGSC did not conduct the TCP testing for the sand gradeability requirement in accordance with the terms of the RFP. As such, Oshkosh’s complaint on that ground is valid.

IV.         Failure to Use a Proper Mobile Dynamometer

Positions of Parties

153. With respect to the drawbar pull requirement, Oshkosh alleged that PWGSC erred by not using a mobile dynamometer in conducting the test. Specifically, Oshkosh argued that PWGSC improperly conducted the test by in place of the mobile dynamometer. Oshkosh maintained that the use of this substitute vehicle meant that the test vehicle could not or maintain .

Analysis

154. At issue before the Tribunal is the question of whether PWGSC conducted the testing in question in accordance with the terms of article 4.7 of the RFP. The relevant portion of article 4.7 provides as follows:

     B. Instrumentation and Equipment

     . . . 

     All data will be collected with a digital data acquisition system at a minimum of 256 Hz. Prior to analog-to-digital conversion, the system will internally filter all data at approximately 100 Hz to prevent aliasing.

     Equipment necessary for this test includes:

     Mobile Dynamometer

     Cone Penetrometer

     Soil Trafficability Measurement Kit

     Tape Measure and One (1) Meter Ruler

     Video Camera

     Digital photographic camera

     A tow vehicle will be used as the mobile dynamometer for the dynamic tractive effort evaluation.

155. While Oshkosh correctly points out that the test profile used for the drawbar pull requirement lists several types of equipment, one of which is a mobile dynamometer, which are necessary for the test, the test profile also clearly provides as follows:

     Canada reserves the right to use alternate test instrumentation and equipment at its discretion.^[140]

     As such, Oshkosh also recognizes that the use of a substitute vehicle in place of a mobile dynamometer for this test was not, in and of itself, contrary to the terms of the RFP.

156. However, Oshkosh goes on to argue that use of the substitute vehicle impaired the effectiveness of the test. Oshkosh maintained that the test had to be carried out in accordance with the test controls set out in TOP 2-2-604, which provides as follows:

     3.3 Test Controls

     f. Tests are conducted at full throttle, with vehicle speeds and gear ranges controlled by varying the applied load with the dynamometer vehicle. The test should be conducted at as wide a speed range as possible within the limitations of the load application equipment. During soft soil testing, the applied load shall be sufficient to produce 100 percent slip or very neat 100 percent slip.^[141]

     [Underlining in original]

     Oshkosh contended that the videos produced by PWGSC show that the test was not conducted with the test vehicle  ██████ and did not  █████████████████. As such, Oshkosh contended that the drawbar pull requirement was not tested in accordance with the requirements of the RFP.

157. While TOP 2-2-604 may be used in several industries to standardize testing operating procedures, a review of the testing procedures for the drawbar pull requirement reveals that it was not incorporated into the terms of the RFP. Although TOP-2-2-604 was listed at the back of the final report as a “Reference Document”, there is no mention of it in the discussion of the evaluation of the drawbar pull requirement, nor is there any indication that it was to be incorporated into the terms of the RFP.
158. In fact, the test procedures for the drawbar pull requirement specifically provide as follows:

     1. The test vehicle will achieve the desired test speed, 5 km/h, prior to entering the test course.

     2. Once the test vehicle and towed load have entered the test area the towed load will increase the draw bar load while the test vehicle increases throttle to achieve various load and the tire slip conditions while maintaining constant speed. A minimum of 3 seconds of constant load, slip and speed are required for each test condition.

     [Emphasis added]

     Contrary to Oshkosh’s contention, the actual provisions specifically contemplate that the test vehicle will increase throttle at various points in the test. Nowhere do they state that the vehicle will operate at . Similarly, while Oshkosh argued that a was required for the test, the provision simply states that constant load must be maintained for a minimum of 3 seconds.

159. In support of its argument, Oshkosh submitted a sworn affidavit by Mr. Daniel N. Binder, in which he discussed what he believed to be essential characteristics of a drawbar pull test. While the Tribunal appreciates that Mr. Binder may have significant experience in the testing of Oshkosh vehicles, the fact remains that the manner in which the vehicles were to be tested was set out in the provisions as the RFP. If Oshkosh had concerns about that testing, or believed that elements such as a or should have been included, it was incumbent on Oshkosh to bring these to PWGSC’s attention when it became aware of them. The Tribunal has repeatedly held that a bidder cannot wait until the results of an evaluation to raise grounds of complaint that should have been apparent on the face of the solicitation documents.^[142]
160. Given the foregoing, the Tribunal finds that NATC officials acted reasonably in carrying out the testing for the drawbar pull requirement. This ground of complaint is not valid.

V.           Failure to Award Points for Performance Achieved

Video of Testing

–           Positions of Parties

161. In its additional submissions of April 15, 2016, Oshkosh argued that the videos reveal that the Oshkosh test vehicle was able to navigate to the top of the sand dune, thereby demonstrating that it was able to meet the maximum grade climbing ability. As such, Oshkosh contended that it should have been awarded the maximum points available for this TCP test.
162. In response, PWGSC argued that Oshkosh’s position misinterpreted the video. It explained that the RFP required the “Sand Dune Maximum Gradeability” test to be performed on “. . . a grade slope which increases 0 to more than 40% slope through a minimum of 50 meters of linear distance . . . .”^[143] Test readings would be taken once the test vehicle became immobilized on the sand dune slope. The test was to be performed three times for each test vehicle, though relevant test profiles noted that additional runs could be conducted to account for inconsistencies in soil, vehicle or measurements.^[144]
163. PWGSC submitted an unsworn letter written by the NATC official responsible for the conduct of the evaluation, in which he explained that the video in fact showed .”^[145] He stated that, as documented in subsequent videos, Test Run 3 . As such, PWGSC maintained that the video did not in fact demonstrate that the Oshkosh test vehicle was able to navigate to the top of the sand dune.
164. For its part, Mack Defense argued that Oshkosh’s submissions were based on the unsupported assumption that the video showed the Oshkosh test vehicle traversing a path that is consistent with the testing protocol for the “Sand Dune Maximum Gradeability” test. It noted that Oshkosh did not provide any explanation as to how its vehicle, which was recorded in the NATC’s final report as becoming immobilized at grades of , and percent, could then traverse a grade in excess of 40 percent. Mack Defense maintained that the only reasonable inference is that the video showed Oshkosh’s vehicle “. . . traversing a path in the test area that was not consistent with the requirements of the Sand Dune Test and BA‑668 . . . .”^[146]
165. In its reply, Oshkosh argued as follows:

     . . . the submissions made by PWGSC and Mack Defense highlight the prejudice created by PWGSC’s and the NATC’s failure to generate contemporaneous records of the TCP testing conducted on the Oshkosh test vehicles.^[147]

     Moreover, Oshkosh pointed out that the handwritten witness forms made on the dates of the tests did not make any mention of an aborted test run or record any reasons why the test run was not scored—nor was this information included in the NATC’s final report.

166. Oshkosh contended that the NATC’s explanation that it assessed the incline and did not realize that the dune is not supported by the video evidence. In particular, Oshkosh pointed to several observations of the physical conditions displayed by the video (i.e. the video being taken from the top of the dune, and tire tracks to the left and right of the Oshkosh vehicle demonstrating that the NATC had recently driven up that exact sand dune) as being inconsistent with the NATC’s stated position that it had only assessed the sand dune from the bottom and had not realized that it had .^[148]
167. Oshkosh submitted that the videos “speak for themselves” and that the Tribunal should conclude that the videos demonstrate that Oshkosh’s test vehicle was able to successfully navigate to the top of the sand dune, thereby demonstrating that it was able to meet the maximum grade climbing ability.

–           Analysis

168. The Tribunal must determine whether PWGSC’s evaluation of the performance of Oshkosh’s test vehicle was in keeping with the language of test profile 4.8 of the RFP. Again, unless the evaluators have not applied themselves in evaluating a bidder’s proposal, have ignored vital information provided in a bid, have wrongly interpreted the scope of a requirement, have based their evaluation on undisclosed criteria or have otherwise failed to conduct the evaluation in a procedurally fair manner, the Tribunal will generally not substitute its judgment for that of the evaluators.^[149]
169. First, it must be noted that there are significant evidentiary issues with the positions put forward by the parties. Both Oshkosh’s and Mack Defense’s arguments deal with inferences and speculation made about the videos. Although PWGSC produced handwritten witness forms and handwritten test notes which were completed when the tests were completed, neither make any mention of an aborted test run or provide any reference to the videos of any such test runs.^[150] However, in video MSVS-4 21805-1227 018.MPG, the videographer does provide some commentary on the test being recorded.^[151] In particular, at the 2:10 mark, while the test vehicle is , he clearly states “”.^[152]
170. This comment, to which Oshkosh did not refer in its submissions, is important in order to view the videos in context. The testing instructions for the “Sand Dune Maximum Gradeability” evaluation stated that the test would be repeated three times, but that “[a]dditional runs may be conducted based on soil, vehicle or measurement inconsistencies.”^[153] If the initial effort at the third test run shown in video MSVS‑3 21805-1227 023.MPG and in the first half of video MSVS-4 21805-1227 018.MPG were valid, there would have been no need to make a second attempt at that test run. It seems reasonable to presume that it was necessary to undertake a second attempt at Test Run 3 due to some inconsistency in the first test attempt. In this respect, Oshkosh is correct in noting that the contemporaneous records do not explain what caused the first attempt of Test Run 3 to be aborted. PWGSC has attempted to correct this deficiency in the evidentiary record by submitting the unsworn evidence of the NATC official.
171. Although it would have been preferable for the NATC official’s evidence in this matter to have been presented by sworn affidavit, the fact that it is unsworn does not mean that it needs to be discounted. Rather, it must be assessed in light of the evidence and arguments presented by the other parties in order to determine the weight to be given to it.
172. While Oshkosh argued that the NATC letter is inaccurate, it made no submissions on the fact that video MSVS-4 21805-1227 018.MPG clearly showed two attempts at Test Run 3, as noted by the videographers recorded statement.^[154] The arguments that it did make regarding the remainder of the videos are quite speculative.
173. For instance, while Oshkosh argued that the videographer at the top of the dune should have been able to assess the slope prior to the test being undertaken, there is no evidence that the individual was qualified or capable of determining the slope with any degree of accuracy. Similarly, for both the additional tracks and footprints that Oshkosh observed on the dune, upon viewing the video, there is no way to conclusively know when they were made, nor whether they were made in the course of testing the Oshkosh vehicle.
174. Oshkosh noted that the “re-test was conducted mere feet away from where the “aborted” test run three was conducted”;^[155] this is consistent with the evidence given by the NATC official. The official stated that, after realizing that the sand dune “. . .  . . .”, he , before the vehicle to try the test again.^[156] Even more importantly, for the second attempt at Test Run 3, conducted “mere feet” from the first attempt, the video shows the Oshkosh vehicle becoming before it reached the top of the dune, which the NATC’s handwritten test notes indicate was at a recorded slope of percent.^[157]
175. Of course, any uncertainty would have been avoided had the NATC officials made a record of why two attempts were made for Test Run 3, and why the first attempt was not scored.^[158] However, on the basis of handwritten test notes, witness forms and videos, together with the unsworn letter presented by the NATC official responsible for the testing, the Tribunal finds on balance that Oshkosh’s allegations are insufficient to demonstrate that NATC officials failed to apply themselves or misapplied the test criteria when determining the points to be awarded for this test.
176. The Tribunal finds this ground of complaint not valid.

Score Awarded

–           Positions of Parties

177. Oshkosh also contended that, even if the test had been conducted properly, PWGSC failed to properly award points for the results achieved by the Oshkosh vehicle:

     From an automotive engineering perspective, a  ███████████████████████████████████████████████████████████████
     █████████████████████████████████████████████████████████████.^[159]

     As such, Oshkosh argued that it should have been awarded full points for sand gradeability.

178. In response, Mack Defense noted that the RFP clearly stated that the vehicle must attain a minimum grade of 30 percent in order to be awarded points. As Oshkosh’s vehicle did not achieve this result, Mack Defense argued that its complaint was not valid on this ground. PWGSC did not make any submissions on this ground.

–           Analysis

179. The RFP stated that, and Oshkosh confirmed in its complaint, in order to earn points for the test, the vehicle needed to achieve a grade measurement of at least 30 percent.^[160] Moreover, as part of the sand gradeability requirement included in requirement BA-668, vehicles were required to ascend sand grades up to 45 percent. In addition, the scoring criteria in the applicable test profile indicated that the scoring would be based on “. . . the best (of 3 attempts) maximum grade attained at the point of immobilization . . . .”^[161] Nonetheless, Oshkosh argued that, since it almost but did not quite reach the 30 percent threshold necessary, it should have received at least some points. In doing so, Oshkosh gave no explanation as to why the clear wording of the RFP—that no points would be awarded for a result less than 30 percent—should be ignored in this circumstance, other than to point out that Oshkosh was close to achieving this result. In short, Oshkosh acknowledged that it did not achieve the necessary result to receive points for this requirement.
180. Oshkosh also contended that there was a greater than percent variability in the test results recorded, which indicated that “. . . from an engineering perspective . . . the test conditions were unstable, flawed and were not resulting in repeatable results.”^[162] However, beyond this general assertion, Oshkosh did not provide any evidentiary basis to support a conclusion that the NATC officials were required to disregard variable test results.^[163]
181. The Tribunal finds that the RFP was clear; vehicles were required to achieve a grade measurement of at least 30 percent and to achieve sand grades of up to 45 percent. In light of the requirements in the RFP, and in light of the fact that Oshkosh failed to achieve a 30 percent grade score, the Tribunal finds that PWGSC acted reasonably in not awarding full points for this test.

VI.         Failure to Consult Vehicle Checklist

Positions of Parties

182. Oshkosh contended that, in accordance with the instructions in the RFP, it included in the cab of its test vehicle a settings checklist for each test vehicle. Oshkosh maintained that the information in its settings checklist was critical to the performance evaluation of its vehicle and, as it formed part of Oshkosh’s bid, had to be considered by the NATC in the evaluation of its proposal. Oshkosh claims that the NATC’s failure to consult the checklist meant that the TCP testing was not conducted in accordance with the RFP.
183. In response, Mack Defense noted that the RFP distinguished between settings that were to be set “as required” (for instance, the CTIS settings discussed above) and those which were to be set as recommended by the bidder in its checklist. Mack Defense asserted that Oshkosh did not actually identify any settings that would fall within the “as recommended” category.
184. In addition, PWGSC argued that it was incumbent on Oshkosh to have brought the checklist to the attention to the NATC personnel during the bidder-led training if Oshkosh believed it to be necessary. Similarly, Mack Defense stated as follows:

     . . . PWGSC sought to ensure that settings falling within the “as recommended” category were appropriately set by allowing bidders to train the NATC testers to operate their respective vehicles.^[165]

     As such, Mack Defense maintained that the comprehensive training provided an opportunity for bidders to ensure that NATC testers were made aware of any settings for the “as recommended” category.

Analysis

185. The issue before the Tribunal is whether it was reasonable for PWGSC not to have consulted the aforementioned checklist in light of the wording of the RFP. The RFP provided as follows:

     Tire pressures and adjustable configurations such as transfer case setting, ride height, differential locks will be set as required. Any other adjustable settings will be set as recommended by the Bidder on his submitted settings checklist IAW Part 3, Attachment 3, Section 2, paragraph 5.3.3(c).^[166]

     [Emphasis added]

     The words “will be” indicate a mandatory requirement. It is the procuring entity’s onus, once it has set down mandatory requirements, to evaluate bids with regard to those requirements, thoroughly and strictly.^[167] As such, NATC officials were required to follow this procedure when configuring vehicles for testing. Moreover, the language of the RFP stated that the checklist referenced above would include CTIS settings to be used during TCP testing. The Tribunal therefore finds it reasonable to conclude that the checklist was intended to be consulted when determining the “required” CTIS settings.

186. However, there is no question that, despite Oshkosh providing a checklist in the cab of its test vehicles, PWGSC failed to consult it. In its submissions, PWGSC suggested that, because Oshkosh had the opportunity to provide bidder-led training to NATC officials, the responsibility to ensure that the checklist was consulted somehow shifted to Oshkosh. Similarly, Mack Defense pointed to the “comprehensive training” which provided bidders an opportunity to “. . . ensure that NATC testers were made aware of any settings falling in the ‘as recommended’ category.”^[168] While Oshkosh could have taken the opportunity to remind NATC officials of the checklist during bidder-led training, PWGSC nonetheless remained responsible for ensuring that the testing provisions of the RFP were properly followed. More specifically, PWGSC remained responsible for ensuring that NATC officials were aware of the existence of the checklist and its intended purpose.
187. If PWGSC’s interpretation were followed, it would mean that Oshkosh’s presence in providing bidder-led training (which presence was required by the terms of the RFP) somehow modified the actual terms of the RFP, without notice being given to the bidders. Such an approach would have been inconsistent with the obligation on government institutions to clearly identify the requirements of the procurement and the criteria that will be used in the evaluation of bids, and the methods of weighing and evaluating the criteria.^[169]
188. By admitting that the checklist was not consulted, PWGSC acknowledged that it did not follow the directives of the RFP. The Tribunal finds that this ground of complaint is valid.

VII.        Failure to Properly Perform the Vehicle Payload Test

Positions of Parties

189. Oshkosh alleged that, but for the errors made by PWGSC with respect to the vehicle payload test in requirement BA-486, Oshkosh would have received an additional points. It argued that it received points for this requirement and stated as follows:

     . . . on the basis that the NATC recorded a maximum grade of  █████ instead of the required 2% at a speed of 80 km/h. But for the  ███ deficit in the Oshkosh scoring, it would have received full points – 5.118 points – for the Vehicle Payload requirement . . . .^[171]

190. As part of this test, the “Speed On Grade” test requirement was whether the LHS variant at GVW-R was able to sustain a speed of 80 km/h or greater with a 2 percent grade drawbar load.^[172] Oshkosh initially argued that NATC officials erred by not using a substitute vehicle instead of a towed dynamometer in order to conduct the “Speed on Grade” test.^[173] However, as was stated earlier, Oshkosh later conceded that the RFP allowed for the NATC to use alternative test equipment in place of a towed dynamometer.^[174] Nonetheless, Oshkosh argued that the equipment used in place of a towed dynamometer was not capable of accurately carrying out the testing procedures established by the RFP.
191. Oshkosh noted that the NATC’s final report incorrectly stated that a towed dynamometer was in fact used to conduct the “Speed on Grade” test. Oshkosh argued that, where substitute equipment was used, it had to be properly recorded in the final report. In Oshkosh’s contention, this misstatement demonstrated the “lack of rigour” brought to the evaluation.^[175]
192. In response, PWGSC stated that the Crown and NATC engineers used their professional judgment, as contemplated by the RFP, to determine that a substitute vehicle was appropriate. PWGSC noted that the substitute vehicle was equipped with a load cell and GPS speed sensor, which generated and captured the necessary data for the required evaluation.

Analysis

193. All parties agree that the terms of the RFP expressly permitted PWGSC to use substitute equipment when conducting the “Speed on Grade” test. The issue is whether the substitute chosen by PWGSC was capable of accurately testing Oshkosh’s vehicle in order to properly evaluate its performance during the “Speed on Grade” test. In order to determine whether this was the case, each of Oshkosh’s allegations regarding the function of the substitute vehicle will be analyzed in turn below.

–           If the NATC used a load cell between the cable (as opposed to a drawbar) and the Oshkosh test vehicle, this could have resulted in multi-directional forces being applied through the cable connection, which would have improperly affected the test results.^[176]

194. This is a highly speculative allegation. The Tribunal has repeatedly held that there must be an evidentiary basis to support a ground of complaint, not simply speculation.^[177]
195. The Tribunal recognizes that there may have been limitations to the information to which Oshkosh had access. For instance, while PWGSC disclosed the video of the “Speed on Grade” test to Oshkosh, it may not have been possible by viewing the video to definitively determine each piece of equipment used. However, while PWGSC confirmed that , it noted that .”^[178] Thus, it appears that NATC officials were cognizant of the potential impact of multi-directional sources and structured the testing in such a way as to discard any test runs which suffered from this inconsistency. When combined with the lack of evidence presented by Oshkosh, this leads the Tribunal to conclude that, on this particular ground, there is no sufficient evidence to demonstrate that the evaluators did not act reasonably in using the substitute vehicle.

–           If the dump truck’s electrical system was used to power the data collection equipment, this could have impacted the test data, since the dump truck’s electrical system could produce both voltage variation and electromagnetic interference.^[179]

196. This ground of complaint is also speculative. Moreover, PWGSC confirmed that the dump truck’s electrical system was not used to power the data collection equipment. Rather, the data acquisition system was installed , and power for the data acquisition system was supplied by .^[180] Thus, the Tribunal finds no merit in this argument.

–           If the NATC used an engine retarder or transmission retarder of the dump truck as the power absorption unit to create a load, this would result in limited incremental drawbar loads with distinct settings rather than an infinitely variable load of a towing dynamometer, thereby limiting the testing effectiveness of the substitute tow vehicle.^[181]

197. In response, PWGSC submitted that, in the NATC’s professional judgment, an was not required for the test since testers used the towed vehicle substitute to to determine test results.^[182] According to NATC officials, this was consistent with standard practice when a substitute vehicle was used in place of a towed dynamometer.^[183]
198. The Tribunal notes that Oshkosh’s argument does not definitively state that NATC officials erred in conducting this aspect of the test. Rather, Oshkosh asserted that if a certain methodology was followed, it may have limited the effectiveness of the substitute vehicle. It is therefore not clear whether this alleged limitation had any actual measurable impact on the outcome of the testing or whether this is simply speculation about a potential, but unconfirmed, consequence. In other words, Oshkosh has not linked this possible limitation to any demonstrable result or actual flaw in the evaluation.

–           The data show that the substitute vehicle does not collect data at each of the specified simulated grade loads.^[184]

199. The RFP describes the steps to be followed in conducting the test as follows:

     1. The towed dynamometer will be hooked up to the test vehicle with a cable and the tension link load cell in line.

     2. The test vehicles and the dynamometer will accelerate to 80 km/h.

     3. Once 80 km/h is achieved, drawbar load to simulate a 2 percent grade will be applied by the dynamometer while the test vehicle operator increases throttle to full pedal travel. Dynamometer load will remain constant and allow the speed of the test vehicle to achieve steady state.

     4. For the duration of the evaluation, the transmission selector will be placed in “drive” and will be allowed to shift automatically through gears.

     5. Drawbar load and GPS speed will be monitored on a digital readout while the test is conducted. Full throttle will be maintained until a sustained speed (3 seconds or more) on a 2% longitudinal grade is reached.

     6. Steps 2-5 will be repeated using approximately 3, 4 and 5 percent grade load.

     7. At each load, the maximum speed will be evaluated in both directions to negate the effects of wind.

     The results will be plotted on a speed versus grade plot and a linear trend link applied to the data. The speed on a 2 percent grade and the maximum grade at 80 km/h will be determined based on the trend line.^[185]

     [Emphasis added]

     The results of this test were then used, in relevant part, to determine whether the LHS variant at GVW-R was able to sustain a speed of 80 km/h or greater with a 2 percent grade drawbar load.^[186]

200. As noted by Oshkosh, the raw test data provided by NATC officials show that the maximum drawbar load grade achieved during the “Speed on Grade” test of the LHS variant at GVW-R was  percent.^[187] Oshkosh argued that this was at least in part because the ballast weight used for the substitute vehicle was not adequate to generate sufficient drawbar pull force for the necessary grade loads required by the test.^[188] By contrast, PWGSC suggested that there was no need to test the LHS variant GVW-R at anything greater than 2 percent, since it was only required to demonstrate whether it could sustain a speed of 80 km/h or greater with a 2 percent grade drawbar load.
201. In the Tribunal’s opinion, this position ignores two key points in the testing procedures, as set out in the RFP. The testing protocol clearly states that, after the testing is completed at a 2 percent grade load, it will be repeated using approximately 3, 4 and 5 percent grade loads. Given that the highest grade load achieved was percent, these steps were clearly not followed. The fact that the data collected did include a simulated 2 percent grade is not sufficient to meet these procedures. Moreover, while PWGSC noted that the language of the RFP required “approximately” 3, 4, and 5 percent grade loads, the Tribunal finds, as a matter of fact, that the percent grade load actually achieved cannot be considered “approximately” 5 percent in the context of these testing provisions. When considered within the applicable language, a difference of percent grade seems to be significant and a percent grade load cannot be considered to be “approximately” 5 percent.
202. PWGSC’s argument ignores the actual methodology that was to be used to determine the test results, as set out in the RFP. According to the provision, after testing the vehicle at 3, 4 and 5 percent grade loads, the results of all these tested grade loads were then to be used to plot a linear trend link, which in turn was to be used to determine whether LHS variant at GVW-R was able to sustain 80 km/h at a 2 percent grade load. By only testing to a maximum percent grade load, PWGSC collected an incomplete data set, contrary to the provisions of the RFP.
203. By finding as it does, the Tribunal is cognizant that Oshkosh’s results for the “Speed on Grade” test would not necessarily have been better if the full range of tests were fulfilled. However, the fact remains that, by not collecting the data required by the testing procedures, PWGSC could not have produced the complete linear trend line described in the RFP and that Oshkosh’s test results were necessarily impacted. Without these higher grade loads being tested and incorporated into the linear trend link, it is not possible for the Tribunal to determine how Oshkosh’s vehicle would have performed.
204. As a result of the foregoing, the Tribunal finds that the substitute vehicle did not achieve, or was not capable of achieving, the necessary grade load percentage in order to properly test Oshkosh’s vehicle. Thus, the NATC was not able to properly carry out the evaluation in accordance with the provisions in the RFP. This ground of complaint is valid.

VIII.      Failure to Conduct a Proper Debriefing

Positions of Parties

205. Oshkosh alleged that PWGSC provided it with an unsatisfactory debriefing of the results of the solicitation. Oshkosh noted that it received one 40-minute verbal debriefing and that it was only provided with one document containing a high-level overview of the scores that it received.
206. Oshkosh pointed to a Tribunal decision^[189] and noted that it was not provided with an explanation as to “. . . how or why these scores were achieved”^[190] [underlining in original] and thus deprived of the opportunity to understand its evaluation scores.^[191] Moreover, Oshkosh argued that PWGSC unduly delayed providing Oshkosh with a meaningful debriefing.^[192] Oshkosh submitted that this delay was prejudicial and inconsistent with the Government’s policies of transparency and fairness in procurement.
207. In response, PWGSC pointed to the scope of the documentary disclosure that it provided to Oshkosh, noting that, in its regret letter, it provided Oshkosh with the name of the winning bidder, the value of the resulting contract and Oshkosh’s technical score. The deck presented at the debriefing provided Oshkosh with a full breakout of its total technical score, including the scores awarded in respect of each test performed by the NATC on Oshkosh’s vehicles. In addition, PWGSC noted that, on November 9, 2015, it provided Oshkosh with detailed answers to its questions following the debriefing.

Analysis

208. The Tribunal has held that the “primary purpose” of a debriefing is to “. . . provide transparency as to the reasons for not selecting the proposal . . .”^[193] so as to enable unsuccessful bidders to determine the nature of their rights.
209. In CGI, the Tribunal held that the contracting authority should address the reasons for not selecting the proposal and the justification for taking those reasons into account. In that case, the government institution was found not to have met the basic requirements of its debriefing obligation, as it had not disclosed the detailed evaluation plan used by the evaluators, including the evaluation criteria, scales, weights and methodology applied.^[194]
210. The Tribunal notes that a large component of the scoring related to the physical conduct of the TCP tests. Several of Oshkosh’s allegations ultimately related to the specific conduct of these tests. As Oshkosh noted, PWGSC initially declined to produce the videos and a significant amount of documents and, finally, did so only in response to a production order by the Tribunal. While the videos and accompanying documents did constitute important evidence for this inquiry, it does not necessarily follow that they were required to be released at the debriefing stage. In fact, given the size and scope of the data involved, it may not have been feasible for PWGSC to do so at that time.
211. In assessing whether PWGSC provided an adequate debriefing and met its obligations under the trade agreements, the Tribunal notes that a review of the information provided by PWGSC during both the debriefing and in the letter dated November 9, 2015, indicates that Oshkosh was given a detailed breakdown of its scoring during the testing and information concerning all testing for which it did not receive successful marks. Although PWGSC did not provide a complete copy of the TCP final report or evaluator notes and scoring sheets, it did provide Oshkosh with extracts of the final report, as well as the evaluation results and evaluation methods used for those tests for which Oshkosh did not receive full marks. In light of the information provided during the debriefing, the Tribunal concludes that Oshkosh had sufficient information to know why its proposal was not selected and the nature of its rights.
212. The Tribunal finds that PWGSC did not breach its obligation to provide an adequate debriefing to Oshkosh. This ground of complaint is not valid.

IX.         Failure to Properly Document Evaluation Process

Positions of Parties

213. Oshkosh noted that PWGSC had not produced any documentation to demonstrate what vehicle configuration was used by the NATC for the “Speed on Grade” testing of Oshkosh’s vehicles.^[195] Oshkosh submitted that PWGSC attempted to fill the evidentiary void created by the lack of test logs or notes for some of the test profiles by providing unsworn correspondence from one of the NATC testers involved in the testing of the vehicle. Oshkosh alleged that, by doing so, PWGSC undermined the Tribunal’s adjudicative process and deprived the evidence of any weight.^[196]
214. PWGSC submitted that the scope of documentary disclosure that it provided was extensive and comprehensive. It drew the Tribunal’s attention to the list of documents that it provided over the course of the proceedings, including but not limited to the debriefing deck, which included a breakout of the total technical score awarded to Oshkosh, the letter dated November 9, 2015, providing Oshkosh with detailed answers to the questions that it had submitted, the NATC’s final report, notes of DND TCP test witnesses, PWGSC’s weekly reporting notes, individual evaluator scoring notes, and NATC videos, photographs and data records of tests of Oshkosh vehicles. In PWGSC’s submission, the documentation, videos and photographs that it provided constituted a complete record of the evaluation process of Oshkosh’s bid and vehicle, as they contained detailed information describing the conditions under which each test was conducted.^[197]
215. PWGSC argued that Oshkosh’s reliance on the Tribunal’s determination in CGI was misplaced. PWGSC noted that, in CGI, the government institution had declined to provide the complainant with any of the basic evaluation documents. PWGSC highlighted that, unlike the government institution in CGI, it had provided Oshkosh with the full results and background information of the technical vehicle evaluation as set out in the final report, as well as with extensive information on the testing program. PWGSC further distinguished the present scenario from CGI by noting that, in that case, the government institution had destroyed the individual scoring sheets from the bid evaluation process.^[198]

Analysis

216. The Tribunal has repeatedly held that the adequate record taking and retention are essential for both the integrity of the competitive procurement system and the Tribunal’s ability to analyze complaints of potential suppliers. In a previous decision,^[199] the Tribunal noted as follows:

     29. Proper record keeping allows the procurement authority to justify its decisions and the reviewing body to determine precisely what has transpired. In addition, it is conducive to fulfillment by a procurement authority of its duty of fairness to all potential suppliers. Finally, proper record keeping contributes to the Tribunal’s ability to fully assess the validity of procurement complaints brought before it. It is therefore incumbent upon procurement authorities to keep proper records.^[200]

     [Footnotes omitted]

     The Tribunal recognized that one of the fundamental purposes of the AIT is to ensure good faith and transparency in the competitive procurement process.

217. Before analyzing this ground of complaint, it is important to note that the obligation to keep adequate records is distinct from the requirement for a government institution to provide an adequate debriefing on the results of a solicitation. While some of the same materials may be used in both processes (i.e. evaluator notes, test reports), each obligation arises at a different point in time and is designed to achieve distinct, albeit related, purposes.
218. Specifically, the debriefing obligation allows a bidder to understand the reasons for which its bid was not selected, with a view of allowing it to determine its rights under the respective trade agreements.^[201] To this end, debriefings often include presentations or documents created by the government institution after the evaluation has been completed, which explain the relative merits of various proposals and the rationale for not selecting a bidder’s proposal. As part of the process, the government institution often includes evaluator score sheets and test reports made when the evaluation occurred. The disclosure of such information is not always mandatory at the debriefing stage and may depend on the particular circumstances of the case.^[202]
219. While the Tribunal has recognized that the debriefing requirement is often a question of substance over form,^[203] the requirement to keep adequate records of the solicitation process is more exacting. The records made during the evaluation process form the core of the evidentiary foundation on which the evaluators base their decision and by which the Tribunal is able to determine the validity of any complaints brought before it. As such, documents such as the debriefing presentation or PWGSC’s November 9, 2015, responses to Oshkosh’s questions, both of which were produced well after the evaluation was completed, cannot meet the obligation of keeping adequate contemporaneous records.
220. In the present case, it must be acknowledged that the solicitation, and the evaluation process for it, was lengthy and complex. As the record for this matter demonstrates, PWGSC, and NATC officials in particular, produced over 10,000 pages of data and test records, in addition to video footage and photographs, documenting the evaluation process. The Tribunal recognizes that PWGSC, and the NATC officials conducting the TCP testing, made a genuine effort to document the evaluation process.
221. Nonetheless, the record shows that there was unevenness and, at times, crucial gaps in the record keeping for this solicitation. For instance, there was no explanation provided as to why the evaluators reversed their initial determination that Oshkosh’s vehicle requirement BA-9-5. As a result, the decision appears to have been made arbitrarily and without any justifiable foundation.
222. Moreover, a key component of both the evaluation process and this inquiry related to the bidder-led training, during which bidders were required to train NATC test drivers on the proper operation of the test articles for the performance tests.^[204] Included in the topics to be covered during the bidder-led training were such critical issues as vehicle specifications, vehicle features and controls, and vehicle configurations for terrain types.^[205] However, PWGSC was not able to point to any documentation of what occurred during this training or what directions were given by Oshkosh officials. Nor were there any records to show the complete configurations (including all adjustable settings) that were used when testing the Oshkosh vehicles.
223. In an attempt to rectify this deficiency, PWGSC filed an unsworn letter by the NATC official responsible for the conduct of the performance testing written approximately two years after the events in question occurred.^[206] While the Tribunal considered this evidence when attempting to reconstruct what had occurred during the bidder-led training, such after-the-fact explanations for what took place during the evaluation process are not an adequate substitute for contemporaneous records. Similarly, while the video of the “Sand Dune Maximum Gradeability” test did provide some context for what happened during that test,^[207] it was not until PWGSC once again submitted an unsworn letter written some two years after the video was made that any explanation was given as to why the first attempt of the test was aborted.
224. Both gaps in the records of the evaluation process significantly impacted the Tribunal’s ability to assess the validity of the arguments before it. The Tribunal is satisfied that it was able to adequately analyze these grounds of complaint by considering what records did exist in combination with the affidavits and letters created by the parties during the course of this inquiry. However, PWGSC was not able to provide any documentation regarding the “additional settings” to be set in accordance with the bidder checklists, as set out in each of the TCP profiles.^[208] As a result of this lack of records, there is no way to determine what additional settings were actually used in conducting vehicle testing. This is a serious deficiency in PWGSC’s record keeping and highlights why such contemporaneous record keeping is vital to an open and transparent procurement process.
225. In addition to the lack of records discussed above, some of the evaluation records that were made were inaccurate or contained omissions. As Oshkosh noted, the handwritten test notes of the “Sand Dune Maximum Gradeability” test did not mention the aborted attempt on Test Run 3, despite it being recorded in the raw video footage of the test. The Tribunal recognizes that it may have been preferable for the NATC officials to have noted the aborted test run in its handwritten test notes. However, the raw video footage of the test formed part of the record produced for this test. Given that the aborted test run was noted on video MSVS-4-21805, it appears that attempts were made to document this aspect of the evaluation process.
226. Similarly, while a substitute was used for both a towed dynamometer and a mobile dynamometer when conducting the drawbar pull and vehicle payload tests, the final report for each of those tests made no mention of a substitute vehicle being used.^[209] However, the video footage of both tests unmistakably shows the substitute vehicles in use.^[210] Again, while it may have been preferable for the final report to have stated that the NATC exercised the option to use substitute vehicles, this inaccuracy was tempered by the actual video footage recorded as part of the evaluation record.
227. In sum, while numerous records were kept during the evaluation process, the Tribunal finds that significant gaps existed, which seriously impacted both the transparency of the procurement process and the Tribunal’s ability to analyze Oshkosh’s complaints. The Tribunal finds that this ground of complaint is valid.

Conclusion

228. As a result of the foregoing analysis, the Tribunal has made the following determination for each of the grounds of complaint:

229. Of the grounds of complaint which the Tribunal has found to be valid, however, the only requirement which it can definitively conclude should have received additional points is for the “Crane Lifting Capacity” evaluated in requirement BA-9-5. Given that Oshkosh’s proposal demonstrated that its vehicle’s crane lifting capacity exceeded 3,700 kg, as noted by the evaluators’ initial conclusion that Oshkosh with the requirements of requirement BA-9-5, Oshkosh should have been awarded full points for this requirement. As Oshkosh received a score of points out of a possible 0.7630 points, the Tribunal finds that it should be awarded an additional points.
230. With respect to the remaining grounds of complaint that the Tribunal has found to be valid, it is not possible to conclusively determine what additional points, if any, Oshkosh should have received. For those grounds, the crux of Oshkosh’s complaint is that the physical testing of its vehicles was improperly carried out.^[211] Although the Tribunal agrees that the tests were not properly conducted, it is not in a position to determine how the performance of Oshkosh’s vehicles would have changed if the tests had been done correctly. While this uncertainty does not impact the validity of Oshkosh’s complaints, it does go to appropriate remedy to be awarded.

Remedy

231. The Tribunal has found Oshkosh’s complaints to be valid in part. Therefore, the Tribunal must consider the appropriate remedy, pursuant to subsections 30.15(2) and (3) of the CITT Act.

–           Positions of Parties

232. In its complaints, Oshkosh requested that the Tribunal recommend a re-evaluation of its bid and that it be awarded the contract if it achieved the highest score following re-evaluation. In the alternative, Oshkosh asked that it be compensated in an amount equal to the profits that it would have earned over the duration of the contract, had it been awarded the contract.^[212]
233. PWGSC contended that a re-testing of the TCP requirements would be unreasonable, as the process is “. . . long finished, and the Bidder test vehicles have been returned to the Bidders . . . .”^[213] PWGSC also submitted that Oshkosh’s allegations that the officials involved in the procurement process “inflicted prejudice” on Oshkosh through their “pattern of conduct” amounted to a request for a remedy of a punitive nature. PWGSC insisted that such allegations were entirely unfounded and amounted to “. . . an unwarranted attack on the Crown, on the individual PWGSC and DND officials responsible for the MSVS process, and the NATC personnel conducting the TCP testing.”^[214]
234. For its part, Mack Defense argued that it would not be appropriate to order a re-testing of Oshkosh’s vehicle and a re-evaluation of Oshkosh’s proposal, as this would give Oshkosh an unfair opportunity to improve its score and an unfair advantage over other bidders.^[215] Moreover, Mack Defense claimed that a re‑evaluation would not be practical, as test vehicles may not have been maintained in the same state in which they were when originally provided for testing.^[216] Mack Defense further stated that it would be inappropriate for the Tribunal to terminate the designated contract, as work has commenced and the contract has been in place for 10 months.^[217]

–           Analysis

235. The Tribunal has a wide degree of discretion in determining what remedy to recommend. In accordance with subsection 30.15(3) of the CITT Act, when recommending a remedy, the Tribunal must consider all the circumstances relevant to the procurement including:
     • the seriousness of any deficiency in the procurement process found by the Tribunal;
     • the degree to which the complainant and all other interested parties were prejudiced;
     • the degree to which the integrity and efficiency of the competitive procurement system was prejudiced;
     • whether the parties acted in good faith; and
     • the extent to which the contract was performed.

236. The point differential between Oshkosh and the successful bidder was points. Oshkosh argued that, but for the alleged evaluation errors by PWGSC, it would have received more than enough points to be deemed the successful bidder. However, as noted above, since the Tribunal is only able to definitively determine that Oshkosh should have been awarded an additional points, it is not possible to conclude that Oshkosh should have been awarded the contract. As a result, it is not appropriate for the Tribunal to recommend that Oshkosh be awarded the contract.
237. Since the errors made in the evaluation process impacted the actual conduct of the TCP tests, the Tribunal finds that the only way to determine what score Oshkosh should have received under proper testing procedures is to conduct physical re-evaluations of the following TCP tests:
     • Soft Soil Mobility – Sand Gradeability (BA-668)
     • Ride Quality RMS (BA-645)
     • Vehicle Stopping Distance (BA-516)
     • Maximum Speed (BA-120)
     • Acceleration Time (BA-514)
     • Gradeability – High Speed (BA-542)
     • Vehicle Payload (BA-486)^[218]

     For all the above TCP tests, the Tribunal finds that any re-testing should be done after consulting the bidder checklist to determine the proper vehicle configurations. In addition, PWGSC should ensure that a proper towed dynamometer, or substitution vehicle, that is capable of generating sufficient drawbar load grade as specified in the “Speed on Grade” test be used for the re-testing.

238. Should the results of the re-evaluation show that Oshkosh would have been the winning bidder, but for the evaluation errors made by PWGSC, the Tribunal recommends that PWGSC compensate Oshkosh for the lost profits that it would have received, had it been awarded the contract. Although Oshkosh had suggested that, in the event that it was found to be the winning bidder, the Tribunal should recommend that the contract awarded to Mack Defense be cancelled, the Tribunal does not believe that would be appropriate in the present circumstances. In particular, the Tribunal notes that the contract with Mack Defense has already been in place for approximately 10 months and that Mack Defense has taken significant steps towards fulfilling the contract.^[219]
239. When determining what remedy is appropriate, the Federal Court of Appeal has stated the following:

     The Tribunal must reconcile the injured bidder’s interest in being adequately compensated for the prejudice suffered with the public interest in having the contract performed as soon as possible and, where possible, without interruption, unless warranted by, inter alia, the other factors listed at subsection 30.15(3) of the Act.^[220]

     In the present instance, the undoubted delay that would result from cancelling the contract and re-awarding after a re-evaluation of Oshkosh’s vehicles will be significant, given the extent to which the contract has been performed. This leads the Tribunal to conclude that cancelling the contract would not be an appropriate remedy.

240. Nonetheless, the Tribunal is mindful of both PWGSC’s and Mack Defense’s arguments that a re‑testing of Oshkosh’s vehicles may not be possible, whether due to the unavailability of the vehicles or the NATC testing facilities, or the challenges posed by re-creating the same testing conditions that existed when the tests originally took place from January to June, 2014. If both PWGSC and Oshkosh determine that re‑testing is not feasible, the Tribunal recommends that PWGSC instead compensate Oshkosh for its lost opportunity. Should the parties be unable to agree on the feasibility of re-testing, they should make submissions to the Tribunal outlining their respective positions, and the Tribunal will issue further direction.
241. Finally, the Tribunal does not believe that the circumstances of this case warrant a recommendation of punitive “damages”. While there were significant shortcomings in the evaluation process, the Tribunal finds no evidence that either PWGSC or NATC officials acted in bad faith, or intentionally engaged in a pattern or conduct designed to “inflict prejudice” on Oshkosh in the evaluation of its bid.

DETERMINATION

242. Pursuant to subsections 30.15(2) and (3) of the CITT Act, the Tribunal recommends that PWGSC conduct a re-evaluation of the bid submitted by Oshkosh, including a physical re-evaluation of the technical compliance program test protocols performed by the NATC. If the re-evaluation demonstrates that Oshkosh would have been the winning bidder but for the breaches of the applicable trade agreements, the Tribunal recommends that PWGSC compensate Oshkosh for the profits that it would have received, had it been awarded the contract. The Tribunal will not recommend that the contract awarded to Mack Defense be cancelled.
243. The Tribunal recommends that PWGSC and Oshkosh determine the feasibility of a physical re‑evaluation and report back to the Tribunal within 15 days of the issuance of the statement of reasons for this determination.
244. In the event that a physical re-evaluation is no longer feasible at this time, the Tribunal recommends that PWGSC compensate Oshkosh for its lost opportunity. In that circumstance, the Tribunal recommends that the parties negotiate the amount of compensation to be paid for lost opportunity and report the outcome of the negotiations to the Tribunal within 30 days of the issuance of the statement of reasons for this determination.
245. Should the parties be unable to agree on the amount of compensation, Oshkosh shall file with the Tribunal, within 40 days of the issuance of the statement of reasons for this determination, submissions on the issue of compensation. PWGSC will then have 7 working days after the receipt of Oshkosh’s submission to file a response. Oshkosh will then have 5 working days after the receipt of PWGSC’s reply submission to file any additional comments.

COSTS

246. At the outset of these reasons, the Tribunal noted that these complaints involved an extremely complex procurement process for the acquisition of highly specialized goods and services. The solicitation called for the provision of SMP vehicles for the MSVS project in five variants and associated services. The RFP for this procurement was almost 1,500 pages in length and contained over 500 mandatory and point‑rated technical requirements. The evaluation consisted of both a “paper” process which evaluated Oshkosh’s bid and physical vehicle testing at the NATC facilities which took approximately five months to complete. There can be no question that the procurement was extremely complex.
247. Similarly, the complaints themselves were also complex. In total, there were 9 different grounds of complaint, involving 10 different point-rated technical requirements, in addition to Oshkosh’s complaints regarding PWGSC’s debriefing and record-keeping obligations. While only 5 of these 9 grounds of complaint were found to be valid, the errors that the Tribunal identified in the evaluation process impacted all but one of the point-rated technical requirements. Moreover, the methods of evaluation involved multifaceted assessments and often involved exceedingly technical nuances. As such, the Tribunal finds that the complaints were unusually complex.
248. Finally, the proceedings themselves were complicated. The intervention request by Mack Defense was accompanied by several rounds of submissions regarding the access limitations necessitated by the application of the Defence Production Act. In addition, the proceedings were moved to the extended 135‑day timeline in response to PWGSC’s request for an extension of time to file its GIR. These occurrences in themselves would be sufficient to elevate the proceedings to a higher level of complexity. However, the Tribunal also wishes to comment on PWGSC’s actions which, it finds, both prolonged and created difficulties during these proceedings.
249. To begin, the Tribunal notes that it was only in response to the Tribunal’s order of February 19, 2016, made pursuant to Oshkosh’s motion, that PWGSC produced a substantial amount of highly relevant documentation and evaluation records. PWGSC indicated that it had fully complied with this order when it filed its documents on February 24, 2016, and did so again on March 4, 2016, when asked by the Tribunal to confirm that it had produced all documents included in the Tribunal’s order.^[221] However, despite these assurances, it was not until April 4, 2016, that PWGSC produced additional photographs, videos and data which pertained directly to the issues at hand. Given that this subsequent production was not made until after both Mack Defense and Oshkosh had filed their comments on the GIR, additional submissions were required to address the previously undisclosed evidence.
250. The Tribunal is aware that the record for this procurement process is copious, containing as it does raw data, handwritten evaluation notes, photographs, videos, and both draft and final reports. However, the size of the evidentiary record does not excuse parties from fully complying with production orders. Although the Tribunal recognizes that PWGSC took steps to rectify its error once it had been identified, the significant delay and additional submissions that resulted undoubtedly added to the complexity of this proceeding.
251. In addition, PWGSC’s failure to keep adequate records of the evaluation process had a direct impact on the complexity of these proceedings. As noted above, PWGSC attempted to rectify the gaps in its records by submitting unsworn letters, written approximately two years after the events in question, to describe what had occurred. Not only did this make it difficult for the Tribunal to fully analyze the validity of the evaluation process but, given the lack of contemporaneous records, Oshkosh was required to produce sworn affidavits in an attempt to provide evidence as to what had actually occurred. This was an additional burden which was the direct result of PWGSC’s lack of adequate record keeping.
252. Pursuant to section 30.16 of the CITT Act, the Tribunal awards Oshkosh its reasonable costs incurred in preparing and proceeding with the complaints, which costs are to be paid by PWGSC. Given the extraordinary level of complexity of these complaints, the Tribunal exercises its discretion to deviate from its Procurement Costs Guideline. The Tribunal will inform the parties of the schedule for submissions on costs at a later date.
253. The Tribunal reserves jurisdiction to establish the final recommended remedy and amount of cost award.