File No. PR-2018-017 Critical Software S.A. v. Department of Public Works and Government Services

Determination and reasons issued Monday, December 3, 2018

 

TABLE OF CONTENTS

DETERMINATION  i

STATEMENT OF REASONS  1

SUMMARY OF COMPLAINT  1

PROCEDURAL BACKGROUND  1

PRELIMINARY MATTERS  3

The Intervener Request  3

Critical Software’s Requests for Additional Disclosure from PWGSC  4

ANALYSIS  5

Ground 1: The conclusion that Critical Software’s bid was non-compliant with mandatory criterion M-2 was reasonable  6

Ground 2: The W3C validation service demonstration was not required for the technical evaluation  9

Conclusion  12

COSTS  12

DETERMINATION  12

APPENDIX  14

RELEVANT TERMS OF THE SOLICITATION  14

 

IN THE MATTER OF a complaint filed by Critical Software S.A. pursuant to subsection 30.11(1) of the Canadian International Trade Tribunal Act, R.S.C., 1985, c. 47 (4th Supp.);

AND FURTHER TO a decision to conduct an inquiry into the complaint pursuant to subsection 30.13(1) of the Canadian International Trade Tribunal Act.

BETWEEN
CRITICAL SOFTWARE S.A.	Complainant
AND
THE DEPARTMENT OF PUBLIC WORKS AND GOVERNMENT SERVICES	Government Institution

DETERMINATION

Pursuant to subsection 30.14(2) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal determines that the complaint is not valid.

Pursuant to section 30.16 of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal awards the Department of Public Works and Government Services its reasonable costs incurred in responding to the complaint, which costs are to be paid by Critical Software S.A. In accordance with the Procurement Costs Guideline, the Canadian International Trade Tribunal’s preliminary indication of the level of complexity for this complaint case is Level 1, and its preliminary indication of the amount of the cost award is $1,150. If any party disagrees with the preliminary indication of the level of complexity or the preliminary indication of the amount of the cost award, it may make submissions to the Canadian International Trade Tribunal, as contemplated in Article 4.2 of the Guideline for Fixing Costs in Procurement Complaint Proceedings. The Canadian International Trade Tribunal reserves jurisdiction to establish the final amount of the cost award.

 

 

Tribunal Panel:	Randolph W. Heggart, Presiding Member
Support Staff:	Laura Little, Counsel
Complainant:	Critical Software S.A.
Counsel for the Complainant:	Ranald McGillis
Government Institution:	Department of Public Works and Government Services
Counsel for the Government Institution:	Susan Clarke
	Ian McLeod
	Roy Chamoun
	Kathryn Hamill
Intervener:	EMS Technologies Canada Limited
Counsel for the Intervener:	Louis Sévéno
	Marie-Pier Cloutier

Please address all communications to:

The Registrar
Secretariat to the Canadian International Trade Tribunal
333 Laurier Avenue West
15th Floor
Ottawa, Ontario  K1A 0G7

Telephone: 613-993-3595
Fax: 613-990-2439
E-mail: citt-tcce@tribunal.gc.ca

 

STATEMENT OF REASONS

[1]  On July 27, 2018, Critical Software S.A. (Critical Software), of Coimbra, Portugal, filed a complaint with the Canadian International Trade Tribunal (the Tribunal) under subsection 30.11(1) of the Canadian International Trade Tribunal Act [1] concerning a procurement (Solicitation No. W8474-17SM06/B) by the Department of Public Works and Government Services (PWGSC) on behalf of the Department of National Defence (DND). The solicitation was for the design, assembly, delivery and associated maintenance and support of a Search and Rescue Mission Management System Application.

[2]  The Tribunal decided to accept the complaint for inquiry on August 2, 2018, pursuant to subsection 30.13(1) of the CITT Act and in accordance with the conditions set out in subsection 7(1) of the Canadian International Trade Tribunal Procurement Inquiry Regulations. [2]

[3]  The Tribunal conducted an inquiry into the validity of the complaint as required by sections 30.13 to 30.15 of the CITT Act. For the reasons that follow, the Tribunal finds that the complaint is not valid.

SUMMARY OF COMPLAINT

[4]  Critical Software alleged that PWGSC improperly concluded that its bid did not demonstrate “experience successfully developing and deploying national interactive HTML5 WEB based applications” as per mandatory technical evaluation criterion M-2 of the Request for Proposal (RFP) (“ground 1”). In particular, Critical Software’s bid was found to be non-compliant with the requirement to provide a user distribution percentage by region for each of its two reference projects. In its submission, the requirement was ambiguous and irrelevant for the purposes of meeting the M-2 criterion and, therefore, it should not have been included in the RFP.

[5]  In addition, Critical Software alleged that PWGSC did not follow the technical bid evaluation process set out in the RFP in relation to mandatory criterion M-2 because it did not use the W3C validation service demonstration to test bid compliance with that criterion (“ground 2”). Critical Software submitted that the winning bidder was incapable of compliance with the M-2 criterion and, therefore, it should not have been recommended for an award of contract.

[6]  As a remedy, Critical Software requested compensation for its lost profits in relation to the contract award, as well as reimbursement of its bid preparation and complaint costs.

PROCEDURAL BACKGROUND

[7]  The RFP was issued by PWGSC on October 13, 2017, with a bid closing date of November 27, 2017. The bid closing date was later extended to December 18, 2017. Five bids were received, including one from Critical Software.

[8]  The technical evaluation of bids began on January 8, 2018. The W3C validation service was not used by the evaluators to assess compliance of bids with mandatory criterion M-2. [3] The technical evaluation process is discussed in more detail below, to the extent that it is relevant to the Tribunal’s analysis of the above-mentioned grounds of complaint. The technical evaluation was completed and, on February 8, 2018, the signed consensus evaluation sheets were provided from DND to PWGSC. [4]

[9]  On June 28, 2018, PWGSC notified Critical Software that a contract had been awarded to MDA Systems Ltd. (MDA). PWGSC indicated in its letter that Critical Software’s bid had been deemed non‑compliant with mandatory criterion M-2 of the RFP and, as a result, it was disqualified from the evaluation process and did not receive a technical score. [5]

[10]  On June 29, 2018, Critical Software requested a debriefing with PWGSC. At the debriefing held on July 18, 2018, Critical Software objected to the award of contract on the basis of the above-mentioned grounds of complaint. PWGSC undertook to review and follow-up with Critical Software regarding the concerns raised with respect to the W3C validation service demonstration in relation to mandatory criterion M-2. PWGSC further indicated to Critical Software, however, that even if its bid had been found to be compliant with all the mandatory requirements of the RFP, and had it received a perfect score on the technical rated criteria, it still would not have won the contract. [6]

[11]  On July 19, 2018, PWGSC advised the winning bidder, MDA, that “as per section 4.2(d) of the RFP” it wanted to conduct a W3C validation service demonstration in relation to mandatory criterion M-2. [7] The demonstration was conducted on July 27, 2018, and it was determined to be successful. [8]

[12]  On July 27, 2018, Critical Software filed its complaint with the Tribunal.

[13]  On August 2, 2018, PWGSC advised Critical Software that the results of the W3C validation service demonstration in respect of mandatory criterion M-2 had confirmed that MDA was in full compliance with the mandatory requirements of the RFP. [9]

[14]  On August 3, 2018, the Tribunal informed the parties that it had accepted the complaint for inquiry.

[15]  On September 4, 2018, the Tribunal received a letter from EMS Technologies Canada Ltd. (EMS), another bidder in the solicitation process in question, seeking leave to intervene in this inquiry. The parties filed comments on the request on September 7, 2018.

[16]  On September 14, 2018, the Tribunal granted limited intervener status to EMS and revised the schedule for submissions by the parties. As a result of the intervener’s participation and additional filing deadlines, the Tribunal extended the deadline for the issuance of its determination on the validity of the complaint to 135 days, pursuant to section 12 of the Regulations.

[17]  PWGSC filed its Government Institution Report (GIR) on September 13, 2018, and EMS filed its intervener’s submission on October 2, 2018. Critical Software filed its reply on October 12, 2018.

[18]  PWGSC and Critical Software filed an additional round of submissions on October 12 and 15, 2018, respectively. In its submission of October 15, 2018, Critical Software also requested that PWGSC be required to provide further information in relation to certain exhibits filed with the GIR. That request is addressed as a preliminary matter below.

[19]  On October 22, 2018, EMS filed a supplemental submission. [10]

[20]  The Tribunal did not consider an oral hearing necessary and therefore made its determination based on the written record.

PRELIMINARY MATTERS

The Intervener Request

[21]  In its request for intervener status dated September 4, 2018, EMS sought to make representations on the following matters:

• The ability of the successful bidder to provide the required system because, according to EMS, MDA does not appear to have any demonstrable prior experience with this type of requirement, and this Search and Rescue management requirement does not appear to be an area where MDA has participated previously.
• Based on EMS’s extensive experience with the current system, it believes that the project price provided by the successful bidder is significantly lower than is practical for the scope of effort set out in the RFP.

[22]  As indicated above, the Tribunal granted limited intervener status to EMS. In its letter to parties dated September 14, 2018, the Tribunal noted that, in conducting the inquiry, it must limit its considerations to the subject matter of the complaint, pursuant to subsection 30.14(1) of the CITT Act.

[23]  The Tribunal did not consider EMS to be in a position to assist in resolving ground 1 of the complaint, as it relates to the evaluation of Critical Software’s bid. The Tribunal, therefore, granted EMS limited intervener status for the purpose of making submissions in relation to ground 2 of the complaint only. To the extent that EMS’s request referred to matters that are outside the scope of ground 2 of the complaint, such as the project price of the successful bid, the Tribunal advised EMS that such matters could not be considered in the present inquiry.

[24]  In its comments on the GIR, EMS alleged that the winning bidder was non-compliant with a standard clause incorporated by reference in the RFP, requiring that the experience requirements of the solicitation be satisfied by the bidder alone (as opposed to a parent company or affiliate). In the Tribunal’s view, that allegation clearly falls outside the scope of the complaint and, therefore, the present inquiry. Specifically, ground 2 deals with the alleged non-compliance of the winning bidder with the M-2 criterion, and does not refer to the experience requirements found elsewhere in the solicitation documents. Therefore, the allegation raised by EMS will not be addressed in the Tribunal’s analysis. [11]

Critical Software’s Requests for Additional Disclosure from PWGSC

[25]  Critical Software requested, in its submission filed on October 15, 2018, that PWGSC be required to provide further information in relation to certain statements made in the GIR and exhibits filed thereto. In particular, the GIR indicated that on July 20, 2018, a member of the technical evaluation team wrote to the contracting authority for PWGSC advising that three of the five bids received (including Critical Software’s bid) did not include sufficient information to allow for the conduct of the W3C validation service demonstration, whereas the winning bid did include sufficient information. [12] As indicated above, a demonstration was subsequently conducted with the winning bidder, MDA, on July 27, 2018, and it was determined to be successful.

[26]  Critical Software took issue with the evidence that only the winning bidder was compliant with the requirement to include in its bid what was needed to complete the W3C validation service test. According to Critical Software, that statement is factually incorrect and it sought further information from PWGSC in this regard.

[27]  The Tribunal did not consider it necessary to obtain further information on this point given its determination, for the reasons provided below, that PWGSC had reasonably concluded that Critical Software’s bid was non-compliant with the M-2 criterion and that the evaluation of that requirement did not require the conduct of a W3C validation service demonstration.

[28]  In its comments on the GIR, Critical Software requested that PWGSC provide additional disclosure with respect to the technical evaluation process that was described in the GIR. In particular, contrary to what was stated in PWGSC’s rejection letter dated June 28, 2018, [13] the GIR indicated that the evaluators proceeded to the evaluation of the technical rated criteria before concluding their final evaluation of the technical mandatory requirements of the RFP. [14] Critical Software raised a concern that this approach “would introduce bias towards the top ranked technical solution and would influence the re-evaluation of the mandatory requirements.” It further submitted that PWGSC was inappropriately involved in the conduct of the evaluation process, contrary to its own policy, namely, section 5.40 of the Supply Manual, which states that the client department (in this case, DND) is responsible for the technical evaluation of bids.

[29]  PWGSC, as the contracting entity in this solicitation, bears the responsibility of committing Canada to any resultant contract from the process and, also, of ensuring that the procurement process, including the technical evaluation, has been carried out in accordance with any applicable trade agreement. As such, communication with the technical evaluation team from DND during the evaluation process is not unusual and does not automatically constitute a violation of the provisions of the RFP or the applicable trade agreements. In the present case, the written communications between PWGSC and the technical evaluators included discussions about the technical evaluation, how it was conducted or the results. The Tribunal finds no indication that those communications constitute a violation of the RFP terms or the applicable trade agreements.

[30]  In terms of the RFP itself, section 4.2 sets out four distinct parts of the technical evaluation. Those are: a) Mandatory Technical Criteria, b) Point-Rated Technical Criteria, c) Reference Checks, and d) Demonstration. Although it would seem logical for efficiency purposes that an evaluation would follow the order listed, neither that section nor section 4.4 (“Basis of selection”) expressly state that the evaluation of the mandatory technical requirements must be completed before assessing the rated technical criteria. Contained within the wording of c) Reference Checks and d) Demonstration, there are express terms indicating that these elements are discretionary and that any reference checks would only apply to “bidders who have not, at that point, been found non-responsive” and that any demonstration would only be applicable to “the top-ranked Bidder (identified after the financial evaluation)”. This implies that these elements would logically occur after the evaluation of mandatory and point-rated technical requirements.

[31]  Finally, the Tribunal was not provided with any evidence to support the claim that the sequence of an initial evaluation of the mandatory criteria and then an evaluation of the rated criteria in this case introduced an inherent bias into the evaluation process. In this regard, it is noted that the names of bidders were not provided to the evaluators; they were identified by a letter only. [15] Furthermore, the circumstances in evidence before the Tribunal did not raise any reasonable apprehension of bias.

[32]  As a result, the Tribunal did not consider it relevant or necessary for the purposes of its determination to request that PWGSC provide further information with respect to the sequence of the technical evaluation process, as requested by Critical Software.

ANALYSIS

[33]  Subsection 30.14(1) of the CITT Act requires that, in conducting an inquiry, the Tribunal limit its considerations to the subject matter of the complaint. At the conclusion of the inquiry, the Tribunal must determine whether the complaint is valid on the basis of whether the procedures and other requirements prescribed in respect of the designated contract have been observed.

[34]  Section 11 of the Regulations specifies that the Tribunal must determine whether the procurement was conducted in accordance with the requirements set out in the applicable trade agreement, the World Trade Organization Revised Agreement on Government Procurement. [16]

[35]  In the present case, the Tribunal must decide the validity of the following grounds of complaint:

1. whether it was reasonable for the DND technical evaluation team to conclude that Critical Software’s bid did not meet mandatory technical requirement M-2, particularly the provision of customer references for a minimum of two projects that include “[u]ser distribution percentage by region” for each project; and
2. whether the technical evaluation of bids (including the winning bid) was conducted in accordance with the process set out in the RFP in relation to mandatory requirement M-2.

Ground 1: The conclusion that Critical Software’s bid was non-compliant with mandatory criterion M-2 was reasonable

[36]  The applicable trade agreement provides that a bid must, at the time of opening, comply with the essential requirements set out in the tender documentation to be considered for an award. [17] In addition, a procuring entity must award contracts in accordance with the criteria and essential requirements specified in the tender documentation. [18]

[37]  In assessing whether there has been a breach of the above obligations, the Tribunal typically accords a large measure of deference to evaluators in their evaluations of bids. In general, the Tribunal will only interfere with an evaluation that is unreasonable [19] and will substitute its judgment for that of the evaluators only when they have not applied themselves in evaluating a bidder’s proposal, they have ignored vital information provided in a bid, they have wrongly interpreted the scope of a requirement, they have based their evaluation on undisclosed criteria, or they have otherwise not conducted the evaluation in a procedurally fair way. [20]

[38]  Bidders bear the onus of demonstrating that bids meet the mandatory criteria of a solicitation. [21] In other words, the Tribunal has made it clear that bidders bear the responsibility of preparing their bids diligently in accordance with the instructions in the solicitation and taking care to ensure that the information provided clearly demonstrates compliance.

[39]  In the present case, Critical Software’s bid provided customer references for two projects in response to mandatory requirement M-2. The requirement, which is set out in Attachment 4.1 of the RFP, indicated that the “Bidder must have experience successfully developing and deploying national interactive HTML5 WEB based applications” and stated that, “[t]o demonstrate this experience, the W3C validation will be used . . .” (as will be discussed further below). The requirement further indicated that the “Bidder must also provide customer references for each project (minimum of two projects) the Bidder has successfully developed and deployed national interactive HTML5 WEB based applications servicing at least 1000 users”, with a list of information that each reference “must” include. [22]

[40]  One of the stated requirements mentioned above was to include “user distribution percentage by region” for each customer reference. For Project 1 (“Blue Eye”), Critical Software’s bid addressed this requirement as follows: [23]

This application today has more than 1000 users. Users access the application from:

• three (3) Portuguese Naval Operation Centres;

• the Portuguese National Maritime Authority;

• various Port Authorities; and

• ships at sea (or in port).

With the exception of users accessing the system from Naval ships operating in Portuguese Areas of Interest (deployed in regions such as the Mediterranean Sea, the Gulf of Guinea, the Horn of Africa, etc,) all users access the system from the Portuguese National Region.

[41]  For Project 2 (“Vision Centre”), Critical Software’s bid stated as follows: [24]

This application is used by more than 1000 end-users in 30+ countries in Europe, Asia, Africa and South America.

[42]  In the initial evaluation of Critical Software’s bid against the mandatory technical requirements, the evaluators determined that it was non-compliant with M-2 (“no distribution by percentage for Project 1 and 2”). [25] In fact, all five bids had been deemed non-compliant by the evaluators at that stage. [26]

[43]  In a re-evaluation recommended by the contracting authority (PWGSC), the evaluators determined that Critical Software remained non-compliant in respect of mandatory requirement M-2, with a consensus note indicating that this conclusion was reached in relation to Project 2 only. [27]

[44]  In relation to Project 1, the evaluators were “able to understand both distribution (the Portuguese National Region) and the percentage (100%, because all users were in one region).” [28] Accordingly, this requirement was found to have been met on the basis that, although no user distribution by percentage was explicitly stated in Critical Software’s bid, the evaluators were able to determine that 100 percent was in a single region, namely, Portugal. Conversely, in respect of Project 2, although the bid identified regions in its response, it was not possible, without speculation, for the evaluators to infer from this response the percentage distribution of users to be attributed to each region. [29]

[45]  Critical Software admitted (in its complaint) that its bid did not provide user distribution percentages, by region, for either of the two projects it submitted in response to mandatory criterion M-2. Nevertheless, it submitted that this was an “ambiguous and irrelevant sub component” of the criterion. [30] In particular, Critical Software argued that the user distribution requirement was unclear because the term “region” was not defined in the RFP. In its comments on the GIR, Critical Software further submitted that the requirement to provide a user distribution percentage by region was not a legitimate mandatory evaluation criterion.

[46]  The Tribunal points out that, if Critical Software was uncertain as to the meaning of the term “region” or the manner in which it was to respond to M-2, it should have raised such concerns with PWGSC prior to bid closing and/or filed a complaint with the Tribunal within the time limits prescribed under section 6 of the Regulations. [31] Likewise for its assertion that the user distribution percentage by region was an irrelevant or illegitimate requirement. Such allegations are now time-barred and, therefore, cannot be considered by the Tribunal in these proceedings.

[47]  Furthermore, PWGSC/DND was under no obligation to seek clarification from Critical Software regarding its bid response to M-2. Even if clarification had been sought, the evaluators would not have been able to take any information into account if it would have effectively altered the bid in a material way (i.e. amounted to bid repair), which is strictly prohibited. [32] Although the evaluators were able to infer, in relation to Project 1, that 100 percent of the users were in Portugal, that inference was made on the basis of the information that was already provided in the bid. [33] The Tribunal finds that a similar inference could not reasonably have been made in respect of Project 2 on the basis of the information provided in the bid.

[48]  In light of the above, the Tribunal finds that it was reasonable for PWGSC to conclude that Critical Software’s bid did not fully satisfy the M-2 criterion.

Ground 2: The W3C validation service demonstration was not required for the technical evaluation

[49]  The AGP requires that a procuring entity award contracts in accordance with the mandatory requirements specified in the RFP. Accordingly, the Tribunal must consider whether the evaluation team breached that obligation by not conducting the W3C validation service demonstration as part of the evaluation of mandatory criterion M-2. Contrary to Critical Software’s submission that the validation process was an essential requirement for compliance with the M-2 criterion, PWGSC’s position is that the validation process was optional pursuant to the provisions of the RFP. Accordingly, the question is whether PWGSC and/or the evaluators wrongly interpreted the scope of the mandatory requirements in relation to criterion M-2.

[50]  As indicated above, the M-2 criterion stated that, “[t]o demonstrate this experience, the W3C validation will be used . . . .” This phrase, read in isolation, could potentially be misunderstood to mean that bidders could expect as part of the evaluation process that PWGSC would conduct a demonstration using the W3C validation service to verify the compliance of bids with the M-2 criterion. [34] PWGSC submitted that this phrase must be read together with paragraph 4.2(d) of the RFP, which specifically describes the use of the W3C validation service to demonstrate compliance with the M-2 criterion and describes it as a discretionary process. The Tribunal’s interpretation of that phrase is that during a demonstration, PWGSC would be obligated to use a particular method, the “W3C validation service” to verify compliance. This is a self-imposed obligation that makes it transparent to bidders how their proposals would be evaluated should a demonstration be conducted. 

[51]  Is the demonstration of M-2 discretionary? The technical evaluation procedures set out in section 4.2 of the RFP distinguish between the evaluation of bids for compliance with the mandatory requirements (under paragraph “a”) and the demonstration of that compliance, including in respect to the M-2 criterion and, specifically, its W3C validation service demonstration process (under paragraph “d”). The latter process is described as follows:

(d) Demonstration

Canada may, but will have no obligation, to require that the top-ranked Bidder (identified after the financial evaluation) demonstrate any features, functionality and capabilities described in this bid solicitation or in its bid, in order to verify compliance with Mandatory requirements M2 and M3 of this bid solicitation. To demonstrate M2, the Bidder must make available to DND staff a copy of their proposed application or modules described in M2 online via the internet. DND will use the W3C Markup Validation Service (https://validator.w3.org/#validate by uri+with options) to validate that up to four (4) of the Bidder’s randomly selected web pages generated by the HTML5 WEB based application described in M2 are valid HTML 5 pages. The result must indicate complete compliance with the HTML5 standard. The Bidder’s proposed application or modules described in M2, as well as the W3C Markup Validation Service will be accessed via the internet. The demonstration will be conducted by DND staff located in Ottawa. Bidders must provide off-site support via telephone and/or on line communication in support of the demonstration. No on-site support is required from the Bidder. To demonstrate M3, . . . [d]espite the written bid, if Canada determines during a demonstration that the Bidder’s proposed solution does not meet the mandatory requirements of this bid solicitation, the bid will be declared non-responsive.

[Emphasis added]

[52]  In the Tribunal’s view, as argued by PWGSC, the above provision when read with the M-2 criterion stipulates the discretionary nature of the demonstration of compliance through the W3C validation service. This is supported by the fact that the M-2 criterion provided the same description of the information on which the validation would be based as paragraph 4.2(2), with additional details in the latter.

[53]  Furthermore, reading those two provisions together indicates that the procuring entity, on behalf of Canada, “may, but will have no obligation” to use the W3C validation service to conduct a demonstration of compliance with the M-2 criterion. Paragraph 4.2(d) further specifies that the conduct of such demonstration would be in respect of the top-ranked bidder, which means that it would have to occur after the technical and financial evaluations were completed. [35] As also indicated in paragraph 4.2(d), if Canada exercised its discretion to require such demonstration by the top-ranked bidder and this demonstration showed that the bidder’s proposed solution did not meet the mandatory requirements of the solicitation, then, “despite the written bid”, the top-ranked bid would be declared non-responsive.

[54]  Both Critical Software and EMS submitted that paragraph 4.2(d) of the RFP should, at best, be read as granting PWGSC the discretion to conduct an additional demonstration by the top-ranked bidder, should it choose to do so. In their view, there was still a requirement for an initial demonstration by all bidders pursuant to the terms of the M-2 criterion.

[55]  While the reference to a W3C validation service demonstration contained within the M-2 criterion description could have been made clearer by repeating the discretionary nature of the demonstration process, as indicated above, a contextual reading of the terms of the RFP as a whole supports the interpretation that recourse to the W3C validation service was an option available to PWGSC, should it choose to ask for a demonstration of compliance of the top-ranked bidder identified following the technical and financial evaluation of the bids. As such, PWGSC was under no obligation to require a demonstration during the technical evaluation of each bid under the M-2 criterion.

[56]  In the alternative, both Critical Software and EMS have argued that the terms of the RFP with respect to the demonstration process were ambiguous and that this vitiated the entire evaluation process. To support its argument that the expectations related to this requirement were unclear, EMS referred to questions regarding the M-2 criterion from potential bidders that were published in various amendments to the RFP, the technical bids of Critical Software and MDA, the technical evaluation grids and internal correspondence of government officials with respect to the demonstration that was conducted with MDA following the award of contract. [36]

[57]  The Tribunal has carefully reviewed the documentation on the record, including that which was specifically cited by EMS in this regard, and finds no support for the assertion that the demonstration process set out in the RFP was a matter of uncertainty among potential bidders, the procuring entity or the evaluators. In particular, the questions from potential bidders referred to by EMS did not raise any ambiguity with respect to the W3C validation service demonstration process relating to the M-2 criterion. The Tribunal has not been presented with any evidence that Critical Software or another potential supplier raised questions or concerns during the solicitation process with respect to the RFP provisions relating specifically to ambiguity in or confusion about the demonstration process, or the fact that no such demonstration was conducted during the technical bid evaluation. On the contrary, the evidence shows that Critical Software first raised its concerns in this regard to PWGSC at the debriefing held on July 18, 2018 (i.e. following the award of contract).

[58]  As indicated above, PWGSC undertook to conduct a demonstration of MDA’s bid subsequent to the debriefing with Critical Software. In this regard, the record does include an e-mail from a DND official to the contracting authority at PWGSC seeking clarification as to why the winning bidder was being requested to complete a demonstration and how the results of such demonstration might affect the resulting contract. However, that e-mail expressly indicated that the DND official seeking such clarification was neither involved in the RFP nor did he have knowledge of the details of the solicitation. [37] Accordingly, the Tribunal does not consider this to be indicative of any lack of understanding of the optional demonstration process set out in the RFP by either the procuring authority or the evaluators.

[59]  As a result, the Tribunal is satisfied that the demonstration referred to in the M-2 criterion had to be read together with the complete procedure for such demonstrations under paragraph 4.2(d) of the RFP, and that, when read as a whole, the RFP did not require a W3C validation service demonstration by MDA or any other bidder during the technical evaluation process. Consequently, the Tribunal finds that PWGSC and the evaluators correctly interpreted the scope of mandatory criterion M-2.

[60]  In any event, PWGSC has provided evidence that the demonstration conducted subsequent to the award of contract validated MDA’s bid in relation to M-2. With respect to the evaluation that occurred prior to the award of contract and the post-award demonstration, the evaluators’ consolidated comments, as supported by their individual notes, indicated that MDA’s bid was compliant with the M-2 criterion. [38] On the basis of that evidence, the Tribunal sees no indication that the evaluators failed to apply themselves in the evaluation, ignored vital information provided in MDA’s bid, wrongly interpreted the scope of the requirement, based their evaluation on undisclosed criteria, or otherwise conducted the evaluation in a procedurally unfair manner. [39]

Conclusion

[61]  In light of the foregoing, the Tribunal finds that PWGSC was not obliged to conduct a demonstration to verify a bidder’s compliance with mandatory criterion M-2; however, if PWGSC chose to conduct such a demonstration it was mandatory for it to use the W3C validation service to demonstrate that compliance. The Tribunal also finds that the evaluation of that criterion in relation to the bids submitted by Critical Software and MDA, respectively, was reasonable. As a result, the complaint is not valid.

COSTS

[62]  Pursuant to section 30.16 of the CITT Act, the Tribunal awards PWGSC its reasonable costs incurred in responding to the complaint, which costs are to be paid by Critical Software. In accordance with the Procurement Costs Guideline (the Guideline), the Tribunal’s preliminary indication of the level of complexity in this case is Level 1. Although the complexity of the procurement could be considered above Level 1, the Tribunal is of the view that the complexity of the complaint and the proceedings fall within the Level 1 range. In terms of the complaint, it raised relatively straightforward issues that related to a single mandatory technical evaluation criterion in the solicitation documents. The proceedings were extended to 135 days as a result of the intervener’s involvement, and the timing thereof, but otherwise they were not overly complicated and no public hearing was held. As such, the Tribunal’s preliminary indication of the amount of the cost award is $1,150.

DETERMINATION

[63]  Pursuant to subsection 30.14(2) of the CITT Act, the Tribunal determines that the complaint is not valid.

[64]  Pursuant to section 30.16 of the CITT Act, the Tribunal awards PWGSC its reasonable costs incurred in proceeding with the complaint, which costs are to be paid by Critical Software. In accordance with the Guideline, the Tribunal’s preliminary indication of the level of complexity for this complaint case is Level 1, and its preliminary indication of the amount of the cost award is $1,150. If any party disagrees with the preliminary indication of the level of complexity or the preliminary indication of the amount of the cost award, it may make submissions to the Tribunal, as contemplated in Article 4.2 of the Guideline. The Tribunal reserves jurisdiction to establish the final amount of the cost award.

 

APPENDIX

RELEVANT TERMS OF THE SOLICITATION

PART 4 – EVALUATION PROCEDURES AND BASIS OF SELECTION

4.1  Evaluation Procedures

(a)  Bids will be assessed in accordance with the entire requirement of the bid solicitation including the technical and financial evaluation criteria.

(b)  An evaluation team composed of representatives of Canada will evaluate the bids.

(c)  In addition to any other time periods established in the bid solicitation:

(i)  Requests for Clarifications: If Canada seeks clarification or verification from the Bidder about its bid, the Bidder will have 2 working days (or a longer period if specified in writing by the Contracting Authority) to provide the necessary information to Canada. Failure to meet this deadline will result in the bid being declared non-responsive.

(ii)  Requests for Further Information: If Canada requires additional information in order to do any of the following pursuant to the Section entitled “Conduct of Evaluation” in 2003, Standard Instructions – Goods or Services – Competitive Requirements:

(A)  verify any or all information provided by the Bidder in its bid; or

(B)  contact any or all references supplied by the Bidder (e.g., references named in the resumes of individual resources) to verify and validate any information submitted by the Bidder,

the Bidder must provide the information requested by Canada within 2 working days of a request by the Contracting Authority.

(iii)  Extension of Time: If additional time is required by the Bidder, the Contracting Authority may grant an extension in his or her sole discretion.

4.2  Technical Evaluation

(a)  Mandatory Technical Criteria:

(i)  Each bid will be reviewed for compliance with the mandatory requirements of the bid solicitation. Any element of the bid solicitation that is identified specifically with the words “must” or “mandatory” is a mandatory requirement. Bids that do not comply with each and every mandatory requirement will be declared non-responsive and be disqualified.

(ii)  The mandatory technical criteria are described in Attachment 4.1.

(b)  Point-Rated Technical Criteria:

Each bid will be rated by assigning a score to the rated requirements, which are identified in the bid solicitation by the word “rated” or by reference to a score. Bidders who fail to submit complete bids with all the information requested by this bid soli citation will be rated accordingly. The point-rated technical criteria are described in Attachment 4.1.

. . .

• (d) Demonstration

Canada may, but will have no obligation, to require that the top-ranked Bidder (identified after the financial evaluation) demonstrate any features, functionality and capabilities described in this bid solicitation or in its bid, in order to verify compliance with Mandatory requirements M2 and M3 of this bid solicitation. To demonstrate M2, the Bidder must make available to DND staff a copy of their proposed application or modules described in M2 online via the internet. DND will use the W3C Markup Validation Service (https://validator.w3.org/#validate by uri+with options) to validate that up to four (4) of the Bidder’s randomly selected web pages generated by the HTML5 WEB based application described in M2 are valid HTML 5 pages. The result must indicate complete compliance with the HTML5 standard. The Bidder’s proposed application or modules described in M2, as well as the W3C Markup Validation Service will be accessed via the internet. The demonstration will be conducted by DND staff located in Ottawa. Bidders must provide off-site support via telephone and/or on line communication in support of the demonstration. No on-site support is required from the Bidder. To demonstrate M3, . . . [d]espite the written bid, if Canada determines during a demonstration that the Bidder’s proposed solution does not meet the mandatory requirements of this bid solicitation, the bid will be declared non-responsive.

. . .

4.4  Basis of Selection

(a)  Evaluation of Bids

Selection Process: The following selection process will be conducted for each bid:

(i)  A bid must comply with the requirements of the bid solicitation, meet all mandatory evaluation criteria and obtain the required pass marks for the point rated criteria identified in this bid solicitation to be declared responsive.

(ii)  The responsive bid that obtain the highest Total Bidder Score will be recommended for award of a contract. For any given Bidder, the greatest possible Total Technical Score is 60 while the greatest possible Total Financial Score is 40.

(A)  Calculation of Total Technical Score: The Total Technical Score will be computed for each responsive bid by converting the Technical Score obtained for the point-rated technical criteria using the following formula, rounded to two decimal places:

  Technical Score     ×   60 = Total Technical Score

Maximum Technical Points

• (B) Calculation of Total Financial Score: The Total Financial Score will be computed for each responsive bid by converting the Financial Score obtained for the financial evaluation using the following formula rounded to two decimal places:

  Lowest Total Bid Price      ×   40 = Total Financial Score

The Bidder’s Total Bid Price

• (C) Calculation of the Total Bidder Score: The Total Bidder Score will be computed for each responsive bid in accordance with the following formula:

Total Technical Score + Total Financial Score = Total Bidder Score

. . .

ATTACHMENT 4.1

BID EVALUATION CRITERIA

. . .

SECTION 1: MANDATORY EVALUATION CRITERIA

. . .

M-2 WEB Application

The Bidder must have experience successfully developing and deploying national interactive HTML5 WEB based applications.

To demonstrate this experience, the W3C validation service will be used to verify up to four (4) randomly selected pages generated by the application for compliance with the HTML5 standard. The result must indicate complete compliance indicating the criteria has been met.

The Bidder must also provide customer references for each project (minimum of two projects) the Bidder has successfully developed and deployed national interactive HTML5 WEB based applications servicing at least 1000 users. Each reference must include:

1. Application name;

2. Customer name and contact information (including the name and either the telephone number ore-mail address of the organization’s contact responsible for the project;

c.  Start and end date of the project; and

d.  Description of the project, specifically emphasizing the WEB elements (check boxes, buttons, sliders, etc.).

e.  Description of the interactive components used within the application.

f.  User distribution by percentage by region.