APPDEPOT WEB SERVICES INC.
Determinations
APPDEPOT WEB SERVICES INC.
File No. PR-2003-069
TABLE OF CONTENTS
Ottawa, Monday, March 8, 2004
File No. PR-2003-069
IN THE MATTER OF a complaint filed by AppDepot Web Services Inc. under subsection 30.11(1) of the Canadian International Trade Tribunal Act, R.S.C. 1985 (4th Supp.), c. 47;
AND FURTHER TO a decision to conduct an inquiry into the complaint under subsection 30.13(1) of the Canadian International Trade Tribunal Act.
DETERMINATION OF THE TRIBUNAL
Pursuant to subsection 30.14(2) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal determines that the complaint is valid in part.
Pursuant to subsections 30.15(2) and (3) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal recommends, as a remedy, that AppDepot Web Services Inc. be compensated in the amount of $1,830, representing an estimate of one twelfth of the profit that AppDepot reasonably would have made, had it been awarded the contract.
Pursuant to section 30.16 of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal awards AppDepot Web Services Inc. its reasonable costs incurred in preparing and proceeding with the complaint, which costs are to be paid by the Office of the Superintendent of Financial Institutions.
Ellen Fry
Ellen Fry
Presiding Member
Michel P. Granger
Michel P. Granger
Secretary
| Date of Determination and Reasons: |
March 8, 2004 |
| Tribunal Member: |
Ellen Fry, Presiding Member |
| Senior Investigation Officer: |
Cathy Turner |
| Counsel for the Tribunal: |
Marie-France Dagenais |
| Complainant: |
AppDepot Web Services Inc. |
| Government Institution: |
Office of the Superintendent of Financial Institutions |
STATEMENT OF REASONS
COMPLAINT
On December 23, 2003, AppDepot Web Services Inc. (AppDepot) filed a complaint with the Canadian International Trade Tribunal (the Tribunal) under subsection 30.11(1) of the Canadian International Trade Tribunal Act 1 concerning the procurement (Solicitation No. OSFI-BSIF 2003/001) by the Office of the Superintendent of Financial Institutions (OSFI) for the provision of Web site re-engineering and Web content management services.
AppDepot alleged that the OSFI incorrectly evaluated its proposal with respect to mandatory criterion M4 of section 1.1.1.1 of the RFP (M4) and the rated criteria regarding project management qualifications. AppDepot requested, as a remedy, that the Tribunal recommend that the OSFI re-evaluate the bids. In the alternative, AppDepot requested that the Tribunal recommend that the OSFI compensate it for lost opportunity. In addition, AppDepot requested its costs incurred in preparing and proceeding with the complaint.
On December 30, 2003, the Tribunal informed the parties that the complaint had been accepted for inquiry, as it met the requirements of subsection 30.11(2) of the CITT Act and the conditions set out in subsection 7(1) of the Canadian International Trade Tribunal Procurement Inquiry Regulations.2 On January 5, 2004, the OSFI informed the Tribunal that a contract had been awarded to i4design inc. On January 23, 2004, the OSFI filed a Government Institution Report (GIR) with the Tribunal in accordance with rule 103 of the Canadian International Trade Tribunal Rules.3 AppDepot filed its comments on the GIR with the Tribunal on February 4, 2004.
Given that there was sufficient information on the record to determine the validity of the complaint, the Tribunal decided that a hearing was not required and disposed of the complaint on the basis of the information on the record.
PROCUREMENT PROCESS
According to the OSFI, a Request for Proposal (RFP) for the provision of Web content management systems was issued on October 7, 2003. The closing date for the submission of proposals was November 17, 2003.
The RFP reads, in part, as follows:
Part III - EVALUATION CRITERIA AND METHODOLOGY
1.0 EVALUATION CRITERIA
The Bidder must provide sufficient information to reflect their qualification against each criterion. This qualifying information may be presented in one of the two following formats:
· A point-by-point response matching the order of this section (preferred); or
· A matrix pointing the evaluator to the section of the response (page and paragraph reference) in which the qualifying information can be found.
1.1 Mandatory Criteria [as amended]
M4 The Bidder will describe its capacity and experience in this area, including listing up to five (5) successfully completed projects similar in size, scope and nature to which the Bidder has provided services. That similarity is described as:
· Providing Web site design and content management systems development or implementation and the attendant services to clients with similar requirements to those described in Part II - Statement of Work.
1.2 Rated Criteria
For each of the mandatory criteria listed in Section 1.1 of this Part above (M1 - M17), a corresponding rated criterion (R1 - R17) appears in this section.
1.2.2 Technical Proposal Criteria
1.2.2.2 Project Management Qualifications [as amended]
R4 This qualification will be rated on the Bidder's breadth and depth of experience in supplying similar services to similar requirements and the relevance of that experience to the OSFI requirement as defined in Section 2.0 - Scope of Work in Part III - Statement of Work.
The following rating scale, based on the number of similar service offerings and their relevance to this requirement, will be applied:
| 20 pts |
The Bidder has five (5) successful instances of providing similar services to similar relevant requirements with at least two of the project[s] with a Federal Government Department and/or Agency |
| 15 pts |
The Bidder has five (5) successful instances of providing similar services to similar relevant requirements. |
| 15 pts |
The Bidder has five (5) successful instances of providing similar services to similar relevant requirements with at least one of the projects with a Federal Government Department and/or Agency |
| 15 pts |
The Bidder has four (4) successful instances of providing similar services to similar relevant requirements with at least two of the projects with a Federal Government Department and/or Agency |
| 10 pts |
The Bidder has four (4) successful instances of providing similar services to similar relevant requirements with at least one of the projects with a Federal Government Department and/or Agency |
| 10 pts |
The Bidder has four (4) successful instances of providing similar services to similar relevant requirements. |
Maximum points available: 20
Question and answer No. 4 read as follows:
Question:
We are a Canadian corporation established in Quebec. We are a software company that creates Web Content Management Software. For your requirements our product is `off the shelf' software that is parameter driven. We feel that our product encounters all of your mandatory requirements and is an excellent fit for your needs.
However, on page 27 in section M4 it indicates that at least 2 of the 5 projects being provided to a Government of Canada Department or Agency. We do possess 5 reference accounts but this is our first attempt to provide product and services to any Department or Agency of the Canadian Government.
Will we be automatically disqualified if we cannot provide the 2 Canadian Government references?
Answer:
Originally YES however this Questions & Answers # 4 also serves as an AMENDMENT which REMOVES the requirement "that at least 2 of the 5 projects must have [been] with a Federal Government [Department] or Agency".
According to the OSFI, 12 proposals were received. Proposals were evaluated from November 18 to 28, 2003. On December 3, 2003, the OSFI informed all bidders, including AppDepot, of the results of the evaluation and solicitation process. The OSFI indicated that AppDepot was non-compliant with M4. According to the OSFI, bidders whose proposals did not meet the mandatory requirements were also advised that the evaluation of their bids was discontinued, once the proposals had been deemed non-compliant.
On December 3, 2003, AppDepot contacted the OSFI to express concerns about the evaluation results and requested a detailed explanation of the process. On December 4, 2003, AppDepot was advised by the OSFI that its proposal did not clearly demonstrate the successful completion of all five projects submitted for M4, due to an omission of a description of the project or services in one of its listed projects. The same day, AppDepot sent a letter to the Honourable Ralph Goodale outlining its concerns regarding the results of its bid evaluation. On December 12, 2003, in response to AppDepot's letter to Mr. Goodale, the OSFI advised AppDepot that the evaluation team had reconvened and evaluated AppDepot's proposal against the rated criteria of the RFP and that its bid did not achieve the minimum number of points required for section 1.2.2.2 (Project Management Qualifications) in order for its bid to be considered compliant. Later that day, AppDepot expressed concerns with the manner in which its proposal was scored and requested that the OSFI remedy the situation. On December 16, 2003, the OSFI advised AppDepot that it was not prepared to undertake the remedies requested by AppDepot. On December 18, 2003, AppDepot submitted its complaint to the Tribunal and, after providing additional information on December 23, 2003, its complaint was considered filed.
POSITIONS OF THE PARTIES
AppDepot's Position
AppDepot submitted that M4 uses the expression "up to five (5)", not "a minimum of five (5)", or "at least five (5)", or simply "five (5)". AppDepot submitted that rated criterion R4 clearly states that a bidder that provides five successful instances of providing similar services would receive a rating of 20 points, whereas a bidder that provides four successful instances of providing similar services would receive a rating of 15 points. Given this rating matrix, AppDepot argued that one would have to assume that providing four successful instances is also acceptable and not grounds for the rejection of a bidder's proposal. In addition, AppDepot submitted that, although it still contends that it provided five relevant project references in its proposal, the OSFI stated the following in its correspondence of December 4, 2003: "[y]our proposal clearly demonstrated 4 similar projects". According to AppDepot, this would imply that it met rated criterion R4.
With regard to the OSFI's contention that AppDepot's submission of the Saskatchewan Industry and Resources project did not clearly demonstrate the successful completion of a project similar in size, scope and nature to the subject of the procurement, AppDepot submitted that it mentioned that it delivered the Saskatchewan Industry and Resources Web site re-design within the executive summary of its proposal, that the introduction to section 2.7 (Relevant Projects) in its proposal directly and indirectly implies that it delivered each of the projects outlined in the five case studies and that the Saskatchewan Industry and Resources project is listed in its proposed project manager's résumé contained in its proposal.
AppDepot submitted that, after it reviewed the detailed scoring of its proposal, it found a discrepancy between the instructions provided in the RFP and the specific scoring criteria used by the evaluation team. It submitted that, in compliance with the terms of the RFP, it provided concise information throughout its proposal, which, it argued, unfairly worked to its disadvantage when its proposal was evaluated. AppDepot also submitted that there are several criteria for which its proposal received no points or very few points although its proposal addressed those criteria.
In its response to the GIR, AppDepot submitted that it reduced its list of projects to five because it understood that it was to supply "up to five" and not "more than five" project references. Therefore, AppDepot argued that reducing its list to five projects does not mean that it understood the phrase to mean "exactly five". AppDepot contended that the phrase "up to five" in M4 meant that bidders were to supply as many as five project references, which, in other words, translates to "less than or equal to".
OSFI's Position
The OSFI submitted that AppDepot failed to meet all the mandatory requirements of the solicitation. In particular, it submitted that, to be considered compliant under M4, bidders were to list and provide descriptions for exactly five "successfully completed projects" that had requirements that were similar to the requirement stated in the solicitation. The OSFI submitted that AppDepot used the "point-by-point" response format in its proposal and provided five projects; however, for its reference pertaining to the Saskatchewan Industry and Resources project, the OSFI contended that information relating to the nature of the work completed by AppDepot was not included in the description and that the proposal did not therefore clearly demonstrate the successful completion of five projects. The OSFI contended that AppDepot therefore only provided descriptions for four successfully completed projects.
The OSFI submitted that the "re-evaluation" of AppDepot's bid, which was the evaluation of the proposal in its entirety, was conducted by the OSFI to provide AppDepot with a comprehensive debrief on the performance of its entire proposal and was not the reason for disqualifying AppDepot's bid.
Regarding AppDepot's allegation that there was a discrepancy between the instructions provided in the RFP and the evaluation criteria used by the evaluation team, the OSFI submitted that the RFP clearly stated that proposals must be concise but must provide sufficient information to demonstrate how each qualification had been met. The OSFI submitted that AppDepot's proposal failed to provide sufficient information to demonstrate that the mandatory requirements had been met.
With respect to AppDepot's submission on the use of the expression "up to five (5)", the OSFI submitted that AppDepot's statement, "[a]lthough we could have provided more than five (5) client project references, to keep the proposal concise and to comply with the `up to five' criteria in the RFP we reduced our reference list to five projects", demonstrates AppDepot's clear understanding of the requirement.
Finally, the OSFI requested its costs in responding to this complaint.
TRIBUNAL'S DECISION
Subsection 30.14(1) of the CITT Act requires that, in conducting an inquiry, the Tribunal limit its considerations to the subject matter of the complaint. Furthermore, at the conclusion of the inquiry, the Tribunal must determine whether the complaint is valid on the basis of whether the procedures and other requirements prescribed in respect of the designated contract have been observed. Section 11 of the Regulations further provides that the Tribunal is required to determine whether the procurement was conducted in accordance with the applicable trade agreements, which, in this instance, are the Agreement on Internal Trade 4 and the North American Free Trade Agreement.5
Article 506(6) of the AIT provides that, "[i]n evaluating tenders, a Party may take into account not only the submitted price but also quality, quantity, delivery, servicing, the capacity of the supplier to meet the requirements of the procurement and any other criteria directly related to the procurement that are consistent with Article 504. The tender documents shall clearly identify the requirements of the procurement, the criteria that will be used in the evaluation of bids and the methods of weighting and evaluating the criteria."
Article 1015(4)(d) of NAFTA provides that "awards shall be made in accordance with the criteria and essential requirements specified in the tender documentation".
AppDepot's first ground of complaint is that the OSFI incorrectly disqualified its submission of the Saskatchewan Industry and Resources project under M4. The OSFI considered that the project did not meet the requirements of M4 because AppDepot did not clearly demonstrate the successful completion of the project due to an omission of a description of the project or the services provided.
In addressing this ground of complaint, the Tribunal was mindful of the principles outlined in Polaris:6
In its complaint, Polaris submitted, among other things, that it was not awarded sufficient points in respect of several aspects of its proposal. After having reviewed the evidence on the file, the Tribunal sees no reason to conclude that Polaris's proposal was not evaluated in accordance with the criteria and methodology set out in the solicitation documents or that the evaluators did not apply their minds to these items at the time of the evaluation. In the absence of evidence that the evaluation was not conducted in a procedurally fair manner, the Tribunal normally defers to the judgement of the evaluation team as to the assignment of points for the rated technical requirements.7
Section 1.0 of Part III of the RFP provides as follows:
The Bidder must provide sufficient information to reflect their qualification against each criterion. This qualifying information may be presented in one of the two following formats:
· A point-by-point response matching the order of this section (preferred); or
· A matrix pointing the evaluator to the section of the response (page and paragraph reference) in which the qualifying information can be found.
The organization of AppDepot's bid indicates that it chose the point-by-point format, and this was also noted by the OSFI in the GIR.
M4 provides as follows:
The Bidder will describe its capacity and experience in this area, including listing up to five (5) successfully completed projects similar in size, scope and nature to which the Bidder has provided services. That similarity is described as:
· Providing Web site design and content management systems development or implementation and the attendant services to clients with similar requirements to those described in Part II - Statement of Work.
The five projects submitted by AppDepot in response to M4 are described in sequence in section 2.7 of its bid. When the material pertaining to the Saskatchewan Industry and Resources project is examined, it appears to lack very basic information on the nature of the project performed by the bidder, in contrast to the four projects that are described immediately before it. The information does not refer the evaluators to other parts of the bid for further information. Therefore, the OSFI was not unreasonable in concluding that the information on the project did not demonstrate its successful completion. Accordingly, the Tribunal finds that the evidence on file does not indicate that the OSFI acted improperly in disqualifying the Saskatchewan Industry and Resources project submitted by AppDepot under M4, and this ground of complaint is not valid.
AppDepot's second ground of complaint is that the OSFI incorrectly disqualified its bid because it submitted four, instead of five, projects meeting the requirements of M4.
The Tribunal notes that the OSFI has admitted that AppDepot did submit four successfully completed projects. The issue is whether the OSFI was correct in considering that five successfully completed projects were required in order to comply with M4.
M4 requires bidders to submit "up to five (5) successfully completed projects". AppDepot argues that "up to five" means that bidders were to supply as many as five project references, i.e. five project references or less. AppDepot also argues that this interpretation is supported by the wording of rated criterion R4, as outlined above. The OSFI submitted that question and answer No. 4 dated October 16, 2003, demonstrate a clear understanding that the requirement was to submit five projects. It also submitted that AppDepot's statement in its letter to Mr. Goodale, "[a]lthough we could have provided more than five (5) client project references, to keep the proposal concise and to comply with the `up to five' criteria in the RFP we reduced our reference list to five projects", demonstrates AppDepot's clear understanding that five project references were required.
In the Tribunal's opinion, the meaning of M4 with respect to the number of projects required is clear: "up to five" means five or fewer projects, based on the plain language meaning of this phrase. The Tribunal does not accept the OSFI's submissions that, based on the evidence, "up to five" was nonetheless understood to mean exactly five.
The OSFI referred to question and answer No. 4 in support of its position. The Tribunal is of the view that the language of question and answer No. 4 indicates that the bidder posing the question intends to submit five projects in response to M4; however, it does not indicate that the reason for this is that the bidder considered that five projects were required to comply with M4. The reason could equally well be that the bidder considered that submitting five projects would cause its bid to do better in the evaluation, given that M4 was the basis of rated criterion R4.
The OSFI also relied upon a sentence in AppDepot's letter of December 4, 2003, to Mr. Goodale, which reads as follows: "Although we could have provided more than five (5) client project references, to keep the proposal concise and to comply with the `up to five' criteria in the RFP we reduced our reference list to five projects." In the Tribunal's view, this sentence merely indicates that AppDepot intended to submit five projects. It does not indicate that AppDepot believed that exactly five projects were required under M4. Furthermore, the Tribunal notes that the next sentence in the letter puts forward the position that "[s]ection M4 of the RFP uses the expression `up to five (5)', not `a minimum of five (5)', `at least five (5)', or simply, `five (5)'".
Therefore, the Tribunal finds that the OSFI incorrectly declared AppDepot's bid non-compliant with M4 on the basis that it submitted only four projects that met the requirements of M4. Accordingly, this ground of complaint is valid.
The Tribunal notes that rated criterion R4 supports the foregoing interpretation of M4, since it contemplates bidders submitting less than five projects under M4. Specifically, rated criterion R4 awards 10 points to a bidder that submits four successful instances of providing similar services to similar relevant requirements.
AppDepot's third ground of complaint is that the OSFI improperly disqualified the bid by using an improper evaluation process when it evaluated the "Project Management Qualifications" portions of the bid after AppDepot's letter to Mr. Goodale.
The OSFI indicated that this "re-evaluation" was performed by the OSFI only to provide AppDepot with additional information. The OSFI indicated that it disqualified AppDepot's bid because AppDepot was non-compliant with M4 and not because of the "re-evaluation". Consequently, the Tribunal will not address this ground of complaint.
In light of the foregoing, the Tribunal determines that AppDepot's complaint is valid in part.
In recommending a remedy, the Tribunal considered all the circumstances relevant to this procurement, including those outlined in subsection 30.15(3) of the CITT Act. AppDepot requested, as a remedy, that the Tribunal recommend that the OSFI re-evaluate the bids. In the alternative, AppDepot requested that the Tribunal recommend that the OSFI compensate it for lost opportunity. In addition, AppDepot requested its costs incurred in preparing and proceeding with the complaint.
It is fundamental to this procurement process and the integrity of the competitive procurement system generally that the government institution follow the evaluation criteria established in the RFP. The OSFI did not do this, even though the wording of M4 regarding the number of projects was clear. As a result, not only AppDepot, but potentially all of the 12 bidders, may have been prejudiced. The Tribunal notes that it also found to be valid a complaint by a second bidder8 that the OSFI failed to follow the criteria in the RFP. However, the evidence does not indicate that the OSFI acted in bad faith.
The Tribunal notes that the contract has been awarded and that the OSFI has indicated that the work is to be completed by the end of March 2004. Therefore, it appears that, at this point, a significant portion of the work will have already been completed. Accordingly, it would not be appropriate to recommend that the bids be re-evaluated or that the contract be re-tendered.
It is impossible to predict what the results of the procurement would have been, had the bids been evaluated according to what was permitted by the RFP, in light of the large number of bidders and the ambiguity in M4 that was noted in the complaint by IHS.
While the Tribunal did not need to inquire into the substance of AppDepot's allegation regarding the OSFI's evaluation of the "Project Management Qualifications" portion of its bid, the Tribunal notes that the correspondence from the OSFI to AppDepot would, in fact, cause a reasonable person to conclude that the evaluation was a reason for which the bid was disqualified. The OSFI's December 12, 2003, correspondence states:
Because of the concerns you raised in your letter, addressed to Mr. Ralph Goodale, of December 4, 2004, the Office of the Superintendent of Financial Institutions (OSFI) reconvened our bid evaluation team to evaluate the AppDepot proposal against the rated criteria of the Request for Proposal (RFP).
I am sorry to inform you that the AppDepot bid did not achieve the minimum points required for Article 1.2.2.2 (Project Management Qualifications) to be compliant. The portion of the evaluation for the Project Management Qualifications Section is attached for your information.
We would be pleased to provide you with a complete de-briefing of the evaluation of your proposal.
The Tribunal is of the opinion that the OSFI's intention regarding the "re-evaluation" did not become clear until the GIR was filed. Therefore, it was the OSFI's conduct that led AppDepot to raise this ground of complaint.
Furthermore, the Tribunal notes that there is a significant possibility that the OSFI's "re-evaluation" of this portion of AppDepot's bid had deficiencies. The Tribunal notes, for example, that, under rated criterion R9, the OSFI awarded AppDepot no points for change management and no points for problem resolution, despite the fact that AppDepot's bid clearly had material that addressed these two requirements.
In light of the foregoing, the Tribunal recommends that AppDepot be compensated in the amount of $1,830,9 representing an estimate of one twelfth of the profit that AppDepot reasonably would have made, had it been awarded the contract.
Finally, the Tribunal awards AppDepot its reasonable costs incurred in preparing and proceeding with its complaint.
DETERMINATION OF THE TRIBUNAL
Pursuant to subsection 30.14(2) of the CITT Act, the Tribunal determines that the complaint is valid in part.
Pursuant to subsections 30.15(2) and (3) of the CITT Act, the Tribunal recommends, as a remedy, that AppDepot be compensated in the amount of $1,830, representing an estimate of one twelfth of the profit that AppDepot reasonably would have made, had it been awarded the contract.
Pursuant to section 30.16 of the CITT Act, the Tribunal awards AppDepot its reasonable costs incurred in preparing and proceeding with the complaint, which costs are to be paid by the OSFI.
1 . R.S.C. 1985 (4th Supp.), c. 47 [CITT Act].
2 . S.O.R./93-602 [Regulations].
3 . S.O.R./91-499.
4 . 18 July 1994, C. Gaz. 1995.I.1323, online: Internal Trade Secretariat <http://www.intrasec.mb.ca/eng/it.htm> [AIT].
5 . 32 I.L.M. 289 (entered into force 1 January 1994) [NAFTA].
6 . Re Complaint Filed by Polaris Inflatable Boats (Canada) Ltd. (23 June 2003), PR-2002-060 (CITT).
7 . Ibid. at 6.
8 . Re Complaint Filed by IHS Solutions Limited, (9 March 2004), PR-2003-067 (CITT).
9 . For this purpose, an estimated profit level of 15 percent of the value of the contract (excluding GST) was used.