DMR CONSULTING GROUP INC.
Determinations
File No.: PR-97-009
Ottawa, Thursday, September 18, 1997
File No.: PR-97-009
IN THE MATTER OF a complaint filed by DMR Consulting Group Inc. under subsection 30.11(1) of the Canadian International Trade Tribunal Act, R.S.C. 1985, c. 47 (4th Supp.), as amended;
AND IN THE MATTER OF a decision to conduct an inquiry into the complaint under subsection 30.13(1) of the Canadian International Trade Tribunal Act.
DETERMINATION OF THE TRIBUNAL
Pursuant to section 30.14 of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal determines that the complaint is not valid.
Dr. Patricia M. Close
_________________________
Dr. Patricia M. Close
Member
Susanne Grimes
_________________________
Susanne Grimes
Acting Secretary
Date of Determination: September 18, 1997
Tribunal Member: Patricia M. Close
Investigation Manager: Randolph W. Heggart
Counsel for the Tribunal: Hugh J. Cheetham
Complainant: DMR Consulting Group Inc.
Counsel for the Complainant: G.P. (Patt) MacPherson
Ronald C. Lefebvre
Government Institution: Department of Public Works and Government
Services
FINDINGS OF THE TRIBUNAL
INTRODUCTION
On June 20, 1997, DMR Consulting Group Inc. (DMR) filed a complaint under subsection 30.11(1) of the Canadian International Trade Tribunal Act [1] (the CITT Act) concerning the procurement (Solicitation No. 19162-6-C040/A) by the Department of Public Works and Government Services (the Department) on behalf of the Department of Justice (Justice Canada) for the provision of professional services as a system integrator for the Canadian Firearms Registration System (CFRS).
DMR alleged that the Department improperly evaluated its proposal by using an undisclosed document entitled “Evaluation Scoring Criteria” which imposed many specific requirements that could not be perceived or deduced from the Request for Proposal (RFP). DMR claims that this action is a breach of Article 506(6) of the Agreement on Internal Trade [2] (the AIT).
DMR requested, as a remedy, that it be compensated for the reasonable costs incurred in relation to its bid and this complaint. It also requests that the Canadian International Trade Tribunal (the Tribunal) provide guidance to the Department with respect to Article 506(6) of the AIT.
INQUIRY
On June 24, 1997, the Tribunal determined that the conditions for inquiry set forth in section 7 of the Canadian International Trade Tribunal Procurement Inquiry Regulations [3] (the Regulations) had been met in respect of the complaint and, pursuant to section 30.13 of the CITT Act, decided to conduct an inquiry into this matter.
On July 22, 1997, the Department filed with the Tribunal a Government Institution Report (GIR) in accordance with rule 103 of the Canadian International Trade Tribunal Rules. [4] On August 7, 1997, DMR filed its comments on the GIR with the Tribunal. On August 15, 1997, the Department filed additional comments with the Tribunal. DMR also provided additional comments on August 21, 1997.
Given that there was sufficient information on the record to determine the validity of the complaint, the Tribunal decided that a hearing was not required and disposed of the complaint on the basis of the information on file.
PROCUREMENT PROCESS
A vendor information session was held on June 14, 1996, to present the procurement strategy to be used for the CFRS and to invite feedback. DMR was one of the vendors present at the briefing. On June 21, 1996, DMR forwarded written comments and suggestions to improve the procurement strategy. On September 16, 1996, the updated procurement strategy was issued to those who had attended the vendor information session.
A Notice of Proposed Procurement was published on the Open Bidding Service and in Government Business Opportunities on March 12, 1997, with a closing date for receipt of bids of April 21, 1997. The RFP contained, among other things: (a) Appendix “A” - a detailed statement of work; (2) Appendix “C” - a description of technical proposal requirements; and (3) Appendix “E” - a listing and weighting of mandatory and rated evaluation criteria. The method of evaluation was to consist of four steps: (1) verification of compliance to mandatory terms; (2) evaluation of technical proposals to rated requirements - worth 65 percent overall (proposals had to receive 75 percent of a possible 850 points to be considered further); (3) oral presentation - worth 20 percent overall; and (4) financial proposal evaluation - worth 15 percent overall. Section D of the RFP contained instructions for enquiries and clarifications during the bidding period. Four updates were issued prior to the closing date. Amendment No. 001 increased the total points available for the technical evaluation of proposals to 865 points. Some amendments were in response to vendor questions and, in one case, extended the bid closing date to April 25, 1997.
According to the Department, prior to the closing date, the evaluation team, consisting of members from Justice Canada, the Royal Canadian Mounted Police, specialists on contract to the CFRS project, a representative from one of the provinces and a senior procurement officer from the Department, created a detailed evaluation guideline for assigning points in relation to the evaluation criteria contained in the RFP. This guideline was entitled “Evaluation Scoring Criteria - Rated Requirements,” followed by the applicable section of the technical proposal requirements as listed in Appendix “C” of the RFP.
Four proposals were received by the closing date, including one from DMR.
According to the Department, each of the 11 members of the evaluation team independently evaluated the technical proposals. The evaluators then met to review the scores and comments for each bidder and, wherever there was a variance of more than 20 percent, discussions were conducted and members were able to finalize their individual scores so that all were within what was considered by the Department to be an acceptable 20 percent variance. Some questions of clarification were sent to all bidders prior to the final evaluation of proposals. After the completion of the evaluation stage, two proposals, DMR’s proposal being one, failed to achieve the required minimum score of 75 percent. On May 7, 1997, the results of the evaluation were sent to the respective bidders. On May 30 and June 6, 1997, DMR was provided a debriefing on the technical evaluation of its proposal and, at that time, was given a copy of the detailed evaluation guide used during the process. After taking into consideration some of DMR’s concerns raised during the debriefing, the evaluation score for DMR’s proposal was slightly amended upward, but was still below the required minimum score. On June 20, 1997, DMR filed a complaint with the Tribunal.
VALIDITY OF THE COMPLAINT
DMR’s Position
In summary, DMR’s complaint is that the Department, contrary to the requirements in Article 506(6) of the AIT, used undisclosed criteria in evaluating the proposals for this procurement. DMR states that the undisclosed criteria impose specific requirements which could not be perceived or deduced from the requirements set out in the RFP.
In the complaint, DMR presents three specific examples of areas where it believed certain requirements were added that were not contained in the RFP. DMR disputes the score assessed for its proposal in the section for large project experience. DMR’s position is that its proposal was assessed with unannounced requirements, that the resources proposed have been responsible as a prime contractor in the delivery of large projects, that referenced projects contain specific components similar to the CFRS, that points were reduced if named resources had not participated in the referenced projects and that points would be based on the number of referenced projects presented. In relation to quality assurance and control, DMR believed that declaring DMR as certified to the ISO 9001:1994 standard for quality management, supplying details on DMR’s quality assurance approach, offering additional information on request and supplying 82 pages on the DMR Quality Management System should have enabled it to score the maximum 20 points allowable for the category. It considered the score that it received to be grossly unfair, whether judged on the basis of the RFP or the undisclosed criteria. DMR’s third example relates to the section relating to experience in the area of other hardware/software procurements. DMR contends that the undisclosed criteria contained far more specific requirements, i.e. if the bidder cited fewer than three projects with values exceeding $5 million, points were to be deducted.
DMR submitted that its proposal was wrongly disqualified on the basis of criteria not clearly specified in advance.
In response to the GIR, DMR submitted that the Department has tried to confuse the issue by introducing the word “guidelines” to describe what it calls criteria or standards by which the proposals were judged. DMR also questions the need for creating such a tool if the RFP clearly identifies the criteria. DMR states that, so vague and unclear were the stated requirements that the Department chose to call criteria, many interpretations could legitimately be placed upon their meaning. DMR addresses the Department’s analysis of the technical evaluation of its proposal by citing more examples of where it disagrees with the score that its proposal received and the apparent basis for that score.
Department’s Position
The Department submitted that detailed evaluation guidelines were developed using the evaluation criteria and weighting specified in the RFP. The guidelines were developed to ensure consistency among evaluators, to ensure a fair and equitable evaluation of bids, to guard against relative scoring of one proposal in relation to another proposal, to provide a measurement device for evaluators and to guard against possible evaluator bias. The Department stated that the guidelines were not published because they are consistent with criteria in the RFP, the guidelines did not introduce new capability requirements inconsistent with the RFP and sufficient transparency was evident in the criteria and weighting published in the RFP.
The Department submitted that, although DMR alleges that there were criteria used in the evaluation of proposals that were not disclosed in the RFP, DMR has failed to provide any evidence of this. The Department submitted that DMR is disputing the judgment used in arriving at the scores that its proposal received in various categories. The Department provides extensive detail on each aspect of the RFP requirements and explains the rationale behind the scores assessed to DMR’s proposal in relation to the allegations contained in the complaint.
The Department submitted that two other experienced bidders, privy to exactly the same information as DMR, were able to interpret the requirements specified in the RFP and offered technical proposals that met the minimum required scoring. The Department submitted that DMR had significant time and opportunity to request clarification and explanation of any aspect of the RFP prior to bid closing.
The Department also submitted that, unless evidence is presented that indicates that bids were evaluated with criteria that were not stated in the RFP, the Tribunal’s jurisdiction should be restricted to situations where there is a deliberate attempt on the part of a government institution to discriminate in favour of one bidder over another. It requests that the Tribunal enunciate an interpretation of its jurisdiction along these lines.
Tribunal’s Decision
Subsection 30.14(2) of the CITT Act requires that the Tribunal determine whether the complaint is valid on the basis of whether the procedures and other requirements prescribed in respect of the designated contract have been observed. Section 11 of the Regulations provides, in part, that the Tribunal is required to determine whether the procurement was conducted in accordance with the requirements set out in the AIT. In deciding these matters, the Tribunal must determine whether the Department, in conducting this procurement, acted according to the provisions of the AIT. Specifically, the Tribunal must decide whether the Department, in evaluating DMR’s proposal, applied the evaluation criteria set out in the RFP in a manner consistent with the AIT.
The Tribunal is of the opinion that the Department did not introduce new criteria in the evaluation of the bids in this procurement and that the guideline document used to perform the evaluation reflects the criteria in the RFP. The Tribunal is satisfied that the evaluation committee conducted a thorough review of DMR’s proposal, and the other proposals, in a manner consistent with the provisions of the RFP. The evaluation committee applied the evaluation method consistently and documented its decisions and findings in a satisfactory manner. As to whether or not DMR should have received more points on any specific items, as stated in Mirtech International Security Inc., [5] the Tribunal is of the view that, if it is satisfied that the Department acted in a procedurally fair manner, it must generally defer to the Department’s judgment on specific items. This does not mean, however, that the Tribunal will not examine the actual scoring received by a proposal in appropriate circumstances. For instance, in FPG/HRI Joint Venture (Fall Protection Group Inc. and HRI Human Resources International Inc., [6] the Tribunal found that the evaluation rating guide used in that case was seriously flawed in relation to the evaluation criteria set out in the tender documentation. Furthermore, the Tribunal found that there was too much subjective judgment left in the hands of the evaluators and that the scores assigned for certain items were not sufficiently consistent with the content of the particular bid in question.
In this instance, the Tribunal is satisfied that the Department set out in the RFP a clear evaluation approach and criteria. The creation of a guide for the evaluation committee was a rigorous exercise to organize in an objective manner the judgment of a rather large and diverse committee. It should not be characterized as an attempt to add additional unknown criteria after the RFP was published, but rather to measure the responses of the bidders in a consistent manner. In fact, the committee reviewed the guide and corrected some inconsistencies in the guide on the basis of the criteria set out in the RFP. While the guide and the RFP are not identical in all respects, there is a reasonable relationship between the two documents. The examples presented by DMR with respect to specific scoring by the evaluators, in the Tribunal’s opinion, did not demonstrate an unreasonable or discriminatory exercise of discretion on the part of the evaluators. For example, with respect to the criteria in the RFP for “Secure Environment Experience” (item 3.8 of Appendix “C” of the RFP), the guide provides more precise indicators for assessing the criteria. The guide does not, however, in the Tribunal’s opinion, introduce new criteria.
In addition, if, as argued by DMR, the criteria in the RFP were so obviously vague and unclear, DMR had, as did all bidders, ample opportunity to seek clarification or explanation of the requirements prior to bidding. DMR did not fully avail itself of this opportunity.
For the reasons stated above, the Tribunal finds that the Department properly declared DMR’s proposal non-compliant with the specifications for failing to meet the minimum mandatory score in the technical proposal evaluation. The Department arrived at this decision fairly and equitably and, in so doing, used criteria and weighting that were clearly set out in the tender documents.
DETERMINATION OF THE TRIBUNAL
In light of the foregoing, the Tribunal determines, in consideration of the subject matter of the complaint, that the procurement was conducted in accordance with the AIT and, therefore, that the complaint is not valid.
1. R.S.C. 1985, c. 47 (4th Supp.).
2. As signed at Ottawa, Ontario, on July 18, 1994.
3. SOR/93-602, December 15, 1993, Canada Gazette Part II, Vol. 127, No. 26 at 4547, as amended.
4. SOR/91-499, August 14, 1991, Canada Gazette Part II, Vol. 125, No. 18 at 2912, as amended.
5. File No. PR-96-036, Determination of the Tribunal, June 3, 1997.
6. File No. PR-95-031, Determination of the Tribunal, June 6, 1996.
[ Table of Contents]
Initial publication: September 19, 1997