ARRAY SYSTEMS COMPUTING INC.
Determinations
File No.: PR-95-024
Ottawa, Monday, March 25, 1996
IN THE MATTER OF a complaint filed by Array Systems Computing Inc. under subsection 30.11(1) of the Canadian International Trade Tribunal Act, R.S.C. 1985, c. 47 (4th Supp.), as amended by S.C. 1993, c. 44;
AND IN THE MATTER OF a decision to conduct an inquiry into the complaint under subsection 30.13(1) of the Canadian International Trade Tribunal Act.
DETERMINATION OF THE TRIBUNAL
Pursuant to section 30.14 of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal determines that the complaint is not valid.
Anthony T. Eyton
_________________________ Anthony T. Eyton
Member
Michel P. Granger
_________________________ Michel P. Granger
Secretary
File No.: PR-95-024
Date of Determination: March 25, 1996
Tribunal Member: Anthony T. Eyton
Investigation Manager: Randolph W. Heggart
Counsel for the Tribunal: Joël J. Robichaud
Complainant: Array Systems Computing Inc.
Government Institution: Department of Public Works and Government Services
FINDINGS OF THE TRIBUNAL
Introduction
On January 10, 1996, Array Systems Computing Inc. (the complainant) filed a complaint under subsection 30.11(1) of the Canadian International Trade Tribunal Act [1] (the CITT Act) concerning the procurement by the Department of Public Works and Government Services (the Department) (Solicitation No. W7714-5-9921/A) for the provision of an advanced communications electronic support measure (ESM) system (ACES) architectural study for Defence Research Establishment Ottawa (the Scientific Authority), a constituent of the Department of National Defence (DND).
The complainant alleges that the technical evaluation criteria, as currently constituted in the Request for Proposal (RFP), form part of the technical specifications and that they are biased in favour of the preferred sole source contractor, Applied Silicon Inc. Canada (the Incumbent). Specifically, the complainant objects, in part, to the weight attributed to knowledge and expertise of electronic warfare (EW) communications as opposed to signal processing analysis and the requirement for superfluous experience. The complainant requested, as a remedy, that the Department review and amend the technical evaluation criteria to match the project goals, that bid closing be extended by 40 days after the issuance of new evaluation criteria and that the Scientific Authority institute management procedures to correct the bias.
On January 15, 1996, the Canadian International Trade Tribunal (the Tribunal) determined that the conditions for inquiry set forth in section 7 of the Canadian International Trade Tribunal Procurement Inquiry Regulations [2] (the Regulations) had been met in respect of the complaint and decided to conduct an inquiry into whether the procurement was conducted in accordance with the requirements set out in Chapter Five of the Agreement on Internal Trade [3] (the AIT).
Inquiry
On February 12, 1996, the Department filed with the Tribunal a Government Institution Report (GIR) in accordance with rule 103 of the Canadian International Trade Tribunal Rules. [4] The complainant’s comments on the GIR were filed with the Tribunal on February 23, 1996.
Given that there was sufficient information on the record to determine the validity of the complaint, the Tribunal decided that a hearing was not required and disposed of the complaint on the basis of the information on file.
Procurement Process
On October 26, 1995, the Department received a requisition from the Scientific Authority to negotiate, on a sole source basis, a contract for the ACES architectural study with the Incumbent. On November 8, 1995, an Advance Contract Award Notice [5] (ACAN) was published on the Open Bidding Service (OBS). The ACAN indicated that the procurement was subject to the AIT and stated, in part: “Applied Silicon Inc. Canada has a unique knowledge in the hardware and software in the existing ACES system implementation as a result of previous work under several contracts including ‘Development of an Advanced Communications ESM (ACES)[ [6] ] Prototype.’ They have the detailed knowledge of the algorithms implemented in software and the technical issues involved in mapping them to parallel processor hardware which is essential to perform this work.”
On November 20, 1995, the complainant wrote to the Department challenging the proposed sole source award. Essentially, the complainant submitted that the ACES architectural study was well within its capabilities. The complainant re-iterated its challenge on November 23, 1995, this time requesting that the sole source award be replaced by an RFP, e.g. open competition, and stating that, after having read the Statement of Work (SOW), it was “more convinced than ever that this study falls directly within Array’s specific expertise.”
On December 8, 1995, a Notice of Proposed Procurement for the subject solicitation was published on the OBS, and the related RFP, including the SOW, was made available to potential suppliers. On December 18, 1995, the complainant wrote to the Department, objecting to certain aspects of the RFP which prevented a fair and open competition. Specifically, the complainant submitted that strong echoes of the faulty sole source justification still existed amongst the evaluation criteria, that many of the evaluation criteria had little or no relevance to the stated goals of the study, that the evaluation criteria were tailored to the originally intended sole source contractor and that the time period to bid was inappropriate in the circumstances. On December 19, 1995, the Department issued an amendment to the RFP extending the bid closing date by two weeks to January 19, 1996. On December 22, 1995, the Department issued a second amendment to the RFP, this time removing the need for a collaboration plan and making additional ACES information available to interested suppliers.
The RFP, as amended, described the evaluation methodology, in part, as follows:
Points
Maximum Minimum
(A) Technical Proposal
1. Understanding objectives and scope of work. 20.0 14.0
2. Knowledge and understanding of analogue and digital
communications signals. 10.0 7.0
3. Knowledge and understanding of the signal processing
algorithms for communications ESMs. 20.0 14.0
4. Knowledge and understanding of digital signal processor
technology, particularly, very high performance systems. 30.0 21.0
5. Understanding technical issues in interfacing digital
receivers to digital signal processor hardware. 10.0 7.0
(B) Management Proposal
1. The management plan is to provide a project plan describing
how the work will be organized and carried out. It must
identify and demonstrate the technical capabilities and
expertise of the proposed project team in the fields of
high performance signal processing and communications
EW systems. The plan must describe how the project will be
managed and identify the single project manager detailing
his/her experience in system engineering, project management,
high performance signal processing systems and communications
or communications EW signal processing. 10.0 7.0
2. Demonstrated expertise of the proposed personnel team in
high performance signal processing systems and communications
EW systems. 20.0 14.0
3. Demonstrated capabilities and experience of the proposed
Project Manager in system engineering, project management,
high performance signal processing systems and communications
or communications EW signal processing. 15.0 10.5
(C) Collaboration Plan
Deleted in its entirety.
The RFP also stated that “the successful bidder will be the firm offering the HIGHEST RATED VALID PROPOSAL at a cost of $100,000.00 or less (GST extra).... To be considered valid, a bid must achieve a score of 70 per cent or better in each of the two (2) areas (technical and management) of the evaluation criteria.”
Ten firms requested a copy of the bid package from OBS and three firms submitted bids. Two firms requested the additional ACES information. The complainant did not request the additional information nor did it bid. On January 10, 1996, the complainant filed its complaint with the Tribunal.
Validity of the Complaint
Complainant’s Position
In its comments on the GIR, the complainant submits that the Department and the Scientific Authority have failed to justify the evaluation criteria advanced in the RFP and are, therefore, in contravention of paragraph 3(b) of Article 504 of the AIT and the Department’s Supply Manual (SM) (Annex 4.1, Article 1007, 1 and 2(a), formerly Supply Policy Manual 3002, 3(a) and 15 (c)). The complainant goes on to say that, while the responses provided in the GIR are very eloquent and rigorous, these responses, nonetheless, underline the hidden requirements buried in the criteria, which would have seriously impacted the evaluation of any offer. The complainant organizes its comments on the GIR in three sections: AIT and SM Infringements; Technical Subjectivity; and Process.
Concerning AIT and SM infringements, the complainant submits that evidence still exists throughout the evaluation criteria and in the arguments contained in the GIR that the essential points underlying the sole source justification, and put in “doubt,” have, in fact, been maintained and bolstered by the Scientific Authority. This re-emergence of the sole source argument within the evaluation criteria is a clear indication of the biasing of the technical specification. Further, the SOW remained substantially unchanged upon transition from sole source to open competition. For example, there still does not exist a direct link between many evaluation criteria and the work which must be performed. The complainant also submits that the Department has not made its case as to why expertise in communications and communications EW is required from all personnel proposed by suppliers, including the program manager. This requirement is excessive and is designed to match the profile of the Incumbent.
Concerning technical subjectivity, the complainant submits that, even if the SOW had specified all relevant interfaces to the complete system and the signal propagating across the interfaces (omitted in the SOW), the potential to subjectively evaluate one contractor’s ACES system knowledge above that of another is enormous. As well, the Department’s representation of a hands-off relationship existing or to exist between the contractor and the Technical Authority is at variance with the facts. Indeed, the complainant submits that there is always design collaboration between government and industry, as one would expect when talented and creative scientists and engineers on both sides meet across the table. The fact that the Department argues otherwise in the circumstances cannot change the significant role played or to be played by the Scientific Authority and the Incumbent during the ACES architectural study. Further, the Department’s opinion that “similar experience” in the field of algorithm implementation is not valid and that “identical” experience is required represent nothing less than the unwarranted exclusion of relevant experience by means of evaluation criteria restrictively formulated.
The Department’s need for the use of special purpose hardware is yet another example of “a priori” decisions resulting in “putting the cart before the horse.” Why, indeed, define the solution up front in terms of specific hardware requirements when the Department knows that the complainant has recently found commercial off-the-shelf implementations for some of Canada’s most “sacred cows” of special purpose signal processing hardware? On the question of the location of the work, particularly as it relates to the ACES testbed, the complainant submits that the Department, in its arguments, again shows an unnecessary bias, overlooking different approaches to using a testbed or ignoring the possible use of a modem to conduct such tests, as has already been done by the complainant on other contracts for DND.
Concerning process issues and the Department’s statement that the SOW was developed “in isolation from bidder involvement,” the complainant submits that it strains credulity to deny that, with all the associated contracts where the Incumbent was working side by side with the Scientific Authority and the postulated expertise of the Incumbent, some exchanges directly related to this work did not occur. Further, the complainant notes that the Department is silent in the GIR on the complainant’s concern that the technical proposal should address the technical approach to be used to satisfy the SOW as its primary rather than secondary goal. The complainant submits that the phrasing and content of the evaluation criteria have, in effect, made the SOW subordinate to the evaluation criteria. Finally, the complainant notes that the Department has completely ignored, in the GIR, its suggestion for a complete independent peer review of the SOW and related evaluation criteria to remove any and all possibility of bias.
In summary, the complainant submits that the Department has not proven its case. Indeed, “insufficient review and too much haste in the issuance of the RFP [have] led to biased technical specifications.” As well, the Department has shown directly, and by implication, that it is prepared to interpret the evaluation criteria in favour of the technical approach, staffing profile and location of the Incumbent. This, the complainant concludes, has not been a fair and open competition and states that it has been “at a disadvantage from the start.”
Department’s Position
The Department submits, in the GIR, that the SOW and evaluation criteria for the subject procurement are not biased towards any one supplier. Rather, they accurately define the requirement in generic terms and stipulate the minimum acceptable criteria upon which offers will be evaluated.
Specifically, the Department disputes the complainant’s contention that the Department implicitly agreed that the original RFP was deficient in respect of the time period to prepare a bid or in respect of section C of the evaluation criteria dealing with the requirement for a collaboration plan. Time was extended because a substantial amount of additional technical information was being made available. Further, the company collaboration section was removed because there was no connection between the need to collaborate and the requirement. The Department also submits that the SOW was produced by highly qualified personnel from the Scientific Authority in isolation of any bidder’s involvement. Moreover, other senior technical personnel within the Scientific Authority was involved, thereby ensuring the integrity of the technical requirement. Finally, the Department’s personnel reviewed the SOW and evaluation scheme and found them acceptable.
The Department adds that it was sensitive to the complainant’s representation during the bidding process and, for example, accepted to delete the requirement for a collaboration plan and made additional technical information available to all bidders to accommodate some of its requests. With respect to the complainant’s other requests, “[a]ll agreed that Array’s demands for changes in the technical area were unreasonable and technically unacceptable.” In respect of the restrictive, biased nature of the evaluation criteria, the Department submits that these were designed to ensure that the successful bidder would have the requisite technical knowledge to be able to provide useful results without an inordinate amount of assistance from the Scientific Authority. The criteria reflect the fact that ACES is a very complex system. The Department also argues that the ACES signal processor architecture is an integral part of the system architecture and cannot be treated in isolation. The inclusion of communications EW knowledge in the evaluation criteria reflects the important need to have a systems perspective in carrying out the work.
The Department then submits that there is no substantiation to the complainant’s claim that superfluous experience is required, for example, in respect of the project manager. Indeed, the evaluation criteria provide for expertise in communications or communications EW signal processing and, consequently, it is actually possible to obtain a full score on this point without any communications EW expertise. On the issue of the location of the work, the Department submits that there is nothing in either the SOW or the RFP which deals with the location of the contractor. The contractor, however, must use the ACES testbed located in Ottawa to ensure system integration. If the software for the ACES signal processing algorithms does not run on the system, it is completely worthless. This, the Department contends, is not a trivial issue. On the question of the relevance of the evaluation criteria to the stated goals of the study, the Department submits that the evaluation criteria were developed by experts in the field, are fully consistent with the nature and the scope of the work required and were designed to screen out suppliers that did not have the necessary capabilities to carry out the work. The fact that three companies submitted bids confirms that the evaluation criteria are not excessively restrictive.
The Department further submits that the study focussed on an analysis of the signal processing architecture, but within the ACES system overall. The reference to “second generation” ACES system in the RFP does not contradict the statement referring to “[a] clean sheet of design paper exercise.” The latter statement indicates that the government is prepared to consider proposals advancing architectures involving more than changes of an incremental nature. On the question of “similar” or “identical” expertise, the Department submits that the ACES system is very much a testbed at this stage and that the algorithms have not reached a state of maturity. The study goal, therefore, is not to design highly optimized implementations of a few algorithms as the complainant did in the past, but to determine an architecture which has the potential to implement more sophisticated algorithms.
On the question of the management proposal, specifically, the project manager qualifications and expertise, the Department submits that communications EW expertise in the project manager is seen to be an important capability. Nevertheless, out of the 45 points attributed to the management proposal, no more than 10 points are given for this requirement. Notwithstanding all the above, the evaluation criteria are sufficiently flexible that any one of a number of firms could have satisfied this requirement.
In summary, the Department states that this procurement, initiated as a sole source award, was converted to a competitive procurement after the complainant demonstrated that it potentially could respond to the requirement. Having made the conversion, every opportunity was accorded to the complainant to present a responsive bid. The evaluation criteria were formulated and critically reviewed by personnel in both the Department and DND. The criteria are specific to the project and represent, in the Scientific Authority’s opinion, the minimum acceptable requirements without jeopardizing the successful completion of the project.
Tribunal’s Decision
Section 30.14 of the CITT Act requires that, in conducting an inquiry, the Tribunal limit its considerations to the subject matter of the complaint. Furthermore, at the conclusion of the inquiry, the Tribunal must determine whether the complaint is valid on the basis of whether the procedures and other requirements prescribed in respect of the designated contract have been observed. Section 11 of the Regulations further provides, in part, that the Tribunal is required to determine whether the procurement was conducted in accordance with the requirements set out in the AIT.
Paragrah 3(b) of Article 504 of the AIT provides, in part, that:
3. Except as otherwise provided in this Chapter, measures that are inconsistent with paragraphs 1 and 2 include, but are not limited to, the following:
(b) the biasing of technical specifications in favour of, or against, particular goods or services, ... or in favour of, or against, the suppliers of such goods or services for the purpose of avoiding the obligations of this Chapter.
Article 501 of the AIT provides, in part, that the purpose of the Chapter is to “establish a framework that will ensure equal access to procurement for all Canadian suppliers in order to contribute to a reduction in purchasing costs and the development of a strong economy in a context of transparency and efficiency.”
Article 506(6) of the AIT provides that, “[i]n evaluating tenders, a Party may take into account not only the submitted price but also quality, quantity, delivery, servicing, the capacity of the supplier to meet the requirements of the procurement and any other criteria directly related to the procurement that are consistent with Article 504. The tender documents shall clearly identify the requirements of the procurement, the criteria that will be used in the evaluation of bids and the methods of weighting and evaluating the criteria.”
The Tribunal, having examined the evidence and arguments presented by both parties and considered the obligations specified in the AIT, has concluded that the complaint is not valid.
The complaint can be viewed in three distinct parts: first, that the technical specification or SOW is restrictive; second, that the evaluation criteria are unfairly biased in favour of the Incumbent; and, third, that the evaluation, given the first and second parts of the grounds, can only be conducted in a biased manner.
With respect to the technical specification or SOW, the Tribunal is not in a position to second guess the judgement of the Scientific Authority in its determination of the need or requirement in this procurement. The procedure followed in establishing the SOW contained some verifications to ensure that the requirement was not formulated in such a manner as to deliberately exclude certain suppliers. In fact, 10 firms requested a copy of the solicitation documentation from OBS and 3 firms submitted offers.
The Tribunal would comment, however, that there may be some merit in setting up a standing committee at the Scientific Authority to review technical specifications with the objective of testing the specifications of future intended procurements, i.e. to ensure that the requirement is not worded so as to exclude effective competition. The Tribunal makes this suggestion in order to help alleviate any apprehensions on the part of potential suppliers that a procurement may not be conducted in a manner that is consistent with the purpose of the AIT. This seems particularly applicable in situations where the government’s intention to sole source a requirement is successfully challenged, as is the case here. Opening up procurements to more competition has merit for the government and the taxpayer because it can result in better procurement decisions and lower cost alternatives.
With respect to the evaluation criteria being unfairly biased in favour of the Incumbent, again the Tribunal can find no violation of the AIT with respect to the establishment of these criteria. The requirement of specific expertise, for example, in communications or communications EW signal processing for certain proposed team members, is not unreasonable. In fact, the Tribunal is of the view that the specific requirements for the project manager are not out of line with the stated requirements in the SOW. Further, the Tribunal notes that a potential supplier is at liberty to construct its proposal with expertise coming from within and/or without and, in that sense, the requirement for certain expertise cannot be determined to be company-specific. On the question of the concern towards a regional bias, the Tribunal is of the view that the requirement for confirming work on the existing testbed does not preclude a supplier from conducting a considerable amount of work off-site.
Although there may be some subjectivity in the application of these types of evaluation criteria, this is not prohibited by the AIT and, in fact, in the opinion of the Tribunal, professional judgement is perfectly normal and to be expected for any type of procurement. There is a question as to whether or not the Incumbent has an unfair advantage in this or other like procurements. The Tribunal is of the view that, indeed, the Incumbent may have an advantage from the experience that it has gained in past contracts, but that, in itself, is normal and is not considered to be unfair. Sometimes being an incumbent is a disadvantage, in that it may lock a supplier into a particular mode of operation with attendant costs, and leaves it unable to react when a new supplier puts forth an innovative approach for consideration.
With respect to the potential for the evaluation being conducted in an unfairly biased manner, the Tribunal is of the view that, given its view on the first and second parts of the grounds for complaint, this portion of the complaint is speculative in the circumstances. The Tribunal cannot predetermine and the complainant cannot predict with certainty that the evaluation will be conducted in a manner that violates the AIT. The evaluation process must take place before any complaint relating to the unfair application of evaluation criteria is entertained by the Tribunal. In this case, the complainant chose not to submit a bid, even though its representations, at the beginning of the process, resulted in the procurement being opened to competition. The Tribunal notes that two other potential suppliers did avail themselves of the opportunity to compete against the Incumbent for the contract.
The Tribunal is of the view that the Department has balanced its requirements and the concerns expressed by the complainant both before and after the publication of its Notice of Proposed Procurement up to bid closing.
Determination of the Tribunal
In light of the foregoing, the Tribunal determines, in consideration of the subject matter of the complaint, that the procurement was conducted according to the AIT and that, therefore, the complaint is not valid.
1. R.S.C. 1985, c. 47 (4th Supp.).
2. SOR/93-602, December 15, 1993, Canada Gazette Part II, Vol. 127, No. 26 at 4547, as amended.
3. As signed at Ottawa, Ontario, on July 18, 1994.
4. SOR/91-499, August 14, 1991, Canada Gazette Part II, Vol. 125, No. 18 at 2912, as amended.
5. Notice to signal to potential suppliers the government’s intent to limit tendering generally to one supplier and to indicate the justification therefor.
6. EW communications services, of which ACES is a part, are services that relate to Federal Supply Classification (FSC), Group 58 (Communication, Detection, and Coherent Radiation Equipment). This FSC’s group of goods and the services that refer thereto are excluded from the coverage of the North American Free Trade Agreement when procured by or on behalf of DND.
[ Table of Contents]
Initial publication: August 28, 1996