---

---

TABLE OF CONTENTS

• DETERMINATION OF THE TRIBUNAL
• STATEMENT OF REASONS
• • COMPLAINT
  • PROCUREMENT PROCESS
  • TRIBUNAL’S ANALYSIS
  • • Rated Criterion R.1.3
    • Rated Criterion R.2.3
    • Rated Criterion R.3.2
    • Rated Criterion R.4.1
    • Rated Criterion R.5.1
    • Rated Criterion R.5.2
    • Remedy
    • Costs
  • DETERMINATION OF THE TRIBUNAL

---

IN THE MATTER OF a complaint filed by Femme Cachee Productions Inc. under subsection 30.11(1) of the Canadian International Trade Tribunal Act, R.S.C. 1985 (4th Supp.), c. 47;

AND FURTHER TO a decision to conduct an inquiry into the complaint under subsection 30.13(1) of the Canadian International Trade Tribunal Act.

BETWEEN
FEMME CACHEE PRODUCTIONS INC.	Complainant
AND
THE DEPARTMENT OF PUBLIC WORKS AND GOVERNMENT SERVICES	Government Institution

Pursuant to subsection 30.14(2) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal determines that the complaint is valid in part.

Pursuant to subsections 30.15(2) and (3) of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal determines that the circumstances relevant to the procurement do not warrant the recommendation of a remedy.

Pursuant to section 30.16 of the Canadian International Trade Tribunal Act, the Canadian International Trade Tribunal awards Femme Cachee Productions Inc. its reasonable costs incurred in preparing and proceeding with the complaint, which costs are to be paid by the Department of Public Works and Government Services. The Canadian International Trade Tribunal’s preliminary indication of the level of complexity for this complaint case is Level 2, and its preliminary indication of the amount of the cost award is $2,400. If any party disagrees with the preliminary indication of the level of complexity or the preliminary indication of the amount of the cost award, it may make submissions to the Canadian International Trade Tribunal, as contemplated in the Guideline for Fixing Costs in Procurement Complaint Proceedings. The Canadian International Trade Tribunal retains jurisdiction to establish the final amount of the award.

Dominique Laporte
Dominique Laporte
Secretary

Tribunal Member:	Diane Vincent, Presiding Member
Director:	Randolph W. Heggart
Senior Investigator:	Cathy Turner
Counsel for the Tribunal:	Jidé Afolabi
	Eric Wildhaber
Complainant:	Femme Cachee Productions Inc.
Government Institution:	Department of Public Works and Government Services
Counsel for the Government Institution:	Susan D. Clarke
	Ian McLeod
	Karina Fauteux
	Lissa Mussely

Please address all communications to:

COMPLAINT

1. On July 14, 2009, Femme Cachee Productions Inc. (FCP) filed a complaint with the Canadian International Trade Tribunal (the Tribunal) under subsection 30.11(1) of the Canadian International Trade Tribunal Act ¹ concerning a procurement (Solicitation No. 5P131-090062/A) by the Department of Public Works and Government Services (PWGSC) on behalf of the Parks Canada Agency for the provision of video production services.

2. FCP alleged the following: (1) the evaluation team either made errors or deliberately discounted information provided in the bid, thereby unfairly deducting points; and (2) the evaluation team used criteria not listed in the Request for Proposal (RFP) to evaluate the bid. FCP requested, as a remedy, that the Tribunal recommend that PWGSC terminate the contract awarded to Sound Venture Productions (SVP) and award the contract to FCP. In the alternative, FCP requested that the Tribunal recommend that PWGSC compensate it for its bid preparation costs. It also requested that the Tribunal recommend that PWGSC provide FCP with written assurance that none of the concepts outlined in its creative proposal will be used by SVP in the production of the contracted video. FCP requested the reimbursement of its reasonable costs incurred in preparing and proceeding with the complaint.

3. On July 23, 2009, the Tribunal informed the parties that the complaint had been accepted for inquiry, as it met the requirements of subsection 30.11(2) of the CITT Act and the conditions set out in subsection 7(1) of the Canadian International Trade Tribunal Procurement Inquiry Regulations.²

4. On July 28, 2009, PWGSC informed the Tribunal that a contract had been awarded to SVP. On August 28, 2009, PWGSC filed a Government Institution Report (GIR) with the Tribunal in accordance with rule 103 of the Canadian International Trade Tribunal Rules.³ On September 25, 2009, FCP filed its comments on the GIR.

5. Given that there was sufficient information on the record to determine the validity of the complaint, the Tribunal decided that an oral hearing was not required and disposed of the complaint on the basis of the written information on the record.

PROCUREMENT PROCESS

6. On May 21, 2009, PWGSC issued the RFP. Bids closed on June 10, 2009. According to PWGSC, 12 companies submitted proposals in response to the solicitation. Three proposals, including that of FCP, were found to have met all the mandatory requirements and were awarded at least the minimum 75 percent for each of the point-rated requirements. Of the compliant proposals, SVP’s proposal received the highest score. FCP’s proposal received the third highest score.

7. According to FCP, on June 29, 2009, it received a letter dated June 19, 2009, from PWGSC informing it that its proposal would not be accepted, as it was not the proposal with the highest score.⁴ That letter contained a breakdown of the scores awarded to its proposal with respect to each of the point-rated requirements. Attached to that letter was the record of comments made by the evaluators in the course of the evaluation process. On June 30, 2009, FCP made an objection to PWGSC regarding its concerns about the evaluation results.⁵ On July 7, 2009, PWGSC responded to FCP’s objection.⁶ On July 14, 2009, FCP filed its complaint with the Tribunal.

TRIBUNAL’S ANALYSIS

8. Subsection 30.14(1) of the CITT Act requires that, in conducting an inquiry, the Tribunal limit its considerations to the subject matter of the complaint. At the conclusion of the inquiry, the Tribunal must determine whether the complaint is valid on the basis of whether the procedures and other requirements prescribed in respect of the designated contract have been observed. Section 11 of the Regulations provides that the Tribunal is required to determine whether the procurement was conducted in accordance with the applicable trade agreements, which, in this instance, is the Agreement on Internal Trade. ⁷

9. FCP submitted that, when it received information from PWGSC regarding the evaluation results of its proposal, it became clear to FCP that the evaluation team members had made several errors, had missed or ignored information that was clearly laid out in its proposal and had unfairly deducted points from FCP’s score. It also submitted that it was clear that some of the criteria used to evaluate the proposal were not contained in the RFP.

10. Article 506(6) of the AIT provides that “[t]he tender documents shall clearly identify the requirements of the procurement, the criteria that will be used in the evaluation of bids and the methods of weighting and evaluating the criteria.”

11. FCP mentioned it was unable to claim that its proposal was the most compliant bid;⁸ rather it disputed the score that its proposal received for six rated criteria.⁹

12. The Tribunal will examine each of these criteria in light of the following jurisprudence and the AIT. In previous decisions, the Tribunal has stated that it will not substitute its judgment for that of the evaluators unless the evaluators have not applied themselves in evaluating a bidder’s proposal, have ignored vital information provided in a bid, have wrongly interpreted the scope of a requirement, have based their evaluation on undisclosed criteria or have otherwise not conducted the evaluation in a procedurally fair way.¹⁰

Rated Criterion R.1.3

13. Rated criterion R.1.3 reads as follows:

Experience in a wide variety of visual and dramatic devices, such as graphic animation sequences; use of original artwork, still imagery and imported (stock) film footage; off-camera and on-camera narration. (30 points)

At a minimum, we are looking for: use of four (4) of the nine (9) above-mentioned interpretive video elements.

14. FCP disputes the following evaluation comment made with respect to rated criterion R.1.3:

There are a variety of elements throughout the clips, however, not a minimum of 4 interpretive elements in at least one specific clip.

15. FCP alleged that this comment is inaccurate and indicative of a misevaluation of its bid that resulted in a loss of points. FCP submitted that each clip contained a minimum of four interpretive elements and that FCP included documentation with its demo that listed, among other things, at least four interpretive elements for each clip that it submitted, as requested in the RFP.

16. PWGSC submitted that the evaluation of the submitted sample videos in relation to rated criterion R.1.3 must be read in the context of the provisions of mandatory requirement M.3, which reads as follows:

As a clarification, an interpretive video should have the following elements:

(1) Focus on the “subject” as a tangible resource.

(2) Provide the audience with an opportunity to form an emotional connection with the subject.

(3) Help the audience make intellectual connections to the subject.

(4) Relate the story of the “subject” to the viewer’s own life experience.

(5) Provoke the audience to think, do and/or feel something new.

(6) Tell the “subject’s” story in terms of universally understood concepts, or ‘big ideas’, such as fear and relief, hostility and peace, tradition and change, life and death, etc.

(7) Encourage the audience to care about the “subject” and the story it has to tell.

(8) Facilitate a connection between the individual and collective interests of the audience and the inherent meanings of the subject.

(9) Address a complete storyline, with a clear narrative.

17. The bidders were therefore required to submit a demo that included video clips that had at least four interpretative elements each. Rated criterion R.1.3 established that, for a minimum passing score of 75 percent, at least four of the nine elements had to be in a clip.

18. PWGSC submitted that, according to its letter to FCP dated June 19, 2009, the scores reflected the consensus conclusions of the evaluators. It further submitted that the comments attached to the letter, including the one above, were expressed by individual evaluators during the course of the evaluation process but may or may not have been reflected in the ultimate consensus decision. Finally, PWGSC submitted that the evaluators’ initial individual reactions were provided to each of the bidders to provide them with as much information and feedback as possible to assist them in the preparation of future proposals.

19. The Tribunal observes that each video clip submitted by FCP contained the required minimum of interpretative elements. If PWGSC had come to the conclusion that one of the video clips did not contain at least four interpretative elements, it would have been obligated to reject the proposal for not meeting rated criterion R.1.3, which it did not. A review of the evidence allows the Tribunal to conclude that the above comment is an inaccurate reflection of the basis for the scoring decision.

20. As a general observation, the Tribunal could not accept PWGSC’s approach that comments presented with the consensus score under each criterion evaluated, within a document adopted by an evaluation panel and attached to a letter informing the bidder of its evaluation, should be read as “. . . comments expressed by individual evaluators during the course of the evaluation process, but which may or may not have been reflected in the ultimate consensus decision.”¹¹ It is reasonable for FCP and any other bidders in similar circumstances to read these comments as being reflective of the views of the evaluation panel as a whole and being supportive of the consensus score, unless it is specifically stated that a remark comes from one of the evaluation team members only and was not reflective of the consensus score. In this instance, the Tribunal finds that PWGSC’s comment associated with rated criterion R.1.3 could have led FCP to understand that it did not get more than the minimum compliant score because of confusion as to whether each of the videos contained four interpretive elements. In this instance, taking into consideration the confidential information submitted in PWGSC’s response to the complaint,¹² the Tribunal does not find unreasonable for the evaluators to give a minimum score to the demo submitted by FCP, which includes the minimum number of interpretive elements in each of the video sequences submitted. Accordingly, the Tribunal finds that the evidence does not indicate that the evaluators acted unreasonably in applying their judgement with respect to the scoring of this criterion; they appear to have taken into account all applicable elements in FCP’s proposal. Therefore, the Tribunal will not substitute its judgement for that of the evaluators.

Rated Criterion R.2.3

21. Rated criterion R.2.3 reads as follows:

The degree to which the Evaluation Team is able to visualize the outline of the proposed production treatment, as well as its appropriateness. (25 points)

At a minimum, we are looking for: being able to clearly visualize both the structure and the creative approach of the production treatment of the script.

22. FCP disputes the following evaluation comment made with respect to rated criterion R.2.3:

Concept interesting but worried about using Piping Plover as the main bird due to its status of ‘Endangered Species’. The parks bird symbol is the Osprey and the Eagle is the spiritual bird for the Mi’kmaq.

23. FCP submitted that this comment is not consistent with the directions given in the RFP and is therefore indicative of a misevaluation of its bid that resulted in a loss of points. According to FCP, the RFP mentions the Piping Plover three times and requests that the video promote a conversation among viewers about it being an endangered species that is being rehabilitated at the park. It further submitted that neither the osprey nor the eagle is mentioned in the RFP and that neither one of them can be found in a long list of birds native to the park that is published on the Kouchibouguac Web site.

24. After reviewing the confidential information submitted in PWGSC’s response to the complaint for rated criterion R.2.3, the Tribunal is of the view that the evidence does not indicate that the evaluators acted unreasonably in applying their judgement with respect to the scoring of this criterion. Presented with a criterion that the Tribunal accepts as being inherently subjective, the evaluation panel appears to have taken into account all applicable elements in FCP’s proposal and to have evaluated the proposal against this requirement of the RFP. Therefore, the Tribunal will not substitute its judgement for that of the evaluators.

Rated Criterion R.3.2

25. Rated criterion R.3.2 reads as follows:

Project Plan/Schedule

A detailed work plan and project schedule including a project management chart/table (chart is to be readable with font NOT smaller than Arial 10pt or equivalent) which identifies the significant activities, milestones and deliverables. The work plan/chart should also include expected points of Client input, review and approval over the life of the Project in response to the requirements laid out in this statement of work. (25 pts)

At a minimum, we are looking for: Sufficient details on tasks, milestones and deliverable; timelines suitable and realistic; includes client involvement in plan and approach.

NOTE: Bidders should incorporate approval periods of 4 to 5 working days for client approval of scripts and the provision of feedback.

26. FCP disputes the following evaluation comments made with respect to rated criterion R.3.2:

The number of days is not always realistic for each approval stage.

The number of days for in-park shooting is identified and is somewhat realistic – not certain if it will be enough. We believe that 12 days is more realistic as opposed to 7 days.

27. FCP submitted that, with regard to the first comment, the number of days is realistic. It submitted that it scheduled more time than it needed in most instances. With regard to the second comment, FCP took issue with two aspects of the evaluation: (1) it stated that the RFP made no mention of 12 shooting days, whereas it appears that its bid in respect of this criterion is being judged against such a proposed time frame; and (2) the number of shooting days that any video requires is directly related to the creative concept of the video (e.g. some 90-minute feature films are shot in 18 days, whereas others take 60 days; much will depend on how they are shot). FCP submitted that, since each bid was supposed to have its own creative concept, it does not believe that there is an optimal number of shooting days applicable to all bids.

28. Taking into consideration the confidential information submitted in PWGSC’s response to the complaint, the Tribunal does not believe that the mention of “12 days is more realistic as opposed to 7 days” can reasonably be viewed, under the circumstances, as the introduction into the evaluation of a criterion not disclosed in the RFP. Rather, the Tribunal believes that the context of this comment can reasonably be understood as a comment made by the evaluators based on their own experience as to the time that would realistically be needed to accomplish the work set out in FCP’s detailed work plan and project schedule. In the Tribunal’s view, the evidence does not demonstrate that the evaluators acted unreasonably in applying their judgement with respect to the scoring of this criterion; they appear to have taken into account all applicable elements in FCP’s proposal. Therefore, the Tribunal will not substitute its judgement for that of the evaluators.

Rated Criterion R.4.1

29. Rated criterion R.4.1 reads as follows:¹³

Previous experience in producing similar videos in the same capacity.

30. FCP disputes the following evaluation comment made with respect to rated criterion R.4.1:

A minimum of 3 team members have worked together on at least one demo sample (only 3 have worked together on 3 samples). Details are provided slightly above the minimum points has more team members have worked together on the provided demo sample.

31. FCP alleged that this comment is inaccurate and indicative of a misevaluation of its bid and of the introduction of criteria into the evaluation process that were not set out in the RFP. FCP indicated that the maximum number of team members working on a sample was four. In 9 of the 12 samples at least 2 team members worked together, in 4 of the samples at least 3 team members worked together.

32. In the GIR, PWGSC acknowledged that the RFP did not set out a specific minimum in this regard. It indicated however that the evaluation team decided that a reasonable basis for an award of a minimum compliant score of 75 percent would be that at least three of a bidder’s proposed personnel performing in the seven roles identified in rated criterion R.4 should have worked together on one previous production. PWGSC also indicated that the evaluation team decided that additional marks could be awarded if more than three of a bidder’s proposed personnel in the seven identified roles had also previously worked together.

33. The Tribunal finds that the adoption of this evaluation methodology constitutes the introduction by the evaluators of criteria that were not identified in the RFP. Such a methodology could not have been understood from the wording of rated criterion R.4.1 as found in the RFP. This constitutes a violation of Article 506(6) of the AIT. This ground of complaint is therefore valid.

Rated Criterion R.5.1

34. Rated criterion R.5.1 reads as follows:

Unit Price Breakdown

The firm price should be further broken down into categories and line items for the goods and services required to deliver the completed project. Each line item should indicate the unit prices of hourly, daily or weekly rates as appropriate. The level of effort for each service and quantity of goods should be clearly indicated. There should be an extended price for each line item: the rate times the level of effort for services or unit price times quantity for goods. (25 pts)

At a minimum, we are looking for: sufficient budget details in terms of categories, line items, unit prices/rates, level of effort; consistent budget structures/breakdowns, for example, by production phase.

35. FCP disputes the following evaluation comment made with respect to rated criterion R.5.1:

Hard to compare schedule and cost breakdown as some in weeks others in days but budget often in hours. Hard to figure out travel days with the provided schedule; crew confusing as people have different per diem days.

36. FCP alleged that this comment is inaccurate and indicative of a misevaluation of its bid that resulted in a loss of points. FCP submitted that its proposal specified scouting days and production days, which corresponded exactly to the production costs in its budget. It also submitted that its proposal specified post-production time in increments which corresponded exactly to the post-production costs in its budget. FCP submitted that, in the travel costs section of its budget, all crew members, without exception, were assigned the same per diem.

37. Taking into consideration the confidential information submitted in PWGSC’s response to the complaint, the Tribunal is of the view that the evidence does not indicate that the evaluators acted unreasonably in applying their judgement with respect to the scoring of this criterion; they appear to have taken into account all applicable elements in FCP’s proposal. Therefore, the Tribunal will not substitute its judgement for that of the evaluators.

Rated Criterion R.5.2

38. Rated criterion R.5.2 reads as follows:

Correlation to Project Plan

The Bidder should provide sufficient detail in their financial proposal that demonstrates to the Evaluation Team how and where the money is being spent. The financial proposal should correlate and be justifiable in relation to the proposed Project Plan. (25 pts)

At a minimum, we are looking for: Sufficient detail on where money is allocated; correlation to project plan; assumptions, cost efficiencies and project plan.

39. FCP disputes the following evaluation comment made with respect to rated criterion R.5.2:

Travel estimates were not realistic ($400 for flights?).

40. FCP alleged that this comment is inaccurate and indicative of a misevaluation of its bid that resulted in a loss of points. FCP submitted that, while it did not include the source for the flight costs in its proposal, it is of the view that the evaluators have arbitrarily and mistakenly challenged them.

41. Upon reviewing the evidence, including the score allocated to this criterion, the Tribunal finds that the evidence does not demonstrate that the evaluators acted unreasonably in applying their judgement with respect to the scoring of this criterion; they appear to have taken into account all applicable elements in FCP’s proposal. Therefore, the Tribunal will not substitute its judgement for that of the evaluators.

Conclusion

42. In light of the foregoing, the Tribunal determines that PWGSC used criteria not listed in the RFP to evaluate FCP’s proposal, in violation of Article 506(6) of the AIT, and, therefore, finds that the complaint is valid in part.

Remedy

43. Having found one ground of complaint raised by FCP to be valid, the Tribunal must now recommend the appropriate remedy.

44. In this regard, subsection 30.15(3) of the CITT Act provides as follows:

(3) The Tribunal shall, in recommending an appropriate remedy under subsection (2), consider all the circumstances relevant to the procurement of the goods or services to which the designated contract relates, including

(a) the seriousness of any deficiency in the procurement process found by the Tribunal;

(b) the degree to which the complainant and all other interested parties were prejudiced;

(c) the degree to which the integrity and efficiency of the competitive procurement system was prejudiced;

(d) whether the parties acted in good faith; and

(e) the extent to which the contract was performed.

45. The Tribunal considers that not evaluating a proposal in accordance with the criteria provided in the RFP represents a serious deficiency in the procurement process which prejudices the integrity and efficiency of the competitive procurement system. Bidders need to rely on the prescribed evaluation criteria to formulate their proposals. Otherwise, bidders are unable to optimize their efforts to be the successful bidder. However, the evidence does not indicate that the evaluators were not acting in good faith when they conducted their evaluations.

46. Taking all aspects of this case into consideration, the Tribunal finds that there is no effect on the ultimate result of the competitive process for FCP, in terms of becoming the most compliant bid or winning bid, that is attributable to the violation of Article 506(6) the AIT identified above. The Tribunal accepts PWGSC’s submission that, even if FCP had received perfect scores in all of the rated criteria in dispute, its proposal would still have received fewer technical points than the winning proposal. The same is true, of course, if there is a potential increase up to the maximum of points for only one of these criteria, as is the case here. Given the Tribunal’s finding that only one rated criterion was improperly evaluated, even if the proper evaluation of that criterion had resulted in a perfect score, such an increase in points would still have been insufficient for FCP’s bid to outrank the winning proposal. Accordingly, the Tribunal finds that there is ultimately no consequent prejudice suffered by FCP as a result of the valid ground of complaint that is identified above. As such, the Tribunal will not recommend a remedy in this case.

Costs

47. The Tribunal awards FCP its reasonable costs incurred in preparing and proceeding with the complaint. In determining the amount of the cost award for this complaint case, the Tribunal considered its Guideline for Fixing Costs in Procurement Complaint Proceedings (the Guideline), which contemplates classification of the level of complexity of cases based on three criteria: the complexity of the procurement, the complexity of the complaint and the complexity of the complaint proceedings. The Tribunal’s preliminary view is that this complaint case has a complexity level corresponding to the second level of complexity referred to in Appendix A of the Guideline. The procurement was moderately complex as it dealt with issues of subjective evaluation criteria relating to the video production services. The complaint proceedings were not complex as there were no motions, no interveners and no additional submissions by parties. Accordingly, as contemplated by the Guideline, the Tribunal’s preliminary indication of the amount of the cost award is $2,400.

DETERMINATION OF THE TRIBUNAL

48. Pursuant to subsection 30.14(2) of the CITT Act, the Tribunal determines that the complaint is valid in part.

49. Pursuant to subsections 30.15(2) and (3) of the CITT Act, the Tribunal determines that the circumstances relevant to the procurement do not warrant the recommendation of a remedy.

50. Pursuant to section 30.16 of the CITT Act, the Tribunal awards FCP its reasonable costs incurred in preparing and proceeding with the complaint, which costs are to be paid by PWGSC. The Tribunal’s preliminary indication of the level of complexity for this complaint case is Level 2, and its preliminary indication of the amount of the cost award is $2,400. If any party disagrees with the preliminary indication of the level of complexity or the preliminary indication of the amount of the cost award, it may make submissions to the Tribunal, as contemplated in the Guideline. The Tribunal retains jurisdiction to establish the final amount of the award.

---

¹ . R.S.C. 1985 (4th Supp.), c. 47 [CITT Act].

² . S.O.R./93-602 [Regulations].

³ . S.O.R./91-499.

⁴ . Confidential GIR, exhibit 2.

⁵ . Ibid., exhibit 4.

⁶ Ibid., exhibit 5.

⁷ . 18 July 1994, C. Gaz. 1995.I.1323, online: Internal Trade Secretariat <http://www.ait-aci.ca/index_en/ait.htm> [AIT]. Video production services are excluded from the North American Free Trade Agreement per Annex 1001.1b-2, Section B, as they fall under category T, Communications, Photographic, Mapping, Printing and Publications Services. The Agreement on Government Procurement does not apply because video production services are not in the list of included services detailed in Annex 4. Video production services are excluded from the Canada-Chile Free Trade Agreement per Annex Kbis-01.1-4, Section B, as they fall under category T, Communications, Photographic, Mapping, Printing and Publications Services.

⁸ . FCP’s comments on the GIR at 11.

⁹ . FCP acknowledged that it had made an error relating to rated criterion R.3.1. Therefore, the Tribunal will not further assess this criterion.

¹⁰ . Re Complaint Filed by Vita-Tech Laboratories Ltd. (18 January 2006), PR-2005-019 (CITT); Re Complaint Filed by Marcomm Inc. (11 February 2004), PR-2003-051 (CITT).

¹¹ . GIR at 10, para. 15.

¹² . This confidential version of PWGSC’s response to the complaint was made available to FCP.

¹³ . Although the complaint refers to rated criterion R.4.1, the substance of the comments in the complaint is made in reference to rated criterion R.4.2, which reads as follows: “The proposed person[nel] are the same individuals who were involved, in the same capacity, in the production of the demo samples.”