Agency Human Services Dept

Contractor Shepherd Data Services

SWIFT Contract Number 201717

Project Name doc review/JUUL

Project Number (if applicable) {Empty}

Summarize purpose of the contract, including why it was necessary: To provide document review services in State of Minnesota, ex. Rel. Ellison v. JUUL Labs, Inc.

Project Duration

Start Date Fri, 10/01/2021 - 00:00

Original end date Thu, 08/31/2023 - 00:00

Actual end date Thu, 08/31/2023 - 00:00

Did project finish on time? Yes

Total billable hours utilized (if applicable) {Empty}

Contract Amounts

Original contract: $200000

Amendments $-159653

Contract total $40347

Did project finish within budget? Yes

What was the funding source? DHS Behavioral Health Division program budget

Is this a sole source contract? No

If this was a single source contract, explain why the agency determined there was only a single source for the services (if applicable) {Empty}

Evaluator Name Alexandra McDonough

Evaluator Email Address alexandra.mcdonough@state.mn.us

Evaluate the contractor's timeliness Timeliness:



The vendor estimate that the project would be completed within 21 calendar days of project inception. The document reviewers began on January 17, 2022. When the vendor needed an amended work order for secure additional funds, the updated estimate indicated that additional review hours were also necessary and estimated that work would be completed by February 25, 2022. Work on the project had to stop while DHS secured an amended work order. When the vendor did not finish the review by this amended timeline and needed additional funds to host the data for another month, the vendor's anticipated project completion was again amended, this time to March 31, 2022. The vendor completed the review on March 30, 2022. However, it ultimately took the vendor over two months to review only 10,000 documents, which was far longer than the original estimate and disappointing.

Evaluate the contractor's quality Although it was only 10,000 documents and the document set consisted primarily of emails, the number of errors in the review was unacceptable. The frequency of errors combined with the vendor's underestimation on review time resulted in a DHS staff attorney spending hundreds of hours performing a quality control review as part of the project team.

Evaluate the contractor's cost DHS had originally expected to need many more documents reviewed, so the first work order was amended to reflect the vendor's estimate to review to an updated document count. The first estimate was $22,000. However, the vendor then informed DHS that it was going to need an additional nearly $16,800 to complete the review. Because the work was delayed due in part to the high number of errors and in part to the project having to stop while a new work order was secured, the vendor then requested an additional $1465 to host the data for an additional month. DHS ultimately spent $40,347 to have 10,000 documents reviewed. This was twice the original estimate and nearly $3 per document. For this cost we would have expected the quality to have been better.

Evaluate the contractor's overall performance While the project was ultimately completed and we did not have any concerns about the production from the subpoenaing party, the additional requests for funds and project delays combined with the need to devote internal employee resources to this project was unsatisfactory.

Would you engage the contractor's services again? No

Describe any negative action taken during the duration of this contract (i.e., termination, pursuit of suspension, etc.): {Empty}

Please rate the overall experience with the contractor 1 - very dissatisfied