What we learned from building a pool of agile vendors

In 2015, 18F had an idea for a better way for federal agencies to hire private vendors to build products and services using agile development techniques. We wanted to see what would happen if we used an existing government process (called a blanket purchase agreement) and tailored it to the needs of agencies looking to update their digital services.

What we tried

18F’s goal was to create a pool of vendors capable of delivering software in an agile way. Once we selected these vendors they would be placed into a contracting vehicle called the Agile Delivery Services Blanket Purchase Agreement, or Agile BPA for short. Then, agencies could work with 18F to select vendors from this small pool to work on technology projects. We set up the BPA in the hopes of streamlining the acquisition process to get program teams what they need sooner.

To select the vendors for the BPA, 18F selected GSA’s Schedule 70 as the pool from which to create the vehicle. We started with a Request For Information (RFI) and an industry day to better understand how to acquire agile delivery services up to the standards of 18F. From there, we created a Request For Quotation (RFQ) where vendors were evaluated based on a working prototype instead of a narrative document. We ultimately awarded the contract to 17 vendors, both large and small, from all over the country.

We knew we wanted great vendors that could build software using agile and user-centered methodologies and that we needed these types of acquisitions to work at the speed of agile development cycles. When we set out to begin issuing task orders, we intended the projects to take less than four weeks from solicitation to contract kickoff, and from there no more than three months to deliver a minimum viable product for our agency partners.

Today, we’re taking a look back at where the Agile BPA was able to deliver value to federal agencies and where it fell short of our goals for the project. In August 2015, we looked at what we learned during the solicitation process, and this blog post is the next step in learning from this experiment.

How the Agile BPA helped

The Agile BPA has helped 18F work collaboratively with great vendors and federal agencies to provide tremendous value to the public. Since 2015, 18F has awarded nine task orders off of the Agile BPA. Here are some examples of our successes:

A culture of trust

The Agile BPA helped us build a collaborative relationship grounded in trust between government and vendors. Many of our agency partners have seen past technology acquisitions fail to meet their goals, while vendors have told us that government agencies aren’t always prepared or adequately staffed to dedicate the amount of time it takes to complete a project. 18F served as the Contracting Officer’s Representative (COR) on all the buys and ensured that all packages contained transparent pre- and post-award procedures to set clear expectations and responsibilities. We experienced time and time again that the partnership between Agile BPA vendors, 18F, and our partner agencies created the kind of trusted relationship required to build and deploy complicated software projects.

Forest Service ePermitting

With the U.S. Department of Agriculture’s Forest Service, we awarded and completed three task orders off of the BPA to help them build a way for the public to submit permit applications online. This tool will streamline the process for both Forest Service employees and members of the public who want to use Forest Service lands. With our vendor partners, we’ve built a user intake form that connects to an existing legacy database within the Forest Service, as well as a self-service Christmas Tree Permitting system that allows members of the public to pay for a permit online and print their permit at home.

Department of State TalentMAP program

TalentMAP is a new system designed to transparently match and manage Foreign Service employees to work assignments around the world. The Department of State, 18F, and our vendor partner created a new search function and navigation experience for TalentMAP. This allows Foreign Service employees to understand and analyze positions as well as information on the locales in which those positions were posted. The hope is that this new system will allow State to be able to better carry out its diplomatic mission by improving the searchability and accuracy of position and competition data, streamlining the matching process, and reducing the amount of time it takes for Foreign Service employees to find and fill jobs.

Defense Information System Agency e-QIP program

18F and our vendor partner built a new user interface for the applicant-facing functionality of e-QIP, the central system for managing background checks for federal employees. The project, called eApp, maintains the existing purpose of e-QIP while prioritizing usability and validating data to improve the quality of submitted information. These enhancements will help people complete the lengthy and complex SF-86 while reducing the agency administrative burden required to manually review forms.

How the Agile BPA didn’t meet our goals

While the Agile BPA has been able to help us deliver great work for our agency partners, it wasn’t able to deliver in all of the ways we were hoping when we issued the RFQ in 2015:

Time to award

18F does not have the authority to directly procure services for partner agencies, so we work with other components of the General Services Administration to use the BPA. That process has involved many stakeholders and led to a slower-than-anticipated procurement process. Originally, we had a goal to award task orders in a matter of weeks, but over the course of two years we did not improve upon our timeline of 90-120 days.

Competition

18F struggled to advertise task orders with enough lead time for vendors to develop staffing proposals. Our modular contracting (FAR Part 39) procurement strategy includes smaller software development buys, but the process incentivized much greater competition for larger task orders versus smaller ones. On top of that, we issued a lower volume of orders than anticipated when we set up the BPA in 2015, so vendors could not guarantee a steady stream of work from the Agile BPA alone. Therefore, a number of 4-6 month task orders only received a couple of bids. Given the different problems we’re trying to solve with our partners, we want to ensure that their projects continue to see robust competition and ensure quality commensurate with cost.

User research resources

The original plan was for 18F to create three pools of vendors in the Agile BPA, including a mix of skills, including, notably, user research skills. GSA ultimately awarded only one pool, representing full stack development skills, but not the breadth of user research talent we’d envisioned. Subsequently, some vendors did not propose strong user research talent when bidding on work, which put a burden on 18F to staff these positions with our own team or to provide coaching to the vendor teams. This process didn’t give our agency partners all the talent they needed for user-centered agile software development and increased the cost of a handful of our projects.

Onboarding and offboarding

We saw a set of vendors in the BPA consistently not bidding on task orders. When asked about the reason for abstaining, some indicated that the work did not align with their core company strengths or business objectives. However, we didn’t have a mechanism to easily offboard vendors that weren’t interested in bidding on the work going forward. Since we created the BPA from GSA Schedules, onboarding new vendors to the vehicle would have required using the same evaluation criteria from the original solicitation. This would have replicated the problems we already experienced with getting the right labor category mix and required our staff to take time away from client work.

Conclusion

We’ve enjoyed working with the vendors on the Agile BPA, and we hope they continue to participate in our acquisitions as we iterate on our process. We’re determining our next steps, so keep an eye out for future procurements. In the interim, you can take a look at all of our task orders, procurement packages, and deliverables on the Agile BPA site.

This blog post was written with input from Laura Gerhardt, Hannah Kane, and Waldo Jaquith.