Scalable Solution for

High-Intensity COVID-19

Vaccine Registration

The Challenge

At the end of January, the nationwide COVID-19 vaccine rollout was in full swing. But Pennsylvania, with the fifth-highest COVID-19 death toll, lagged behind almost every other state in distributing the vaccine, mainly due to a lack of a state-coordinated central vaccine registration site.

As one of Pennsylvania’s largest integrated healthcare payers and providers, UPMC felt a strong obligation to contribute to a solution – not only for its 4 million health insurance members, but to everyone in the communities of Western Pennsylvania where it is headquartered. Because of the urgency and critical nature of this request, UPMC turned to Computer Enterprises Inc. (CEI) to build a trusted and reliable vaccine registration system as quickly as possible.

Due to the critical, time-sensitive nature of the need, UPMC required the support of a trusted resource that already knew their people, processes, and concerns of their business. This project required an increase in dedicated hours from CEI, along with coordinated efforts across many teams in the UPMC organization. “We are doing the very best we can to create a robust system that will handle the amount of demand and volume we are expecting,” said Medical Director of Infection Prevention and Hospital Epidemiology at UPMC Dr. Graham Snyder.

 

The Solution

With that mandate in mind, the web team, spearheaded by Balaji Kandasamy from CEI and Rich Collier from UPMC, quickly began solutioning a forms application that could handle the intensity and large-scale number of submissions anticipated given the pent-up demand. Load testing tools proved that the capacity of the existing UPMC form solution would not meet the demand of vaccine seekers.

To ensure that UPMC could handle an influx of submissions, the requirements for the form were scoped to withstand a peak of 10,000 submissions per minute.  The team decided to streamline the front end to a lightweight HTML and JavaScript page that relied on scalable web services. Because of the compressed timeline, the CEI and UPMC teams put in truly heroic efforts working long hours over several weeks to build and integrate a forms solution based on the Azure platform that could handle the requirements.

As the team configured key Azure resources, they consulted with other CEI architects with proven histories of delivering successful solutions, such as Sivamuthu Kumar.  When the initial architecture had been built and tested, CEI reached out to Microsoft in collaboration with UPMC’s Web and Security teams to review and validate the architecture. The success of this project hinged upon it being a cross-team, cross-function, cross-organization effort, executed under great pressure at high speed where each party knew and performed its role flawlessly.

 

The Results

The solution went live on February 17, 2021 and has handled over 498,600 submissions (120 submissions per minute at peak) for vaccine registration to date, with no issues reported.

The UPMC vaccine registration site provides a trusted and reliable vaccination registration and scheduling system that has improved access to the vaccine. It includes options for people who do not use computers or smartphones, providing greater accessibility to those most at risk of severe COVID-19 illness and to communities most harmed by this pandemic.

As of March 15, 2021, UPMC had administered 237,726 doses of the vaccine, including 5,000 in one single day at a mass vaccination clinic at PPG Paints Arena. As supply increases, UPMC anticipates being able to schedule up to 80,000 vaccinations per week, in large part enabled by form solution built by CEI, and contributing greatly to Pennsylvania improving from a rank of 49th in percentage of population fully vaccinated to 30th as of April 7th, 2021.

“Although work is still being done, it was and is an important project and the team did a fantastic job.  Everyone part of that project should be very proud.”

The data gained from the registrations also allows UPMC to reach its members and patients for outreach, follow up, and greater engagement for its offerings.

 

The Technology

To allow the application to handle large bursts of submissions, while providing a responsive user experience, we leveraged:
  • CDN
  • Service Bus
  • Function Apps
To ensure that the system was protected, and data was secured, we used:
  • Application Gateway
  • Azure Table Storage
To provide a combined view of usage and performance of the system, we utilized:
  • Azure Application Insights
To build a solid base for management of static assets and integration with other services (such as CDN), we used:
  • Web Applications