Introducing API Central's New Platform Project (06/21)

We're transitioning from a mix of platforms to full use of the Amazon Web Services cloud platform.

While most changes are occurring behind the scenes and will be transparent to our clients, we'd like to familiarize you with the project, highlight the impacts, and preview upcoming communications.

Benefits

Lower cost

Due to economies of scale, paying only for actual use rather than fixed capacity, and more automated operational tasks, our costs will be drastically reduced by moving to the public cloud.

Increased and scalable capacity

Even while saving lots of money, we'll be able to increase the platform's overall throughput considerably. In addition, during peaks in activity the platform will scale up automatically to maintain performance.

Improved security

The public cloud boasts a more secure environment than even our private network. It attracts the best security personnel available, has been mercilessly tested by benign and malicious hacking, and features the latest technology and patches.

Better operational resiliency

As great as our operations engineering staff is, they just can't compete with the collective knowledge, cognitive bandwidth, and sleeplessness of the scores of engineers AWS employs.

Faster, easier business resumption

The redundancy, physical distribution, and abstracted design of public cloud infrastructure results in better mitigation of, response to, and recovery from any conceivable disaster.

Greater choice of future enhancements

Rather than being locked into a vendor's roadmap, we can meet Berkeley's particular evolving integration needs by making use of AWS' large and growing menu of modular IaaS and PaaS offerings.

Details

Our API management service consists of two components–both changing as part of this transition.

API Central Developer Portal

The web application at "api-central.berkeley.edu" is the portal used to explore APIs and manage access to them. This application is already running in Amazon's Elastic Compute Cloud (EC2), and will remain largely the same.

Planned impacts include:

The host for the portal will no longer be “https://api-central.berkeley.edu” (specifics to come).
The secret "app_key" portion of credentials will no longer be stored in the portal, and will be displayed only once upon creation, so you will need to copy, secure, and manage these locally. (The recommended way to do this is to create a group LastPass Enterprise account to manage them.)
The form used to submit or change APIs will be more detailed to meet the requirements of Amazon's API Gateway (see below).
No customer updates to accounts, credentials, or API contracts will be allowed on the deprecated portal during the transition period—all changes must be made on the new portal only.¹

API Service Gateway

The proxy service at "apis.berkeley.edu" forms a gateway unifying access and obfuscating physical locations of managed APIs. We are replacing this on-premise load-balanced NGINX with AWS Application Load Balancer and NGINX running in EC2. Behind the scenes, we've been using Red Hat's 3Scale product to manage access to and control usage of APIs. The greatest impact to our service is that we’re replacing this with Amazon's API Gateway. (For now, fine-grained authorization will still be provided by our on-premise IBM DataPower appliance.)

Planned impacts include:

The host used in every API call will need to be changed from “https://apis.berkeley.edu” (specifics to come).
Hosts in Berkeley's RFC 1918 private network space will need to be configured for Network Address Translation (NAT) service in order to make API calls.
All APIs will have an upper limit of calls accepted per minute and per day by any specific client. These initially will be set so far above current usage rates that no legitimate client will experience a difference. Over time API providers may choose to adjust these to protect backend services better.

Milestones

While exact dates aren’t set yet, these are our upcoming major steps:

Mid July:

New production-quality platform available for use
Existing platform deprecated and frozen during a three month transition period

Mid October:

Deadline for consumers to change their applications to call APIs via the new gateway
Deprecated platform taken down

Future Communications

In the coming weeks, we'll send targeted email

to API providers:

Detailing applicable API Central portal and gateway changes
Declaring initial rate limits for each of the APIs they own
Alerting to the approaching transition phase update freeze

to API consumers:

Detailing applicable API Central portal and gateway changes
Alerting those making calls from RFC 1918 space to arrange for NAT service
Declaring initial rate limits for every API
Alerting to the approaching transition phase update freeze

During the transition phase, we'll send targeted email