API Gateway Transition: Status at the Midpoint (10/07/22)

Key Facts:

  • Five weeks remain to switch to the new API gateway and secure your credentials.
  • We've discovered and squashed several bugs.
  • We want (and need) your feedback.

Blue pill or Red pill?

It's been a busy two months since we officially released the new developer portal and API service gateway. A good number of you all have at least dipped your toe in the water and starting making calls through the new gateway (thanks!), and an even greater number of you still need to take the leap:

 Histogram of applications using the new API gateway, by percentage of total use.

This graph shows that 36 of 440 active applications (8%) are now using the new gateway for at least half of their calls. Twelve have even moved over entirely! But it also shows that 84% haven't made a single call to the new gateway yet. 

Here's a better look at the rate of transition:

Line graph comparing daily calls to the old and new gateways

The (blue) trend line for the number of calls made daily to the old gateway has gone down from ~1.9 million to ~1.75 million, and the (red) trend line for production use of the new gateway has the corresponding upward slope–but if we extend those out, it would take us until March to switch! We need to work together to reach our goal of all API traffic on the new gateway by Friday, November 11.

Remember that you'll need to secure all of your API credentials elsewhere than the developer portal by that deadline, too. (See our previous message about developer portal changes for details.)

Go ahead, choose the red pill!

Finding and Squashing Bugs

While we subjected the new gateway to a barrage of tests prior to its release, we always knew that the most complete testing would require all of your help–by actually using it in all the varied ways you do. Thanks to your individual feedback, we've discovered a number of issues and been able to provide explanations, work-arounds, or permanent resolutions.

These include:

  • "User-Agent" headers now required in all requests (not a flaw, but a change in requirements)
  • Outdated versions of HTTP request libraries cause requests to be rejected (also not a flaw)
  • Header keys not handled as case insensitive (resolved)
  • Trailing forward slash followed by query parameters causes requests to be rejected (resolved, temporarily)
  • New gateway not reachable from campus RFC 1918 private networks (expected, solution and work-around provided)
  • Underscores in "app_id" and "app_key" header keys now required (not a flaw, but a change in requirements)

Almost all of these issues are rooted in the enhanced precision provided by AWS' WAF firewall service, So while they do cause temporary headaches, they also result in more secure integrations for us all.

Tell us how it's going

None of these discoveries could have been made without you. We're striving to make sure all the work you all do to transition your applications to the new gateway is of benefit to everyone, so please keep us informed of any stumbling blocks you encounter, or best practices you suggest. We'll add them to the website and include them in the next email.

If you have any questions or suggestions, please reach out to us at eis-support@berkeley.edu or # api-transition-help on bIT Slack.