From AWS API Gateway to Tyk: A Seamless Migration With Significant Benefits

At Halodoc, the healthcare platform is built on a microservices architecture, designed for scalability and flexibility. The API Gateway plays a crucial role in our systems architecture, primarily handling request routing by directing requests to the appropriate services. It also serves as a vital component for API security, safeguarding underlying data by controlling traffic and enforcing authentication and authorization. While AWS API Gateway initially served us well, our growing system required more granular control, customization, and better integration with our Kubernetes-based infrastructure. After careful consideration, we chose to migrate to Tyk API Gateway.

In this blog, we’ll explore why we made this switch, its benefits, and our strategy for a smooth migration.

Issues with AWS API Gateway:

While AWS API Gateway is a powerful tool for managing APIs, it presented some challenges for us. One of the primary issues was the lack of integration with centralised authentication component. With API Gateway, authentication often had to be implemented at the individual service level, which became redundant.

Additionally, AWS API Gateway does not natively support aggregator APIs—which combine data from multiple backend services—requiring us to use Lambda functions or create dedicated microservices for aggregation. This added complexity and overhead to our system.

Another challenge was the lack of built-in flexibility for custom request transformations. While AWS API Gateway supports basic functionality like rate limiting and method transformation, more advanced customisations often required additional Lambda functions, increasing operational overhead and costs.

Understanding Tyk and Why It Was Chosen:

Tyk is an open-source, lightweight API Gateway that leverages GraphQL, focusing on API governance and microservice development. It offers flexibility and customization, making it an ideal choice for organizations adopting or maintaining microservices architectures. Its targeted functionality helps teams efficiently build and manage APIs for modern, cloud-native applications.

The decision to choose Tyk was driven by its flexibility, scalability, and ability to integrate seamlessly into existing infrastructure and here’s why Tyk stood out:

• Underlying Technology: GoLang was used in the development of Tyk, guaranteeing a high-performance, lightweight, and independent solution. Tyk offers a set of open-source tools that enhance its core platform and provide better performance.
• Centralized Authentication and Authorization: Initially, we implemented authentication logic at the service level, which became redundant as our services expanded. Tyk provided the capability to handle centralized authentication at the gateway level, eliminating the need for individual services to manage it. This simplified our authentication logic while enhancing security.
• Efficient Request Handling: Aggregating data from multiple services through a single virtual endpoint simplifies client interactions by consolidating responses from internal and upstream APIs. This reduces complexity, eliminates the need for a separate aggregation service, and offloads processing to Tyk, streamlining the architecture and improving efficiency.
• Custom Plugins and Middleware: Tyk’s support for custom plugins in multiple languages, including Python, JavaScript, and Go, allowed us to build dynamic solutions for authentication and request transformations. Alongside built-in features like method transformation, whitelisting, and rate limiting, this flexibility gave us greater control over API behavior and traffic management, enhancing our overall API infrastructure.
• Comprehensive Features and Flexibility: Tyk Gateway is a comprehensive, standalone API management solution offering a broad range of features, while Amazon API Gateway is a more lightweight service that requires integration with other AWS services to address similar API management needs, resulting in less flexibility, often resulting in additional costs for those services.
• Lambda Integration for Concurrency: For high-concurrency tasks, we continued leveraging AWS Lambda. Tyk allowed us to directly invoke Lambda functions through Lambda function URLs, with an additional layer of authentication handled by AWS IAM. This integration ensured that our serverless computing requirements were met without additional overhead.
• Seamless Kubernetes Integration: Tyk’s native support for Kubernetes made it an ideal choice for our microservices-based architecture. We deployed Tyk in our Kubernetes environment, leveraging Helm charts to automate scaling, load balancing, and service discovery. This integration allowed us to manage API traffic efficiently and scale dynamically as traffic increased.
• Support and Community: Tyk has great support and a vibrant open-source community. Most problems can be resolved through comprehensive documentation or direct assistance from Tyk support. Additionally, the active community offers valuable guidance for tackling more complex challenges.

Architecture Overview:

Our architecture is designed for efficient request routing and secure API management, integrating both microservices and serverless functions. Client requests are routed through AWS CloudFront for content delivery and then forwarded from CloudFront to the Application Load Balancer (ALB) of the Tyk API Gateway.

Upon reaching Tyk, the first step is to perform an initial validation. Tyk checks for a custom header, ensuring only authorized clients can access our APIs. This adds an extra layer of security, verifying requests before any further processing takes place.

Depending on the API’s use case, Tyk may execute middleware logic such as authentication, rate limiting, or HTTP request transformations. Once the validation and middleware processing are complete, the request is forwarded to the appropriate backend microservice via its service URL. This step involves routing the request to the designated microservice, which processes the request and sends the necessary response back through the same route.

For APIs that interface with AWS Lambda, Tyk adds a layer of security by leveraging AWS IAM authentication. After validating the request, Tyk invokes the Lambda function directly through Lambda function URLs, ensuring secure execution of serverless functions. This integration of Tyk with Lambda allows us to run specific business logic in a cost-efficient manner, without having to rely on dedicated microservices for every task.

Deployment Setup:

To deploy the Tyk API Gateway, we leveraged Helm charts available in the open-source Tyk repository. The deployment includes two primary components: Tyk Gateway and Tyk Pump. Tyk Gateway manages all API request functions, such as routing, authentication, and transformations, while Tyk Pump focuses on monitoring and analytics by pushing API request data to external data stores for further analysis. We automated the deployment using GitOps with ArgoCD, employing the app-of-apps pattern to manage Kubernetes resources.

In addition to Tyk components, we also needed to deploy Redis which in our case is AWS-managed, as Tyk leverages Redis for multiple purposes. Tyk primarily uses Redis to store short-lived analytics data from API requests, which are later consumed by the Tyk Pump and pushed to external stores like Elasticsearch. Redis also stores tokens, policies, quotas, and rate-limiting information for Tyk, though in our setup, we only used it to store analytics data. This data is purged by the Tyk Pump after processing, ensuring Redis remains optimized.

The specific Redis key used for analytics is a list called "analytics-tyk-system-analytics", where Tyk stores the request logs. We deployed a Redis cluster in AWS and configured the Tyk components to use the Redis endpoint for efficient data storage and retrieval.

Deployment Strategy for Tyk API Gateway Migration:

To ensure a smooth transition from AWS API Gateway to Tyk API Gateway, we adopted a Canary Deployment Strategy using CloudFront for traffic management.

1. Stage Distribution Setup: A replica of the production CloudFront distribution was created, allowing us to route a small portion of traffic to Tyk for testing without affecting the entire system.
2. Canary Traffic Routing: Initially, only 1% of the traffic was routed through Tyk, while the rest continued to flow through AWS API Gateway. This allowed for careful monitoring of Tyk’s performance with real traffic while minimizing risk.
3. Incremental Rollout: As performance remained stable, we gradually increased Tyk’s traffic share. By closely monitoring metrics like latency and error rates, we were able to address any emerging issues early. In the event of any problems, we swiftly reverted the traffic weight to the stable distribution.
4. Full Transition: Once Tyk demonstrated reliable performance under increased load, we transitioned 100% of the traffic to Tyk. This canary approach ensured a smooth, step-by-step migration with minimal disruption.

By controlling traffic allocation and monitoring performance throughout, we minimized risk and ensured a stable transition.

Monitoring:

Tyk provides support for monitoring with multiple backends, allowing for flexible integration with your preferred observability tools. In our case, we primarily use New Relic for monitoring and observability. Integrating New Relic with Tyk was straightforward by injecting the following Environment Variables into our Tyk configuration:

• TYK_GW_NEWRELIC_APPNAME: Sets the application name in New Relic.
• TYK_GW_NEWRELIC_ENABLEDISTRIBUTEDTRACING: Enables distributed tracing for deeper insights across services.
• TYK_GW_TRACER_ENABLED: Activates tracing in Tyk to monitor request paths and latencies.
• TYK_GW_NEWRELIC_LICENSEKEY: Adds our New Relic license key to enable data collection.

With this setup, we can efficiently monitor API traffic, performance, and any anomalies directly through New Relic, ensuring full visibility into the health of our API Gateway.

Resource Allocation:

Performance Metrics and Observations:

During performance testing with a single Tyk pod, we evaluated two types of APIs under different workloads. For a heavy logic-performing API, with 250 concurrent threads and a 10-minute test duration, Tyk achieved a peak throughput of 10,000 RPM. The average response time was 1400ms, with CPU utilization at approximately 35% and memory usage at 16.4%. In contrast, for a medium complexity API, Tyk reached a significantly higher throughput of 90,000 RPM, with an average response time of 224ms. Under this load, CPU utilization peaked at 80%, while memory usage remained consistent at around 17%. This demonstrated Tyk’s ability to handle both complex and moderately complex workloads efficiently. Additionally, Tyk demonstrated the capability to handle 2x the production throughput for PUT requests with enabled authentication. With only the authentication plugin enabled, Tyk reached 160k requests per minute (RPM) using 4 pods, achieving an average latency of less than 100 ms. In performance tests with 50 threads, Tyk consistently responded within ~40ms, proving its efficiency in handling authentication requests with minimal latency.

Conclusion:

The migration from AWS API Gateway to Tyk API Gateway has delivered significant improvements in API management, resource utilization, and cost efficiency. By transitioning 18 AWS API Gateway instances to a standalone Tyk deployment, we eliminated the high costs associated with per-request pricing.

Following the migration and the cleanup of AWS API Gateway, we optimised our microservice deployment, reducing the number of service pods from 8 to 3. The migration resulted in a monthly cost saving of approximately $1800. Additionally, implementing centralised authentication and fine-tuning our infrastructure.

This migration underscores our dedication to evolving our architecture to meet the demands of a growing platform while maintaining efficiency. We hope this guide provides valuable insights for others planning a similar migration, highlighting the challenges faced and solutions implemented during the process.

References:

https://tyk.io/docs/apim/open-source/
https://tyk.io/blog/harnessing-the-power-of-serverless-architecture-with-aws-lambda-and-tyk/
https://tyk.io/docs/plugins/supported-languages/golang/
https://aws.amazon.com/blogs/networking-and-content-delivery/use-cloudfront-continuous-deployment-to-safely-validate-cdn-changes/

Join Us

Scalability, reliability, and maintainability are the three pillars that govern what we build at Halodoc Tech. We are actively looking for engineers at all levels and if solving hard problems with challenging requirements is your forte, please reach out to us with your resumé at careers.india@halodoc.com.

About Halodoc

Halodoc is the number one all-around healthcare application in Indonesia. Our mission is to simplify and bring quality healthcare across Indonesia, from Sabang to Merauke. We connect 20,000+ doctors with patients in need through our Tele-consultation service. We partner with 3500+ pharmacies in 100+ cities to bring medicine to your doorstep. We've also partnered with Indonesia's largest lab provider to provide lab home services, and to top it off we have recently launched a premium appointment service that partners with 500+ hospitals that allow patients to book a doctor appointment inside our application. We are extremely fortunate to be trusted by our investors, such as the Bill & Melinda Gates Foundation, Singtel, UOB Ventures, Allianz, GoJek, Astra, Temasek, and many more. We recently closed our Series D round and in total have raised around USD$100+ million for our mission. Our team works tirelessly to make sure that we create the best healthcare solution personalised for all of our patient's needs, and are continuously on a path to simplify healthcare for Indonesia.