At VGS we build our core proxy applications on Java. We picked Java at the beginning of the company because of its wide adoption, type safety, high performance, and large ecosystem of functionality. Unfortunately, we've had a couple of core apps which used Java 8 and were difficult to upgrade.
Running an earlier version of Java, such as Java 8, poses several challenges and risks in today's evolving technology landscape. One of the most significant concerns is the security aspect, as earlier Java versions may have known vulnerabilities that lack official patches and updates, making applications susceptible to potential attacks. Compatibility issues can also emerge, as libraries and third-party tools are developed with newer Java versions in mind, limiting the ability to leverage the latest features and ecosystem enhancements. Moreover, running Java 8 is leading to reduced performance and productivity for our team. It’s hindering the adoption of modern language features, and accumulating technical debt as we struggle to keep up to date with newer libraries.
This post documents some of the challenges we've faced as we've tried to upgrade our applications to Java 17 and we hope some of the lessons we've learnt will be useful to other engineers.
Our proxies are high performance, low-latency services built to handle rewriting and securing customer payloads in real-time. They typically handle dozens of requests for each instance every second and need to be able to react to each request with less than 50ms of additional latency for each request. With this in mind we knew that we needed to ensure that the upgrade did not impact performance.
First Try - Java 8 and updated GC to G1
Initially, we opted to switch the garbage collection mode to utilize the G1 Garbage Collector while still on Java 8 for the proxy. This approach appeared to be the more cautious and prudent first step.
We've performed a GC update to optimize resources after the team noticed slightly elevated latency and decided to decrease the load by increasing GC pauses, which resulted in unexpected errors, which resulted in elevated retries, which increased the load and caused multiple DB failovers and increased CPU consumption, which caused elevated error rates and processing latencies
Second Try - Update To Java 17 with native G1 GC
Update to Java 17 was successful, but higher memory utilization by G1 GC uncovered that there was a misconfiguration of the container memory usage limits for HTTP proxy on k8s. In certain high load conditions it resulted in containers requesting more memory than limits allowed, resulting in out of memory error and pod restarts. We have fixed the configuration for container memory usage to handle high load conditions.
Consistent Traffic Volumes Pre and Post Java 17 Migration
This graph illustrates the requests per minute to our proxy services over several days before and after the migration to Java 17. The near-identical traffic volumes on the days adjacent to the update underscore that any shifts in application metrics are directly related to the upgrade itself, rather than external traffic variations.
Reduction in Long Request Rates Post Java 17 Upgrade
This chart demonstrates the rate of long requests per minute, which we classify as those exceeding a 400ms processing time threshold. The visible decline in the frequency of long requests — by approximately 50% — following the update to Java 17 illustrates a significant enhancement in our application's performance efficiency and responsiveness.
Enhanced Non-Heap Memory Management After Java 17 Update
The graph depicted here showcases the non-heap memory consumption of our proxy applications before and after the migration to Java 17. Post-update, there is a better utilization in memory usage, alongside more frequent and efficient memory cleaning cycles. This improvement points to the optimized memory management capabilities inherent in the newer Java version, contributing to more robust and reliable application performance.
Java 8 Detailed One-Replica view
Java 17 Detailed One-Replica view
In this visual, we observe the Java heap memory usage patterns before and after the adoption of Java 17. Prior to the update, heap memory usage peaks at 2.5 GB; however, with the introduction of the G1 garbage collector—default in Java 17—memory usage now reaches approximately 5 GB before a garbage collection event occurs. This adjustment reflects the G1 collector's approach to managing memory for improved performance, particularly in systems with large memory capacity and demanding workloads.
Java 8 Detailed One-Replica view
Java 17 Detailed One-Replica view
Decreased Garbage Collection Frequency with Java 17
The data displayed here provides a cumulative count of garbage collection events per minute across all replicas, before and after upgrading to Java 17. There is a pronounced decrease in the frequency of garbage collections, with peak times showing a drop from approximately 800 to 100-150 events. This significant decline indicates enhanced efficiency in memory management at scale after the transition to Java17 and G1 GC.
Halved Cumulative Garbage Collection Time Post Java 17 Update
The stacked chart here quantifies the cumulative garbage collection time across all replicas, illustrating a dramatic reduction of about 50% following the update to Java 17 and G1 GC. This significant decrease underscores the efficiency gains in garbage collection processes, leading to better resource utilization and less downtime for garbage collection across our services.
VGS's transition from Java 8 to Java 17 has been a strategic move to modernize our core proxy applications, addressing several pressing concerns with earlier Java versions and setting a new benchmark for our system's performance.
Consistent Traffic Patterns: Our data analysis confirmed that the migration had no correlation with user traffic, with nearly identical request volumes before and after the Java update. This reinforced that the performance improvements were indeed a result of the Java upgrade.
Performance Enhancement: A significant performance boost was observed with a 50% reduction in the rate of "long requests" - those exceeding 400ms. This directly translates to a more responsive and efficient service for our customers.
Non-Heap Memory Optimization: Post-upgrade, we saw a more efficient pattern of non-heap memory consumption and cleaning, indicating that Java 17 better manages memory outside the heap space.
Heap Memory Utilization: The Java 17 upgrade, along with the adoption of the G1 garbage collector, led to an altered heap memory usage pattern. The usage now peaks at around 5 GB before garbage collection, compared to the 2.5 GB peak seen with the earlier version, demonstrating the G1 collector's ability to handle larger memory pools more effectively.
Garbage Collection Metrics:
Frequency: There was a marked decrease in garbage collection frequency, with the number of collections per minute plummeting from around 800 to between 100-150 during peak times.
Cumulative Time: The cumulative garbage collection time saw a significant reduction of about 50% across all pods.
Both improvements in garbage collection can be attributed to the efficiency of the G1 garbage collector, which is designed to optimize garbage collection pause times without sacrificing throughput.
The Java 17 migration, complemented by the G1 garbage collector, has not only fortified our security posture but also enhanced our system's performance. It has streamlined our garbage collection processes, reduced memory overhead, and allowed us to leverage the latest language features and library updates. These advancements have reinforced our proxy services' ability to manage high-load operations with improved latency, ensuring robust and reliable performance in real-time data handling.
Oleksandr Ahitoliev 
