The advantages of using an up-to-date software stack may appear self-evident: enhanced functionality, a more productive development environment, the convenience of using the latest documentation, and stronger security protocols with the latest patches covering recently revealed breaches and vulnerabilities. But at Gcore, we wanted to dig deeper. A couple of years ago, we spent 2.5 months rigorously updating our software stack, and today we’re harvesting the results and want to share our experience with you. We’ll guide you through our Public API’s technological infrastructure to uncover both obvious and subtle benefits, and show you how to apply our strategy if you want to get those benefits for your own team.
Spotlight on the Gcore Public API Team
Our Public API team carefully and diligently updates the software stack and monitors new opportunities in search of better productivity. In spotlighting their efforts, we want to demonstrate the concrete benefits that an up-to-date software stack offers to developers.
Understanding Public API
The Public API serves as a critical management component of Gcore’s content delivery network (CDN). It is a frontend service that accepts commands from users via GUI or CLI and applies them to the CDN backend. Speed, reliability, flexibility, and security are its cornerstones.
The Public API serves as a gateway for user requests, meticulously processing and translating them into actionable commands for the product backend. Because it acts as a bridge to the external environment, security is the top priority for this software component.
Like other elements in the CDN software landscape, the Public API must also exhibit robust stability, scalability, and high performance. However, what sets it apart from our backend services is its responsiveness to ever-changing customer needs. As a result, the Public API is designed to be not only resilient, but also highly adaptable from a development standpoint.
Our Software Stack and Versions
To provide a more concrete overview of our approach to software updates and versions, let’s look at the specific programming languages, tools, and services used in Gcore’s Public API.
Tech Software Stack
Here are the four most important:
- Python (current version: 3.11) is our main programming language.
- Django (current version: 4.2) is our basic framework.
- Django REST framework or drf (current version: 3.14) is our Django extension for faster API coding.
- Celery (current version: 5.3) is a framework for running asynchronous tasks and on-schedule tasks.
Dev Tooling
To amplify team productivity, we use various service software and tools to improve our code clarity and simplify test procedures. Here are some examples:
- Pytest (current version: Pytest 7.4) is the main framework for writing modules and functional tests.
- Coverage (current version: pytest-cov 4.1.0,) when combined with Pytest, allows us to track the percentage of code lines that are executed during tests.
- Isort is a convenient tool for import formatting control.
- Black is a strict code formatter that we use to assess how our code correlates with the established style.
- Flake8 is a static code analyzer that monitors the code in order to detect syntax errors and weaknesses.
- Pyupgrade helps us to automatically track and upgrade the syntax we used for newer versions of the language.
We update our stack regularly so they always have the latest possible versions. Now let’s turn to how we stay on top of these updates.
Our Software Update Strategy
The Public API team had been suffering from an outdated software stack for a long time. While we recognized the advantages of transitioning to the latest software versions, limited resources kept us stuck with outdated software. It’s a classic conundrum: You’re well aware that an up-to-date environment enhances performance, but there are always urgent tasks with higher priority.
The breakthrough came from the team’s synergy and enthusiasm. The update transformation took about 2.5 months and required not only a shift in our technological landscape, but also new mindsets and daily workflows. Let’s look at the journey we undertook to achieve the new strategy:
Phase 1. Admit the Problem
At this stage, we admitted to ourselves that working with an outdated software stack causes a lot of extra work and potential risks. We also understood that the cumulative effect of this state can easily ruin our ability to respond promptly and efficiently to other “high-priority” tasks and urgent fixes.
This point seems pretty obvious, but it was actually the hardest one.
Phase 2. Update the Cycle
After we admitted the problem, the team designated an experienced developer who became fully dedicated to this project. He was charged with running all changes and orchestrating all dependencies with the infrastructure environment.
The main focus of our updates was our tech software stack. It required careful and attentive manipulation, so we updated each software by one version, checked everything was working properly, fixed any issues that arose, and only then repeated the process by updating the whole stack to the next version. Here are the steps:
- Upgrade Python
- Upgrade Django REST framework (drf)
- Upgrade Celery
- Upgrade Django
- Fix backward compatibility and deprecation issues
- Repeat from the beginning
This iterative approach allows us to move forward without risky changes and any possible disruptions. Sometimes, we got blocked by other teams’ critical work. For instance, we experienced delays when the CDN Ops team updated PostgreSQL on the CDN backend.
Once the main tech stack was fully updated, we moved to dev tooling. Since this part of our software portfolio is less crucial, we decided to make a bulk update to the latest versions without following the gradual approach.
Phase 3. Ongoing Monitoring
The last step was to maintain our successful implementation by following this update approach in our everyday workflow. For this purpose, we set up Dependabot, a GitHub bot responsible for automated monitoring and for suggesting updates for our software.
Once Dependabot finds a new version of something in our stack, we get a notification, and the CI/CD pipeline is launched. If any errors or compatibility issues with existing dependencies are found, the update process is halted and the new version is not integrated. But if everything looks good, we just have to approve the changes and merge them into the master.
Benefits of Working with the Up-to-Date Software Stack
We want to share the advantages our Public API team noticed after several months of work with up-to-date software stack. We grouped these benefits into two categories: obvious and unexpected.
Obvious Advantages
Better Security
Using up-to-date software versions helps to mitigate most known vulnerabilities, enhancing our overall security posture and providing a basic inherent level of code security. Regular updates to libraries and dependencies act as proactive measures against newly discovered vulnerabilities, reducing the potential impact of emerging cyber threats.
Increased Performance
Since Python is the main programming language for our Public API service, the version we use plays a key role in shaping the service’s overall performance. Newer iterations of the Python interpreter are generally faster than their predecessors due to advanced optimizations. For example, synthetic tests reveal that Python 3.11 offers an average speed increase of 20–25% compared to Python 3.10.
While it’s tempting to equate the improved efficiency of the Python interpreter with a corresponding boost in our service’s performance, the reality is more nuanced. Our web application involves multiple integrations and heavy database interaction, complicating any straightforward assessment of how such code-level enhancements affect overall performance. However, we have observed a noticeable acceleration in service operations, particularly in instances of pure code execution, even if we can’t quantify the exact impact.
Heightened Efficiency and Productivity
Python continues to evolve, offering new features and functionalities that make developers’ lives easier, improving code logic and its readability. The same is true for its libraries, which become more effective with every new release. The result for us? Additional flexibility plus faster and more convenient code writing.
Using static code analyzers like Flake8 allows us to fix errors and optimize code consistency on the fly, eliminating the need for bug fixes. This efficiency improvement allows our team to focus on business logic instead of getting bogged down by issues like code formatting or searching for features in outdated libraries.
Easy Access to New Functionality
Being tethered to outdated software can significantly hinder innovation, especially for fast-paced, forward-thinking companies like ours. Staying current ensures that we can seamlessly integrate the newest versions of software like PostgreSQL into our infrastructure without fretting over compatibility issues. Should any questions about compatibility arise, we can quickly and easily consult developer documentation, avoiding the need for makeshift solutions.
Unexpected Advantages
Team Motivation
Team motivation can’t be understated: A conducive work environment is a genuine pleasure that improves the performance and personal experience of a company, and that experience is significantly amplified when you’re armed with state-of-the-art tools. Our team finds it invigorating to be on the cutting edge of technology, focusing on pioneering solutions rather than patching up outdated systems. It’s frustrating to aspire to industry-leading work while tethered to outdated technology, and we’re committed to keeping our developers at the front of the pack in the technological race.
Team Expertise
Another key advantage is maintaining our team’s expertise at peak levels. Engaging with the latest software not only keeps us in the loop but also enables us to apply this fresh knowledge directly to our projects, leading to superior results.
Culture of Innovation
When the status quo involves battling the repercussions of outmoded software and broken dependencies, it can create a culture averse to experimentation. Conversely, when keeping up to date is the norm, it fosters an openness to new possibilities. This mindset promotes an innovative culture, encouraging everyone to explore, experiment, and incorporate the latest features from new software releases.
Attracting Top Talent
Last but not least: When interviewing new candidates for our teams, we often get responses like, “Wow, that’s cool!” when presenting our software stack. Potential developers understand that with our team, they will get an opportunity to be at the forefront of the industry and grow professionally. This cutting-edge environment not only attracts top talent but also retains it, solidifying our standing as a desirable employer in a competitive landscape.
Conclusion
Working in IT is always about being innovative and up-to-date. Using an antiquated software stack is a sure way to lose your competitive edge. Outdated technologies invariably lead to system inefficiencies, making your entire product both less productive and more susceptible to vulnerabilities.
Public API is a pivotal component of Gcore CDN that requires a highly performant and modern technology stack. But beyond the product’s requirements, having an outstanding software environment is important for the developers’ team. It provides better productivity and more motivation, expands the team’s expertise, and fosters innovative behavior among developers. If this sounds like a team you’d like to be a part of, check out our open positions.
Fun fact: During writing the article we updated Celery from 5.3.0 to 5.3.4.
