Centralized logging: A new cloud service

We’ve launched a new Gcore Cloud service—centralized logging. It is currently in beta testing, so you can use it for free.

Here, we’ll explain in detail what this service is about, what its peculiarities are, how it works, and how to use it.

What is centralized logging?

Centralized logging is a service for collecting, storing, and analyzing logs. It allows collecting logs from many different services into a single repository.

Thanks to this service, you can easily analyze data from various sources, making it possible to monitor the operation of different apps from a single, centralized system.

What is centralized logging used for?

• Monitoring and debugging various systems and apps in a centralized manner
• Tracking system performance
• Setting up automatic alerts (notifications) regarding failures
• Performing quick troubleshooting
• Managing and tracking events related to information security

As a rule, centralized logging consists of a few components:

• Logshipper—an element that collects and sends logs to the repository
• Repository—the place where logs are collected and stored
• Log analyzer—a component that helps work with logs: search for the required ones and process information
• Visualizer—a special program that helps transform data into easy-to-understand charts and diagrams

You can organize and configure the service yourself, for example, by using Docker and various open-source tools. But to do this, you’ll need to install, configure, and integrate several programs, build the logic, specify the code, and then administer this system. All this will require a lot of time, effort, and resources from the IT team.

Another option is to use LaaS (Logging as a Service). It is a ready-made cloud solution, a fully managed service that is easy to configure and use.

Centralized logging from Gcore is exactly this kind of service. We provide a simple and convenient solution for collecting, analyzing, and storing logs. All you need is to configure the logshipper on your services. And we’ll take care of the rest.

Peculiarities of centralized logging from Gcore

The service allows collecting logs from 150+ distributed app components as well as local and hybrid cloud systems.

Logs are collected in isolated groups (topics).

The service can be configured in just a couple of clicks. And you can manage it via a convenient panel or API with detailed documentation.

Advantages of the service:

• User-friendly
• Secure storage
• Log analysis with the familiar Elasticsearch interface
• 99.9% SLA

Currently, this service is only available in our location in Luxembourg, but it will soon become accessible from any place where our Cloud is functioning.

How centralized logging works in Gcore Cloud

The service is based on two open-source components:

• Apache Kafka—a distributed message broker used as a link between service components and log sources.
• OpenSearch—a scalable tool powered by Elasticsearch and Kibana. It’s a comprehensive search and analytical resources system that lets you search, collect, protect, view, and analyze data in real-time.

Using open-source solutions will make the service cheaper for our clients in the future, as its price won’t include license fees.

To send logs from your services, you need to install any logshipper on your resources. Centralized logging can be easily integrated with Logstash, Fluentd, Rsyslog, FluentBit, Filebeat, and other popular tools.

API with detailed documentation is available for the integration. And our specialists can advise you on configuration aspects.

To view logs, you can use the OpenSearch dashboard. This tool allows to display data in clear tables, charts, and diagrams.

Like the logshipper, the dashboard is installed on the client-side and integrated via API. But you can also consult our specialists for its installation and configuration.

How to enable centralized logging

The service is currently in beta testing, so you can use it for free.

If you want to try centralized logging, please contact our specialists. They will help you enable the service it and answer any questions you may have.

Detailed instructions on how to work with the service can be found in the knowledge base.

More about logging