Syslog-ng for DevOps: Customized Logging with Python by Peter Czanik
The syslog-ng application collects logs from many different sources, performs real-time log analysis by processing and filtering them, and finally it stores the logs, or forwards them for further analysis. It was originally developed in C due to its low footprint and high performance. Recently Python language bindings were added, so you can collect, process, and store log messages using embedded Python code.
DevOps engineers often use syslog-ng due to its low footprint and high performance. The new Python bindings make it easy to extend syslog-ng without touching the core of the application or requiring a complex development development environment.
Python bindings always have two parts: a configuration part – where you configure a source, parser, or destination – and the actual Python code part. You can store the code in-line in the syslog-ng configuration file, or in external files, if you have a larger project.
Obviously the Python code is slower than native C code. On the other hand, extending syslog-ng in Python offers a lot more flexibility than development in C. It is easy to get started, as there is no need for extra setup. You can get proof-of-concept minimal setup working with a single method quickly. You can extend it later to have proper initialization and connection control for added robustness.
Learn how to customize your log processing infrastructure in Python using syslog-ng, integrate your logs to local and cloud services, and the implementation details and benefits of such solutions.