Convert JSON events to CEF format

If you found this article interesting, your support by following steps will help me spread the knowledge to others:

👏 Give the article 50 claps

💻 Follow me

Deploy your next app in seconds. Get $200 in cloud credits from @DigitalOcean using my link: https://m.do.co/t/a71522db03a2

Monitoring and analyzing data to acquire insights and spot potential security issues becomes more crucial as firms gather more data. To accomplish this, enterprises frequently process and analyze log data using log management systems like Elastic Stack (ELK) and Common Event Format (CEF). In this article, we will discuss how to convert JSON logs format received by Filebeats into Logstash to CEF format.

Overview of ELK and CEF

Elasticsearch, Logstash, and Kibana make up the well-known ELK stack log management solution. Log data is stored and indexed by the distributed search and analytics engine Elasticsearch. A data processing pipeline called Logstash receives data from many sources, processes it, and then transmits the result to a location like Elasticsearch. Data visualization software called Kibana offers a graphical user interface for examining and analyzing log data [1]. CEF (Common Event Format) standard log structure too provides a consistent format for security-related events. Security information and event management (SIEM) systems frequently process and examine log data using CEF.

The Configuration

The process of converting from JSON to CEF entails mapping the fields from the JSON data to the corresponding fields in the Common Event Format (CEF). CEF is a standardized log format that allows log management systems to effectively process and store logs from a variety of security and network devices. The CEF format comprises numerous key-value pairs that provide vital information about the log event. The structure of a CEF log message is fairly simple and is as follows:

CEF:Version|Device Vendor|Device Product|Device Version|Signature ID|Name|Severity|[Extension Key]=[Value] ..

Lets assume we have the following use case scenario: