Apache Log Parser and Data Normalization Application

Python handles File Processing & MySQL or MariaDB handles Data Processing

ApacheLogs2MySQL consists of two Python Modules & one Database Schema apache_logs to automate importing Access & Error files and normalizing data into database designed for reports & data analysis.

Quickly convert a folder of Apache log files into a normalized database. Process unlimited files, from multiple domains, from multiple servers with complete Data Lineage. Small codebase & simple setup.

Runs on Windows, Linux and MacOS. Database runs on MySQL and MariaDB.

4 LogFormats & 2 ErrorLogFormats can be loaded and 6 Stored Procedures can be processed in a single Python `ProcessLogs function` execution.

IP Geolocation data integration using MaxMind GeoIP2 Python API provides IP country, subdivision, city, system organization, network and coordinates information stored and normalized into 6 Database Schema tables.

User-Agent data integration using User-Agents provides browser, device and operating system information stored and normalized into 11 Database Schema tables.

Database Schema designed to accommodate unlimited Servers & Domains. Step-by-step guide for easy installation.

Screenshots of Process Messages and Benchmarks for MariaDB version 11.8 and MySQL version 9.2.

Apache Log Management and Visualization Application

Express handles Data Querying & ECharts handles Data Visualization of MySQL or MariaDB Schema

MySQL2ApacheECharts consists of Express web application frameworks with Drill Down Capability & Apache ECharts frameworks for Log Data Visualization in charts, reports & data analysis interfaces of Database Schema apache_logs created by ApacheLogs2MySQL

Application is currently in early stages of development with initial release scheduled for March.