Script to convert .log files to JSON

[Porter76]

Recently setup a pipeline from AWS S3 to our SIEM (Microsoft Sentinel) using a lambda function to ingest our AWS WAF logs. There is a succesful connection between the two but I noticed that the logs weren't being pushed. The logs are stored in .log format and after converting the format to JSON they went through and I could see them in the SIEM. The problem is we had to do this manually and need to find a way to automate this.

Is there a powershell script that could automate the conversion of a .log file to JSON? The lambda function is a powershell script so this would be easy to add. 

[#2]

Hi @Porter76,

you can automate the process of converting .log files to JSON using PowerShell.

The following PowerShell script example can be used and then integrated into your Lambda function to automate this conversion:

# Define the path to your input .log file
$sourceLogFile = "C:\Path\To\Your\LogFile.log"

# Define the path to the output JSON file
$outputJsonFile = "C:\Path\To\Your\OutputFile.json"

# Initialize an array to store JSON objects
$jsonArray = @()

# Read the .log file line by line
Get-Content -Path $sourceLogFile | ForEach-Object {
# Split the log line based on your log format (assuming space-separated fields)
$logFields = $_.Split(" ")

# Create a custom PowerShell object with the desired JSON structure
$logObject = [PSCustomObject]@{
"Field1" = $logFields[0]
"Field2" = $logFields[1]
# Add more fields as needed
}

# Add the log object to the JSON array
$jsonArray += $logObject
}

# Convert the JSON array to a JSON string
$jsonString = $jsonArray | ConvertTo-Json

# Write the JSON string to the output JSON file
$jsonString | Set-Content -Path $outputJsonFile

# Display a success message
Write-Host "Conversion completed. JSON file saved to $outputJsonFile."

Please click Mark as Best Response & Like if my post helped you to solve your issue.
This will help others to find the correct solution easily. It also closes the item.

If the post was useful in other ways, please consider giving it Like.

Kindest regards,

Leon Pavesic