Collector - Internal Log Receiver

It is recommeded not to allow outbound connection to every machines (DB Servers, Script servers.) in private subnetworks.

You can deploy our collector that will communicate with your machines private inside your data centre and will ingest logs to easySIEM receiver. So the communication will be open for a single machine only.

Host collector and push logs over HTTPs.

Collector TCP

Host the below code as a daemon (supervisor) on a small machine (COLLECTOR_INTERNAL_HOST,1CPU + 512MB RAM will work). Don't worry, we will help you setup this.

import json
import ssl
from tornado import httpclient
from tornado.ioloop import IOLoop
from tornado.iostream import StreamClosedError
from tornado.tcpserver import TCPServer
API_SERVER = "https://<SUBDOMAIN>-collector.easysiem.com/agent/syslog"
def json_decode_utf_safe(data):
"""
This will safely decode data in any encoding, the exception is globally catched for a purpose.
"""
output = '{}'
decode_error = False
try:
output = data.decode('utf-8')
except Exception as e:
print("got exception in decoding",str(e))
decode_error = True
if decode_error:
try:
output = data.decode("utf-8","replace")
except Exception as e:
print("error in replacing so ignoring finally",str(e))
output = data.decode("utf-8","ignore")
return json.loads(output)
async def fetch_url_post(url, data, ip):
"""
Post data to the easySIEM server over TLS.
"""
http_client = httpclient.AsyncHTTPClient()
data = json.loads(data.decode("utf-8"))
data["fromhost-ip"]=ip
try:
await http_client.fetch(url, method="POST", body=json.dumps(data),validate_cert=False)
except Exception as e:
print("Error: %s" % e)
else:
print("request success")
class TCPLogServer(TCPServer):
async def handle_stream(self, stream, address):
while True:
try:
data = await stream.read_until(b"\n")
await fetch_url_post(API_SERVER, data, address[0])
except StreamClosedError:
print("error something like that")
break
if __name__ == "__main__":
#ssl_ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
#ssl_ctx.load_cert_chain("certificate.crt","private.key")
print("server started")
server = TCPLogServer()
server.bind(8888)
server.start(0) #Forks multiple sub-processes
IOLoop.current().start()

Rsyslog without TLS to collector

Comment out the below line

#$DefaultNetstreamDriverCAFile /etc/ssl/certs/ca-certificates.crt

Change the action block as following

Set target to COLLECTOR_INTERNAL_HOST

action(type="omfwd" protocol="tcp" target="COLLECTOR_INTERNAL_HOST" port="8888"
template="json-template")

Collector HTTP

Talk to us and we will help to configure collector to receive internal logs via Http /TCP and then forward it to easySIEM via TLS.