Analyzing Log files of very large size

[Mehmet Erol Sanliturk]

On Sun, Jul 11, 2021 at 3:14 PM KK CHN <kkchn.in at gmail.com> wrote:

> List,
>
> I am in a requirement to analyze large log files of sonic wall firewall
> around 50 GB. for a suspect attack.
>
> What tools and solutions need to be deployed for handling this much large
> files and pls enlighten me with your expertise and reference materials if
> any.
>
> All are tcp / ip communications, DNS UDP transports ..
>
> Regards,
> Kris
> _______________________________________________
>
>


How are you generating your log files ?


Is the logger program open source  ?


Can you modify it if it is open source ?
If yes , are you ( meaning your team ) able to program modifications ?


Is it compulsory to  store and then process the log files or
Is it possible or  useful to process your communication traffic online if
the logger program is open source ?


If please answer ( if it is suitable for you  )  the above questions , I
want to make suggestions as a continuation of
my message mentioning AVL trees utilization .


Thank you very much .


Mehmet Erol Sanliturk