Analyzing Log files of very large size

[Vlad Markov]

On Sun, 11 Jul 2021 19:43:41 +0530
KK CHN <kkchn.in at gmail.com> wrote:

> Yes, it is.
> 
> On Sun, Jul 11, 2021 at 6:02 PM Korolev Sergey <serejk at febras.net> wrote:
> 
> > Is it a plain text file?
> >
> > On 11 Jul 2021, at 22:13, KK CHN <kkchn.in at gmail.com> wrote:
> >
> > List,
> >
> > I am in a requirement to analyze large log files of sonic wall firewall
> > around 50 GB. for a suspect attack.
> >
> > What tools and solutions need to be deployed for handling this much large
> > files and pls enlighten me with your expertise and reference materials if
> > any.
> >
> > All are tcp / ip communications, DNS UDP transports ..
> >
> > Regards,
> > Kris
I used to use split to break up large log files into manageable pieces. From there it depends on how you work. At first we used grep then we moved on to using perl regex to analyze logs.

Vlad



--