pgBadger - a fast PostgreSQL log analyzer

pgBadger is a PostgreSQL log analyzer build for speed with fully detailed reports from your PostgreSQL log file. It's a single and small Perl script that aims to replace and outperform the old php script pgFouine.

By the way, we would like to thank Guillaume Smet for all the work he has done on this really nice tool. We've been using it a long time, it was a really great tool!

pgBadger is written in pure Perl language. It uses a javascript library to draw graphs so that you don't need additional Perl modules or any other package to install. Furthermore, this library gives us more features such as zooming.

pgBadger is able to autodetect your log file format (syslog, stderr or csvlog). It is designed to parse huge log files as well as gzip compressed file. See a complete list of features below.

Features

pgBadger reports everything about your SQL queries:

  • Overall statistics.
  • The most frequent waiting queries.
  • Queries that waited the most.
  • Queries generating the most temporary files.
  • Queries generating the largest temporary files.
  • The slowest queries.
  • Queries that took up the most time.
  • The most frequent queries.
  • The most frequent errors.

The following reports are also available with hourly charts:

  • Hourly queries statistics.
  • Hourly temporary file statistics.
  • Hourly checkpoints statistics.
  • Hourly restartpoints statistics.
  • Locks statistics.
  • Queries by type (select/insert/update/delete).
  • Sessions per database/user/client.
  • Connections per database/user/client.
  • Reports about autovacuum and autoanalyze

All charts are zoomable and can be saved as PNG images.

pgBadger supports any custom format set into log_line_prefix of your postgresql.conf file provide that you use the %t, %p and %l patterns.

pgBadger allow parallel processing on a single log file and multiple files through the use of the -j option and the number of CPUs as value.

If you want to save system performance you can also use log_duration instead of log_min_duration_statement to have reports on duration and number of queries only.

You can also use pgBadger to only report errors.

Authors

pgBadger is an original work from Gilles Darold. It is maintained by the good folks at Dalibo and every one who wants to contribute.

<pgbadger AT dalibo DOT com>

Quotes

Andrew Dunstan was the first to talk about pgBadger after seeing the talk at PGCON 2012: "The biggest thing that piqued my interest yesterday at pgCon was the lightning talk on pgbadger. I'm certainly going to take a look at this as a substitute for pgfouine". See the whole blog post on Andrew's blog.

Christophe Pettus, another PostgreSQL Experts consultant, says June, 30th, 2012: "It’s written in Perl, at least as fast as pgfouine, and can process log files that pgfouine can’t handle. It can read either CSV or standard log format, and can directly read *.gz files. It also produces a wider range of reports that pgfouine, including some very useful locking reports. I threw 25GB of logs with near 80 million lines at it without it complaining". Read it all on his blog post.

Fidel Ramos' tweet says it all: "pgbadger, an alternative PostgreSQL log analyzer to pgfouine which promises to be both faster and more extensive".

Vincent Picavet from Oslandia, says "When we setup a Geographical information system database with PostGIS, we install tools for monitoring its behaviour. This enable us to fine-tune the server and queries. We use PgBadger on various production servers, and it's a real quick and efficient solution to get useful reports. The really dynamic development of this solution has been a big plus in our choice."

Michael Paquier from Postgres-XC project said in his article: "It is one of those utilities that you can use not only for production database systems, but for extra things like benchmark or performance analysis. Its installation is easy, will not heavy your system with packages you might not want, so go ahead and use it ... this utility is 100% compatible with Postgres-XC, the only thing you need to do is just to launch pgbadger for each node of your cluster."

License

pgBadger is free software distributed under the simplified BSD License.