Smart Logging With Log4perl
Mike Schilli, Yahoo!
YAPC, 06/27/2007
Grab A T-Shirt!
Smart Logging With Log4perl
Mike Schilli, Yahoo!
YAPC, 06/27/2007
Logging – why?
• Go back in time and figure out what
happened.
• Measure performance
• Trace activity on live systems
• Debug during development
Log4perl and Log4j
• Log::Log4perl ports Log4j to Perl
• Log4j: de facto Java logging standard, by
Ceki Gülcü
– Latest development: ‘logback’
• http://logging.apache.org/log4j
• Log::Log4perl adds perlisms demanded by
users
Log4perl History
•
•
•
•
0.01 release 2002
Current release 1.11 (06/2007)
Authors: Mike Schilli, Kevin Goess
Used by major banks, target.com,
fedex.com, Yahoo!, Google, …
• Several CPAN modules support it (e.g.
Catalyst, Net::Amazon, …)
Log4perl Release History
Design Goals
• Easy to use in small scripts
• Scales easily with growing architecture
• Log4perl-enabled CPAN modules can be
used with and without Log4perl
initialization
• Optimized for Speed
• Open Architecture, extensible
Why Log4perl and not one of the
20 Logging modules on CPAN?
• Write once, and L4p-enabled scripts/modules
can be used
– with any logging configuration you desire
– by changing loggers, appenders and layouts
independently
• Target specific areas of your application only
• No need to change your source code to change
the logging behaviour
• Log4perl is aimed to be the last logging module
you’ll ever install
Log::Log4perl for Simple Scripts
#!/usr/bin/perl –w
use strict;
use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init($INFO);
DEBUG "Won't make it";
INFO "Will go through";
ERROR "Will also go through";
$ easy.pl
2004/10/19 10:56:56 Will go through
2004/10/19 10:56:56 Will also go through
Log::Log4perl for CPAN Modules
use strict;
use YourCPANModule;
use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init( $INFO );
YourCPANModule::somefunc();
package YourCPANModule;
use strict;
use Log::Log4perl qw(:easy);
sub somefunc {
DEBUG "Won't make it";
INFO "Will go through";
}
Log::Log4perl for CPAN Modules
use strict;
use YourCPANModule;
YourCPANModule::somefunc();
package YourCPANModule;
use strict;
use Log::Log4perl qw(:easy);
sub somefunc {
DEBUG "Won't make it";
INFO "Won’t make it";
}
Log::Log4perl Building Blocks
Loggers
Layouts
Appenders
Log::Log4perl Building Blocks
Loggers
Log it or suppress it
Layouts
Format it
Appenders
Write it out
Log::Log4perl Building Blocks
Loggers
DEBUG “Starting up”);
or
my $log = get_logger();
$log->debug(“Starting up”);
Layouts
Appenders
“Starting up” =>
2007-06-21 07:30:33 Foo.pm-123 Starting up
Log File
Database
…
Easy vs. Standard Mode
use Log::Log4perl qw(:easy);
sub somefunc {
DEBUG “Debug Message”;
INFO “Info Message”;
}
use Log::Log4perl qw(get_logger);
sub somefunc {
my $logger = get_logger();
$logger->debug(“Debug message”);
$logger->info(“Info message”);
}
Log4perl Operation
• Initialize once, at startup:
– Define which loggers should fire base on
• message priority (DEBUG, INFO, …)
• location (Foo::Bar, …)
– Define where messages should end up at
• Loggers planted in program code get
activated
Easy Init vs. Standard Init
use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init($INFO);
use Log::Log4perl;
Log::Log4perl->init(“l4p.conf”);
#l4p.conf
log4perl.category = INFO, Screen
log4perl.appender.Screen = \
Log::Log4perl::Appender::Screen
log4perl.appender.Screen.stderr = 1
log4perl.appender.Logfile.layout = \
Log::Log4perl::Layout::SimpleLayout
Mix&Match:
Standard Init with Easy Functions
use Log::Log4perl qw(:easy);
Log::Log4perl->init(“l4p.conf”);
DEBUG “Debug Message”;
INFO “Info Message”;
#l4p.conf
log4perl.category = WARN, Logfile
log4perl.appender.Logfile = \
Log::Log4perl::Appender::File
log4perl.appender.Logfile.filename = test.log
log4perl.appender.Logfile.layout = \
Log::Log4perl::Layout::SimpleLayout
Mix and Match
Easy Mode
DEBUG(…)
Standard Mode
get_logger()
easy_init()
OK
OK
init()
OK
OK
Categories or:
What the hell does this logo mean?
Categories: Areas of your App
package Bar::Twix;
use Log::Log4perl qw(get_logger);
sub eat {
# Gets a Bar::Twix logger
my $logger = get_logger();
$logger->debug(“Debug Message”);
}
#l4p.conf
log4perl.category.Bar.Twix = WARN, Logfile
log4perl.appender.Logfile = \
Log::Log4perl::Appender::File
log4perl.appender.Logfile.filename = test.log
log4perl.appender.Logfile.layout = \
Log::Log4perl::Layout::SimpleLayout
Categories (Easy Mode)
package Bar::Twix;
use Log::Log4perl qw(:easy);
sub eat {
# Has a Bar::Twix logger
DEBUG “Debug Message”;
}
#l4p.conf
log4perl.category.Bar.Twix = WARN, Logfile
log4perl.appender.Logfile = \
Log::Log4perl::Appender::File
log4perl.appender.Logfile.filename = test.log
log4perl.appender.Logfile.layout = \
Log::Log4perl::Layout::SimpleLayout
Loggers
• Have a
– category (e.g. “Net::Amazon” )
– priority (e.g. $DEBUG )
• Take a message (e.g. “Starting” )
Loggers
• Pass on their message if
– logging has been configured for
• their category ( “Net::Amazon”) or
• a parent category ( “Net”, “” )
– and (!) the message priority (e.g. $DEBUG) is
greater or equal than the configured log level
(e.g. $DEBUG) for the category
Limit Throughput: Levels and
Categories
• Categories determine which parts of the
system are addressed.
my $log = get_logger(“Net::Amazon”);
or simply
package Net::Amazon;
DEBUG “…”;
• Levels indicate the message priority.
$log->debug(“Request sent ($bytes bytes)”;
Log Levels
• Levels
–
–
–
–
–
–
–
$TRACE
$DEBUG
$INFO
$WARN
$ERROR
$FATAL
$OFF
Log Levels
• Methods
–
–
–
–
–
–
$log->trace(“$bytes bytes transferred”);
$log->debug(“HTTP get OK”);
$log->info(“Starting up”);
$log->warn(“HTTP get failed, retrying”);
$log->error(“Out of retries!”);
$log->fatal(“Panic! Shutting down.”);
Log Levels
• Macros
– TRACE(“$bytes bytes transferred”);
– DEBUG(“HTTP get OK”);
– INFO(“Starting up”);
– WARN(“HTTP get failed, retrying”);
– ERROR(“Out of retries!”);
– FATAL(“Panic! Shutting down.”);
Remote Control via Categories
l4p.conf
log4perl.category
= DEBUG, Screen
#!/usr/bin/perl
use Net::Amazon;
Log::Log4perl->init(“l4p.conf”);
DEBUG “Makes it through!”
my $amzn = Net::Amazon->new();
package Net::Amazon;
ERROR “Makes it through!”;
DEBUG “Makes it through”;
Remote Control via Categories
l4p.conf
Log4perl.category
= ERROR, Screen
#!/usr/bin/perl
use Net::Amazon;
Log::Log4perl->init(“l4p.conf”);
DEBUG “Gets blocked!”
my $amzn = Net::Amazon->new();
package Net::Amazon;
ERROR “Makes it through!”;
DEBUG “Gets blocked”;
Remote Control via Categories
l4p.conf
Log4perl.category
= FATAL, Screen
#!/usr/bin/perl
use Net::Amazon;
Log::Log4perl->init(“l4p.conf”);
DEBUG “Gets blocked!”
my $amzn = Net::Amazon->new();
package Net::Amazon;
ERROR “Gets blocked!”;
DEBUG “Gets blocked”;
Remote Control via Categories
l4p.conf
Log4perl.category.main
= DEBUG, Screen
Log4perl.category.Net.Amazon = ERROR, Screen
#!/usr/bin/perl
use Net::Amazon;
Log::Log4perl->init(“l4p.conf”);
DEBUG “Makes it through!”
my $amzn = Net::Amazon->new();
package Net::Amazon;
ERROR “Makes it through!”;
DEBUG “Gets blocked”;
Remote Control via Categories
l4p.conf
Log4perl.category.main
= ERROR, Screen
Log4perl.category.Net.Amazon = DEBUG, Screen
#!/usr/bin/perl
use Net::Amazon;
Log::Log4perl->init(“l4p.conf”);
DEBUG “Gets blocked”;
my $amzn = Net::Amazon->new();
package Net::Amazon;
ERROR “Makes it through!”;
DEBUG “Makes it through!”;
Category Inheritance (1)
l4p.conf
Log4perl.category.main
Log4perl.category.Net
= DEBUG, Screen
= ERROR, Screen
#!/usr/bin/perl
use Net::Amazon;
Log::Log4perl->init(“l4p.conf”);
DEBUG “Makes it through!”
my $amzn = Net::Amazon->new();
package Net::Amazon;
ERROR “Makes it through!”;
DEBUG “Gets blocked”;
Category Inheritance (2)
l4p.conf
Log4perl.category.main = DEBUG, Screen
Log4perl.category
= ERROR, Screen
#!/usr/bin/perl
use Net::Amazon;
Log::Log4perl->init(“l4p.conf”);
DEBUG “Makes it through!”
my $amzn = Net::Amazon->new();
package Net::Amazon;
ERROR “Makes it through!”;
DEBUG “Gets blocked”;
Log4perl Flow
Application sends a log message
(Category, Priority)
Log4perl Configuration
decides go/no go, based
on Category and Priority
?
Layout
Appender
Appender
Appender
l4p.conf: Screen Appender
log4perl.category.main
= DEBUG, Screen
Log4perl.category.Net.Amazon = INFO, Screen
log4perl.appender.Screen = \
Log::Log4perl::Appender::Screen
…
l4p.conf: File Appender
log4perl.category.main
= DEBUG, Logfile
Log4perl.category.Net.Amazon = INFO, Logfile
log4perl.appender.Logfile = \
Log::Log4perl::Appender::File
log4perl.appender.Logfile.filename = test.log
…
l4p.conf: Two appenders
log4perl.category.main
= DEBUG, Logfile
Log4perl.category.Net.Amazon = INFO, Screen
log4perl.appender.Logfile = \
Log::Log4perl::Appender::File
log4perl.appender.Logfile.filename = test.log
log4perl.appender.Screen = \
Log::Log4perl::Appender::Screen
…
Init and Watch
• Log::Log4perl->init(“l4p.conf”);
• Log::Log4perl->init(\$conf_string);
• Log::Log4perl->init_and_watch(“l4p.conf”, 30);
• Log::Log4perl->init_and_watch(“l4p.conf”, ‘HUP’ );
Demo
Appenders
• Appenders are output sinks
• Get called if a message passes category
and level hurdles
Appenders
• Log::Log4perl::Appender::Screen
Log::Log4perl::Appender::File
Log::Log4perl::Appender::Socket
Log::Log4perl::Appender::DBI
Log::Log4perl::Appender::Synchronized
Log::Log4perl::Appender::RRDs
• Log::Dispatch provides even more:
Log::Dispatch Appenders
• Log::Dispatch::ApacheLog
Log::Dispatch::DBI, Log::Dispatch::Email,
Log::Dispatch::Email::MIMELite
Log::Dispatch::File
Log::Dispatch::FileRotate
Log::Dispatch::Screen
Log::Dispatch::Syslog, Log::Dispatch::Tk
Example: Rotating Log File
Appender
• Keep 5 days of logfiles, then delete:
log4perl.category = WARN, Logfile
log4perl.appender.Logfile = Log::Dispatch::FileRotate
log4perl.appender.Logfile.filename
= test.log
log4perl.appender.Logfile.max
= 5
log4perl.appender.Logfile.DatePattern = yyyy-MM-dd
log4perl.appender.Logfile.TZ
= PST
log4perl.appender.Logfile.layout
= \
Log::Log4perl::Layout::SimpleLayout
Rotating Files
• Rotating File Appender:
– Log::Dispatch::FileRotate
– Single request pays for rotation
– Rotation runs as particular user
• External Rotators (e.g. newsyslog):
– recreate flag makes sure file appender adjusts
– recreate_check_interval saves on stat() calls
Write Your Own Appender (1)
package ColorScreenAppender;
our @ISA = qw(Log::Log4perl::Appender);
use Term::ANSIColor;
sub new {
my($class, %options) = @_;
my $self = {%options, …};
bless $self, $class;
}
sub log {
my($self, %params) = @_;
print colored($params{message},
$self->{color});
}
Write Your Own Appender (2)
log4perl.logger = INFO, ColorApp
log4perl.appender.ColorApp=ColorScreenAppender
log4perl.appender.ColorApp.color = red
log4perl.appender.ColorApp.layout = SimpleLayout
Layouts
• SimpleLayout
log4perl.appender.Screen = \
Log::Log4perl::Appender::Screen
log4perl.appender.Screen.layout = SimpleLayout
$log->debug(“Sending Mail”);
DEBUG – Sending Mail
Layouts
• PatternLayout
log4perl.appender.Logfile.layout = \
Log::Log4perl::Layout::PatternLayout
log4perl.appender.Logfile.layout.ConversionPattern = \
%d %F{1}:%L> %m %n
$log->debug(“Sending Mail”);
2004/10/17 18:47:25 l4ptest.pl:25> Sending Mail
Layouts
• PatternLayout
%c Category of the logging event.
%C Fully qualified package (or class) name of the caller
%d Current date in yyyy/MM/dd hh:mm:ss format
%F File where the logging event occurred
%H Hostname
%l calling method + file + line
%L Line number within the file where the log statement was issued
%m The message to be logged
%M Method or function where the logging request was issued
%n Newline (OS-independent)
%p Priority of the logging event
%P pid of the current process
%r Number of milliseconds elapsed from program start
%x The elements of the NDC stack
%X{key} The entry 'key' of the MDC
%% A literal percent (%) sign
Layouts
• Still not enough? Write your own:
log4perl.PatternLayout.cspec.U = sub { return "UID $<" }
…
log4perl.appender.Logfile.layout = \
Log::Log4perl::Layout::PatternLayout
log4perl.appender.Logfile.layout.ConversionPattern = \
%d %U> %m %n
Filters
• Last minute chance for Appenders to block
• More expensive than Level/Category
blocking
• Text/Level Match comes with L4p
• Write your own custom filters
Filters
log4perl.category = INFO, Screen
log4perl.filter.MyFilter = \
Log::Log4perl::Filter::StringMatch
log4perl.filter.M1.StringToMatch = let this through
log4perl.appender.Screen = \
Log::Log4perl::Appender::Screen
log4perl.appender.Screen.Filter = MyFilter
log4perl.appender.Screen.layout = \
Log::Log4perl::Layout::SimpleLayout
Filters
log4perl.filter.M1 = Log::Log4perl::Filter::StringMatch
log4perl.filter.M1.StringToMatch = let this through
log4perl.filter.M1.AcceptOnMatch = true
log4perl.filter.M1 = Log::Log4perl::Filter::LevelMatch
log4perl.filter.M1.LevelToMatch = INFO
log4perl.filter.M1.AcceptOnMatch = true
Speed
Avoid Wasting Cycles
for (@super_long_array) {
$logger->debug("Element: $_\n");
}
if($logger->is_debug()) {
for (@super_long_array) {
$logger->debug("Element: $_\n");
}
}
Performance
• On a Pentium 4 Linux box at 2.4 GHz,
you'll get through
– 500,000 suppressed log statements per
second
– 30,000 logged messages per second (using
an in-memory appender)
– init_and_watch delay mode: 300,000
suppressed, 30,000 logged.
– init_and_watch signal mode: 450,000
suppressed, 30,000 logged.
Advanced Trickery
Infiltrating LWP
• Ever wondered what LWP is doing behind
the scenes?
use LWP::UserAgent;
use HTTP::Request::Common;
use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init($DEBUG);
Log::Log4perl->infiltrate_lwp();
my $ua = LWP::UserAgent->new();
my $resp = $ua->request(GET
"http://www.yahoo.com");
Infiltrating LWP
• Getting LWP to use Log4perl for logging:
2004/10/20 18:36:43 ()
2004/10/20 18:36:43 ()
2004/10/20 18:36:43 GET
http://www.yahoo.com
2004/10/20 18:36:43 Not proxied
2004/10/20 18:36:43 ()
2004/10/20 18:36:43 read 634 bytes
2004/10/20 18:36:43 read 4096 bytes
…
2004/10/20 18:36:43 read 2488 bytes
2004/10/20 18:36:43 Simple response: OK
Resurrect in a Single File
• The :resurrect target uncomments lines
starting with ###l4p:
use Log4perl qw(:easy :resurrect);
sub foo {
# …
###l4p DEBUG “foo was here”;
}
Resurrect L4p in all Modules
• The Log::Log4perl::Resurrector touches all
subsequently loaded modules (experimental!):
use Log4perl qw(:easy);
use Log::Log4perl::Resurrector;
use Foo::Bar;
# Deploy a module without requiring L4p at all!
package Foo::Bar;
###l4p use Log4perl qw(:easy);
sub foo {
###l4p DEBUG “This will be resurrected!”;
}
Pitfalls
Beginner’s Pitfalls
Beginner’s Pitfalls
• Appender Additivity:
– Once a lower level logger decides to fire, it
will forward the message to its appenders.
– It will also forward the message
unconditionally to all of its parent logger’s
appenders.
Beginner’s Pitfalls
• Appender Additivity (with dupes)
log4perl.category = FATAL, Screen
log4perl.category.Net.Amazon = DEBUG, Screen
log4perl.appender.Screen = \
Log::Log4perl::Appender::Screen
log4perl.appender.Screen.layout = SimpleLayout
package Net::Amazon;
DEBUG(“Debugging!”);
DEBUG - Debugging!
DEBUG - Debugging!
Appender Additivity
File Appender
root
FATAL
Screen
Appender
Net
Net::Amazon
DEBUG
Beginner’s Pitfalls
• Appender Additivity (no dupes)
log4perl.category
= ERROR, Screen
log4perl.category.Net.Amazon
= DEBUG, Screen
log4perl.additivity.Net.Amazon = 0
log4perl.appender.Screen = \
Log::Log4perl::Appender::Screen
log4perl.appender.Screen.layout = SimpleLayout
package Net::Amazon;
DEBUG(“Debugging!”);
DEBUG – Debugging!
Q
Log::Log4perl Availability
• Cpan> install Log::Log4perl
– Only requires core modules
• Requires Perl 5.00503 or better
• Included in major Linux distros
• Windows: ppm package available in
ActiveState archives or from
log4perl.sf.net
Best Practices
• Don’t provide program name, function
name in the message. Use Layouts
instead.
• Log plenty. Don’t worry about space, use
rotating log files for chatty output.
What you should log
•
•
•
•
Program starting up, shutting down
Function parameters
Milestones in request processing
Start/end of lenghty operations (e.g. DB
access)
What you shouldn’t log
• Function/method names (although you
want parameters)
• Date
• Process ID
• … everything else provided by the layout
Best Practices
• Rolling out new L4p-enabled Modules
package My::Module;
use Log::Log4perl qw(:easy);
sub new { # … }
sub foo { DEBUG “I’m here”; # … }
sub bar { INFO “Now I’m here”; # … }
# Use w/o Log4perl
use My::Module;
$o = My::Module->new();
$o->foo();
# Log4perl enabled
use My::Module;
use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init($DEBUG);
$o = My::Module->new();
$o->foo();
Gotchas
• Avoid ‘append’ on NFS (creating lots of
‘read’ traffic)
• Don’t put expensive operations into the
logger call which are executed before the
logger decides
• Permission issues with rotating file
appenders
mod_perl
• Init() in startup handler
• Current limitation: One init().
• Use init_once() if you’re not sure who inits
first.
Combat overlapping Messages
• Either live with it or
– Use the Synchronized appender
– Use Appender.File.syswrite=1
Driving Log::Log4perl further
• Bug Fixes:
Submit to log4perl-devel@sourceforge.net
• New Features:
Propose features or send patches
• New Appenders:
Release separately as a module in the
Log::Log4perl::Appender::xx namespace
on CPAN
• … or contact me: mschilli@yahoo-inc.com
Q&A
Q&A
Thank You!
Log::Log4perl Project Page:
http://log4perl.sourceforge.net
Email me:
Mike Schilli
mschilli@yahoo-inc.com
Post to the List:
log4perl-devel@sourceforge.net
Bonus Slides
Error Handling
use Log::Log4perl qw(:easy);
do_something or
LOGDIE “Horrible error!”;
• LOGCARP, LOGCLUCK, LOGCONFESS
etc. also available.
Displaying Logs with Chainsaw
• Log4j features an XML layout and a socket
appender
• XML output is displayed in a nice GUI
called Chainsaw
• Log::Log4perl also features an XML layout
and a socket appender
• … XML output is also displayed in a nice
GUI called Chainsaw!
Displaying Logs with Chainsaw
Log::Log4perl
enabled Perl Script
Log::Log4perl::Layout::XML
Log::Log4perl::Appender::Socket
Java / log4j
Program
Displaying Logs with Chainsaw
The Future
• Allow multiple configurations within the
same process (e.g. for mod_perl)
Easy Mode Init with Files
use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init({
level
=> $DEBUG,
file
=> “>file.log”,
category => “Bar::Twix”,
});
Multiline Layout
Instead of
2007/04/04 23:59:01 This is
a message with
multiple lines
get this:
2007/04/04 23:59:01 This is
2007/04/04 23:59:01 a message with
2007/04/04 23:59:01 multiple lines
with PatternLayout::Multiline:
log4perl.appender.Logfile. layout = \
Log::Log4perl::Layout::PatternLayout::Multiline
log4perl.appender.Logfile.layout.ConversionPattern = %d %m %n
Perl Hooks
• Add parameters known at init() time
log4perl.appender.Logfile.filename = sub { \
“mylog.$$.log” }