Mastering
PostgreSQL Administration
BRUCE MOMJIAN,
ENTERPRISEDB
April, 2009
Abstract
POSTGRESQL is an open-source, full-featured relational database.
This presentation covers advanced administration topics.
Creative Commons Attribution License
http://momjian.us/presentations
Introduction
Installation
Configuration
Maintenance
Monitoring
Recovery
Mastering PostgreSQL Administration
1
Installation
Click-Through installers
–
–
–
–
MS Windows
Linux
OS/X
Solaris
Ports
–
–
–
–
RPM
DEB
PKG
other packages
Source
– obtaining
– build options
– installing
Mastering PostgreSQL Administration
2
Initialization (initdb)
$ initdb
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale C.
The default database encoding has accordingly been set to SQL_ASCII.
The default text search configuration will be set to "english".
creating directory /u/pg/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers/max_fsm_pages ... 32MB/204800
creating configuration files ... ok
creating template1 database in /u/pg/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects’ descriptions ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
Mastering PostgreSQL Administration
3
Initialization (continued)
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the -A option the
next time you run initdb.
Success. You can now start the database server using:
postgres -D /u/pg/data
or
pg_ctl -D /u/pg/data -l logfile start
Mastering PostgreSQL Administration
4
pg_controldata
pg_control version number:
Catalog version number:
Database system identifier:
Database cluster state:
pg_control last modified:
Current log file ID:
Next log file segment:
Latest checkpoint location:
Prior checkpoint location:
Latest checkpoint’s REDO location:
Latest checkpoint’s UNDO location:
Latest checkpoint’s TimeLineID:
Latest checkpoint’s NextXID:
Latest checkpoint’s NextOID:
Time of latest checkpoint:
Database block size:
Blocks per segment of large relation:
Bytes per WAL segment:
Maximum length of identifiers:
Maximum number of function arguments:
Date/time type storage:
Maximum length of locale name:
LC_COLLATE:
LC_CTYPE:
Mastering PostgreSQL Administration
74
200502281
4766833642862247929
shut down
03/03/05 10:49:18
0
1
0/A34010
0/A2D5C0
0/A34010
0/0
1
545
17233
03/03/05 10:49:18
8192
131072
16777216
64
32
floating-point numbers
128
C
C
5
System Architecture
Main
Libpq
Postmaster
Postgres
Postgres
Parse Statement
Traffic Cop
Query
utility
Utility
Command
e.g. CREATE TABLE, COPY
SELECT, INSERT, UPDATE, DELETE
Rewrite Query
Generate Paths
Optimal Path
Generate Plan
Plan
Execute Plan
Utilities
Access Methods
Mastering PostgreSQL Administration
Catalog
Storage Managers
Nodes / Lists
6
Starting Postmaster
LOG:
LOG:
LOG:
database system was shut down at 2009-01-27 10:32:05 EST
database system is ready to accept connections
autovacuum launcher started
manually
pg_ctl
on boot
Mastering PostgreSQL Administration
7
Stopping Postmaster
LOG:
LOG:
LOG:
LOG:
received smart shutdown request
autovacuum launcher shutting down
shutting down
database system is shut down
manually
pg_ctl
on shutdown
Mastering PostgreSQL Administration
8
Connections
local — unix domain socket
host —
TCP/IP,
hostssl — only
both SSL or non-SSL
SSL
hostnossl — never SSL
Mastering PostgreSQL Administration
9
Authentication
trust
passwords
– md5
– crypt
– password
remote authentication
–
–
–
–
host ident using pg_ident.conf
kerberos
gss
sspi
Mastering PostgreSQL Administration
10
Authentication (continued)
local socket ident
host ident using local identd
socket permissions
pam
ldap
reject
Mastering PostgreSQL Administration
11
Access
hostname and network mask
database name
role name (user or group)
filename or list of databases, role
IPv6
Mastering PostgreSQL Administration
12
Pg_hba.conf
# TYPE DATABASE
USER
CIDR-ADDRESS
# "local" is for Unix domain socket connections only
local
all
all
# IPv4 local connections:
host
all
all
127.0.0.1/32
# IPv6 local connections:
host
all
all
::1/128
Mastering PostgreSQL Administration
METHOD
trust
trust
trust
13
Permissions
host connection permissions
role permissions
– create roles
– create databases
– table permissions
Database creation
– template1 customization
– system tables
– disk space computations
Mastering PostgreSQL Administration
14
Data Directory
$ ls -CF
PG_VERSION
base/
global/
pg_clog/
pg_hba.conf
Mastering PostgreSQL Administration
pg_ident.conf
pg_multixact/
pg_subtrans/
pg_tblspc/
pg_twophase/
pg_xlog/
postgresql.conf
postmaster.opts
postmaster.pid
15
Database Directories
$ ls -CF global/
1136
1137
1213
1214
1232
1233
1260
1261
1262
2396
2397
2671
2672
2676
2677
2694
2695
2697
2698
2842
2843
2844
2845
2846
2847
pg_auth
pg_control
pg_database
pgstat.stat
$ ls -CF base/
1/
11510/ 11511/ 16384/
$ ls -CF base/16384
11429
11431
...
Mastering PostgreSQL Administration
2617
2618
2696
2699
16
Transaction/WAL Directories
$ ls -CF pg_xlog/
000000010000000000000000
$ ls -CF pg_clog/
0000
Mastering PostgreSQL Administration
archive_status/
17
Configuration Directories
$ ls -CF share/
conversion_create.sql
information_schema.sql
pg_hba.conf.sample
pg_ident.conf.sample
pg_service.conf.sample
postgres.bki
Mastering PostgreSQL Administration
postgres.description
postgres.shdescription
postgresql.conf.sample
psqlrc.sample
recovery.conf.sample
snowball_create.sql
sql_features.txt
system_views.sql
timezone/
timezonesets/
tsearch_data/
unknown.pltcl
18
Configuration Postgresql.conf
Mastering PostgreSQL Administration
19
PostgreSQL.Conf
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
----------------------------PostgreSQL configuration file
----------------------------This file consists of lines of the form:
name = value
(The "=" is optional.) Whitespace may be used. Comments are introduced with
"#" anywhere on a line. The complete list of parameter names and allowed
values can be found in the PostgreSQL documentation.
The commented-out settings shown in this file represent the default values.
Re-commenting a setting is NOT sufficient to revert it to the default value;
you need to reload the server.
Mastering PostgreSQL Administration
20
PostgreSQL.Conf (Continued)
#
#
#
#
#
#
#
#
#
#
#
#
This file is read on server startup and when the server receives a SIGHUP
signal. If you edit the file on a running system, you have to SIGHUP the
server for the changes to take effect, or use "pg_ctl reload". Some
parameters, which are marked below, require a server shutdown and restart to
take effect.
Any parameter can also be given as a command-line option to the server, e.g.,
"postgres -c log_connections=on". Some paramters can be changed at run time
with the "SET" SQL command.
Memory units: kB = kilobytes MB = megabytes GB = gigabytes
Time units:
ms = milliseconds s = seconds min = minutes h = hours d = days
Mastering PostgreSQL Administration
21
Configuration File Location
#data_directory = ’ConfigDir’
# use data in another directory
# (change requires restart)
#hba_file = ’ConfigDir/pg_hba.conf’
# host-based authentication file
# (change requires restart)
#ident_file = ’ConfigDir/pg_ident.conf’ # ident configuration file
# (change requires restart)
# If external_pid_file is not explicitly set, no extra PID file is written.
#external_pid_file = ’(none)’
# write an extra PID file
# (change requires restart)
Mastering PostgreSQL Administration
22
Connections and Authentication
#listen_addresses = ’localhost’
# what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to ’localhost’, ’*’ = all
# (change requires restart)
#port = 5432
# (change requires restart)
max_connections = 100
# (change requires restart)
# Note: Increasing max_connections costs ~400 bytes of shared memory per
# connection slot, plus lock space (see max_locks_per_transaction). You might
# also need to raise shared_buffers to support more connections.
#superuser_reserved_connections = 3
# (change requires restart)
#unix_socket_directory = ’’
# (change requires restart)
#unix_socket_group = ’’
# (change requires restart)
#unix_socket_permissions = 0777
# begin with 0 to use octal notation
# (change requires restart)
#bonjour_name = ’’
# defaults to the computer name
# (change requires restart)
Mastering PostgreSQL Administration
23
Security and Authentication
#authentication_timeout = 1min
# 1s-600s
#ssl = off
# (change requires restart)
#ssl_ciphers = ’ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH’
# allowed SSL ciphers
# (change requires restart)
#password_encryption = on
#db_user_namespace = off
# Kerberos and GSSAPI
#krb_server_keyfile = ’’
# (change requires restart)
#krb_srvname = ’postgres’
# (change requires restart, Kerberos only)
#krb_server_hostname = ’’
# empty string matches any keytab entry
# (change requires restart, Kerberos only)
#krb_caseins_users = off
# (change requires restart)
#krb_realm = ’’
# (change requires restart)
Mastering PostgreSQL Administration
24
TCP/IP Control
# see "man 7 tcp" for details
#tcp_keepalives_idle = 0
#tcp_keepalives_interval = 0
#tcp_keepalives_count = 0
Mastering PostgreSQL Administration
#
#
#
#
#
#
TCP_KEEPIDLE, in seconds;
0 selects the system default
TCP_KEEPINTVL, in seconds;
0 selects the system default
TCP_KEEPCNT;
0 selects the system default
25
Memory Usage
shared_buffers = 32MB
# min 128kB or max_connections*16kB
# (change requires restart)
#temp_buffers = 8MB
# min 800kB
#max_prepared_transactions = 5
# can be 0 or more
# (change requires restart)
# Note: Increasing max_prepared_transactions costs ~600 bytes of shared memory
# per transaction slot, plus lock space (see max_locks_per_transaction).
#work_mem = 1MB
# min 64kB
#maintenance_work_mem = 16MB
# min 1MB
#max_stack_depth = 2MB
# min 100kB
Kernel changes often required.
Mastering PostgreSQL Administration
26
Sizing Shared Memory
Postgres Backend
Postgres Backend
Postgres Backend
R
A
PostgreSQL Shared Buffer Cache
M
Kernel Disk Buffer Cache
Page Out
Swap
Free
Page In
Kernel
Mastering PostgreSQL Administration
27
Free Space Map and Kernel Resources
# - Free Space Map max_fsm_pages = 204800
#max_fsm_relations = 1000
# min max_fsm_relations*16, 6 bytes each
# (change requires restart)
# min 100, ~70 bytes each
# (change requires restart)
# - Kernel Resource Usage #max_files_per_process = 1000
#shared_preload_libraries = ’’
Mastering PostgreSQL Administration
# min 25
# (change requires restart)
# (change requires restart)
28
Vacuum and Background Writer
# - Cost-Based Vacuum Delay #vacuum_cost_delay = 0
#vacuum_cost_page_hit = 1
#vacuum_cost_page_miss = 10
#vacuum_cost_page_dirty = 20
#
#
#
#
#vacuum_cost_limit = 200
# 1-10000 credits
0-1000 milliseconds
0-10000 credits
0-10000 credits
0-10000 credits
# - Background Writer #bgwriter_delay = 200ms
# 10-10000ms between rounds
#bgwriter_lru_maxpages = 100
# 0-1000 max buffers written/round
#bgwriter_lru_multiplier = 2.0
# 010.0 multipler on buffers scanned/round
Mastering PostgreSQL Administration
29
Write-Ahead Log (WAL)
#fsync = on
#synchronous_commit = on
#wal_sync_method = fsync
#full_page_writes = on
#wal_buffers = 64kB
#wal_writer_delay = 200ms
#commit_delay = 0
#commit_siblings = 5
Mastering PostgreSQL Administration
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
turns forced synchronization on or off
immediate fsync at commit
the default is the first option
supported by the operating system:
open_datasync
fdatasync
fsync
fsync_writethrough
open_sync
recover from partial page writes
min 32kB
(change requires restart)
1-10000 milliseconds
range 0-100000, in microseconds
range 1-1000
30
Write-Ahead Logging (Continued)
Postgres
Backend
Postgres
Backend
PostgreSQL Shared Buffer Cache
Postgres
Backend
Write−Ahead Log
fsync
Kernel Disk Buffer Cache
fsync
Disk Blocks
Mastering PostgreSQL Administration
31
Checkpoints and Archiving
# - Checkpoints #checkpoint_segments = 3
#checkpoint_timeout = 5min
#checkpoint_completion_target = 0.5
# in logfile segments, min 1, 16MB each
# range 30s-1h
# checkpoint target duration, 0.0 - 1.0
#checkpoint_warning = 30s
# 0 is off
# - Archiving #archive_mode = off
#archive_command = ’’
#archive_timeout = 0
Mastering PostgreSQL Administration
#
#
#
#
#
allows archiving to be done
(change requires restart)
command to use to archive a logfile segment
force a logfile segment switch after this
time; 0 is off
32
Query Tuning
# - Planner Method Configuration #enable_bitmapscan = on
#enable_hashagg = on
#enable_hashjoin = on
#enable_indexscan = on
#enable_mergejoin = on
#enable_nestloop = on
#enable_seqscan = on
#enable_sort = on
#enable_tidscan = on
# - Planner Cost Constants #seq_page_cost = 1.0
#random_page_cost = 4.0
#cpu_tuple_cost = 0.01
#cpu_index_tuple_cost = 0.005
#cpu_operator_cost = 0.0025
#effective_cache_size = 128MB
Mastering PostgreSQL Administration
#
#
#
#
#
measured on an arbitrary scale
same scale as above
same scale as above
same scale as above
same scale as above
33
Query Tuning (Continued)
#geqo = on
#geqo_threshold = 12
#geqo_effort = 5
#geqo_pool_size = 0
#geqo_generations = 0
# range 1-10
# selects default based on effort
# selects default based on effort
#geqo_selection_bias = 2.0
# range 1.5-2.0
# - Other Planner Options #default_statistics_target = 10
#constraint_exclusion = off
#from_collapse_limit = 8
#join_collapse_limit = 8
Mastering PostgreSQL Administration
# range 1-1000
# 1 disables collapsing of explicit
# JOIN clauses
34
Where To Log
#log_destination = ’stderr’
#
#
#
#
Valid values are combinations of
stderr, csvlog, syslog and eventlog,
depending on platform. csvlog
requires logging_collector to be on.
# This is used when logging to stderr:
#logging_collector = off
# Enable capturing of stderr and csvlog
# into log files. Required to be on for
# csvlogs.
# (change requires restart)
# These are only used if logging_collector is on:
#log_directory = ’pg_log’
# directory where log files are written,
# can be absolute or relative to PGDATA
#log_filename = ’postgresql-%Y-%m-%d_%H%M%S.log’
# log file name pattern,
# can include strftime() escapes
Mastering PostgreSQL Administration
35
Where To Log (Continued)
#log_truncate_on_rotation = off
#log_rotation_age = 1d
#log_rotation_size = 10MB
#
#
#
#
#
#
#
#
#
#
#
#
If on, an existing log file of the
same name as the new log file will be
truncated rather than appended to.
But such truncation only occurs on
time-driven rotation, not on restarts
or size-driven rotation. Default is
off, meaning append to existing files
in all cases.
Automatic rotation of logfiles will
happen after that time. 0 to disable.
Automatic rotation of logfiles will
happen after that much log output.
# 0 to disable.
# These are relevant when logging to syslog:
#syslog_facility = ’LOCAL0’
#syslog_ident = ’postgres’
Mastering PostgreSQL Administration
36
When to Log
#client_min_messages = notice
#log_min_messages = notice
Mastering PostgreSQL Administration
# values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# log
# notice
# warning
# error
# values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic
37
When to Log (Continued)
#log_error_verbosity = default
#log_min_error_statement = error
#log_min_duration_statement = -1
#silent_mode = off
Mastering PostgreSQL Administration
# terse, default, or verbose messages
# values in order of decreasing detail:
# debug5
# debug4
# debug3
# debug2
# debug1
# info
# notice
# warning
# error
# log
# fatal
# panic (effectively off)
# -1 is disabled, 0 logs all statements
# and their durations, > 0 logs only
# statements running at least this time.
# DO NOT USE without syslog or
# logging_collector
# (change requires restart)
38
What to Log
#debug_print_parse = off
#debug_print_rewritten = off
#debug_print_plan = off
#debug_pretty_print = off
#log_checkpoints = off
#log_connections = off
#log_disconnections = off
#log_duration = off
#log_hostname = off
Mastering PostgreSQL Administration
39
What To Log: Log_line_prefix
#log_line_prefix = ’’
Mastering PostgreSQL Administration
# special values:
# %u = user name
# %d = database name
# %r = remote host and port
# %h = remote host
# %p = process ID
# %t = timestamp without milliseconds
# %m = timestamp with milliseconds
# %i = command tag
# %c = session ID
# %l = session line number
# %s = session start timestamp
# %v = virtual transaction ID
# %x = transaction ID (0 if none)
# %q = stop here in non-session
#
processes
# %% = ’%’
# e.g. ’<%u%%%d> ’
40
What to Log (Continued)
#log_lock_waits = off
#log_statement = ’none’
#log_temp_files = -1
#log_timezone = unknown
Mastering PostgreSQL Administration
#
#
#
#
#
#
#
log lock waits >= deadlock_timeout
none, ddl, mod, all
log temporary files equal or larger
than specified size;
-1 disables, 0 logs all temp files
actually, defaults to TZ environment
setting
41
Runtime Statistics
# - Query/Index Statistics Collector #track_activities = on
#track_counts = on
#update_process_title = on
# - Statistics Monitoring #log_parser_stats = off
#log_planner_stats = off
#log_executor_stats = off
#log_statement_stats = off
Mastering PostgreSQL Administration
42
Autovacuum
#autovacuum = on
# Enable autovacuum subprocess? ’on’
# requires track_counts to also be on.
#log_autovacuum_min_duration = -1
# -1 disables, 0 logs all actions and
# their durations, > 0 logs only
# actions running at least that time.
#autovacuum_max_workers = 3
# max number of autovacuum subprocesses
#autovacuum_naptime = 1min
# time between autovacuum runs
#autovacuum_vacuum_threshold = 50
# min number of row updates before
# vacuum
#autovacuum_analyze_threshold = 50
# min number of row updates before
# analyze
#autovacuum_vacuum_scale_factor = 0.2 # fraction of table size before vacuum
#autovacuum_analyze_scale_factor = 0.1 # fraction of table size before analyze
#autovacuum_freeze_max_age = 200000000 # maximum XID age before forced vacuum
# (change requires restart)
#autovacuum_vacuum_cost_delay = 20
# default vacuum cost delay for
# autovacuum, -1 means use
# vacuum_cost_delay
#autovacuum_vacuum_cost_limit = -1
# default vacuum cost limit for
# autovacuum, -1 means use
# vacuum_cost_limit
Mastering PostgreSQL Administration
43
Statement Behavior
#search_path = ’"$user",public’
#default_tablespace = ’’
#temp_tablespaces = ’’
#
#
#
#
schema names
a tablespace name, ’’ uses the default
a list of tablespace names, ’’ uses
only default tablespace
#check_function_bodies = on
#default_transaction_isolation = ’read committed’
#default_transaction_read_only = off
#session_replication_role = ’origin’
#statement_timeout = 0
# 0 is disabled
#vacuum_freeze_min_age = 100000000
#xmlbinary = ’base64’
#xmloption = ’content’
Mastering PostgreSQL Administration
44
Locale and Formatting
datestyle = ’iso, mdy’
#timezone = unknown
# actually, defaults to TZ environment
# setting
#timezone_abbreviations = ’Default’
# Select the set of available time zone
# abbreviations. Currently, there are
# Default
# Australia
# India
# You can create your own file in
# share/timezonesets/.
#extra_float_digits = 0
# min -15, max 2
#client_encoding = sql_ascii
# actually, defaults to database
# encoding
# These settings are initialized by initdb, but they can be changed.
lc_messages = ’C’
# locale for system error message
# strings
lc_monetary = ’C’
# locale for monetary formatting
lc_numeric = ’C’
# locale for number formatting
lc_time = ’C’
# locale for time formatting
Mastering PostgreSQL Administration
45
Full Text Search
# default configuration for text search
default_text_search_config = ’pg_catalog.english’
Mastering PostgreSQL Administration
46
Other Defaults
#explain_pretty_print = on
#dynamic_library_path = ’$libdir’
#local_preload_libraries = ’’
Mastering PostgreSQL Administration
47
Lock Management
#deadlock_timeout = 1s
#max_locks_per_transaction = 64
# min 10
# (change requires restart)
# Note: Each lock table slot uses ~270 bytes of shared memory, and there are
# max_locks_per_transaction * (max_connections + max_prepared_transactions)
# lock table slots.
Mastering PostgreSQL Administration
48
Version/Platform Compatibility
# - Previous PostgreSQL Versions #add_missing_from = off
#array_nulls = on
#backslash_quote = safe_encoding
#default_with_oids = off
#escape_string_warning = on
#regex_flavor = advanced
#sql_inheritance = on
#standard_conforming_strings = off
# on, off, or safe_encoding
# advanced, extended, or basic
#synchronize_seqscans = on
# - Other Platforms and Clients #transform_null_equals = off
Mastering PostgreSQL Administration
49
Customization
#custom_variable_classes = ’’
Mastering PostgreSQL Administration
# list of custom variable class names
50
Interfaces
Installing
– Compiled Languages (C, ecpg)
– Scripting Language (Perl, Python)
– SPI
Connection Pooling
Mastering PostgreSQL Administration
51
Include Files
$ ls -CF include/
ecpg_config.h
ecpg_informix.h
ecpgerrno.h
ecpglib.h
ecpgtype.h
informix/
internal/
Mastering PostgreSQL Administration
libpq/
libpq-fe.h
pg_config.h
pg_config_manual.h
pg_config_os.h
pgtypes_date.h
pgtypes_error.h
pgtypes_interval.h
pgtypes_numeric.h
pgtypes_timestamp.h
postgres_ext.h
server/
sql3types.h
sqlca.h
52
Library Files
$ ls -CF lib/
ascii_and_mic.so*
cyrillic_and_mic.so*
dict_snowball.so*
euc_cn_and_mic.so*
euc_jis_2004_and_shift_
euc_jp_and_sjis.so*
euc_kr_and_mic.so*
euc_tw_and_big5.so*
latin2_and_win1250.so*
latin_and_mic.so*
libecpg.a
libecpg.so@
libecpg.so.6@
libecpg.so.6.0*
libecpg_compat.a
libecpg_compat.so@
Mastering PostgreSQL Administration
libecpg_compat.so.3@
libecpg_compat.so.3.0*
libpgport.a
libpgtypes.a
libpgtypes.so@
libpgtypes.so.3@
libpgtypes.so.3.0*
libpq.a
libpq.so@
libpq.so.5@
libpq.so.5.1*
pgxs/
plperl.so*
plpgsql.so*
pltcl.so*
utf8_and_ascii.so*
utf8_and_big5.so*
utf8_and_cyrillic.so*
utf8_and_euc_cn.so*
utf8_and_euc_jis_2004.s
utf8_and_euc_jp.so*
utf8_and_euc_kr.so*
utf8_and_euc_tw.so*
utf8_and_gb18030.so*
utf8_and_gbk.so*
utf8_and_iso8859.so*
utf8_and_iso8859_1.so*
utf8_and_johab.so*
utf8_and_shift_jis_2004
utf8_and_sjis.so*
utf8_and_uhc.so*
utf8_and_win.so*
53
Maintenance
Mastering PostgreSQL Administration
54
Backup
File system-level
– tar, cpio while shutdown
– file system snapshot
– rsync, shutdown, rsync, restart
pg_dump/pg_dumpall
restore/pg_restore with custom format
Mastering PostgreSQL Administration
55
WAL
W
AL
File System-
Continuous
Level Backup
Backup
Mastering PostgreSQL Administration
W
00
13
00
11
00
09
02
00
Continuous Archiving /
Point-In-Time Recovery (PITR)
AL
56
PITR Backup Procedures
1. archive_command = ’cp %p /mnt/server/pgsql/%f ’
2. SELECT pg_start_backup(’label’);
3. Perform file system-level backup (can be inconsistent)
4. SELECT pg_stop_backup();
Mastering PostgreSQL Administration
57
WAL
W
AL
File System-
Continuous
Level Backup
Backup
Mastering PostgreSQL Administration
W
55
17
40
17
30
17
17
00
PITR Recovery
AL
58
PITR Recovery Procecdures
1. Stop postmaster
2. Restore file system-level backup
3. Make adjustments as outlined in the documentation
4. Create recovery.conf
5. Add restore_command = ’cp /mnt/server/pgsql/%f %p’
6. Start the postmaster
Mastering PostgreSQL Administration
59
Master-Slave Replication - Slony
0% /% 0% /% 0%/%0% /% 0% /% 0% /% 0%/%0%/%0%/% 0% /% 0% /% 0% /% 0% /% 0 /
6% 5% 6% 5% 6%5%6% 5% 6% 5% 6% 5% 6%5%6%5%6%5% 6% 5% 6% 5% 6% 5% 6% 5% 6 5
#" #" ! ! !
*%# %
* #%
*%
* !%
* !%
* !%
*%
* %)( %
* %
* %)( * )(
)( " %
)( " %
)( %
)( %
)( %
)( %
)( %*%)( %
)( %* %
)( %* %)( %
#" #" ! ! !
#" #" ! ! !
, %+ , +
+ %
+ %
+, %
%
+%
+%
+%
+, %,%+%
%
+, %
%
+ %, %
+ %, %+ %
, %
,%%
, %
, %
, %
, %
.% %
. %
.%
. %
. %
. %
.%
. %- %
. %
. %- . -
- %
- %
-%
-%
-%
-%
- %.%-%
- %. %
- %. %- %
'% %
' %& %'%
' %& %' %
' %& %
'%
' %& %
' %
' %
' %& ' &
& %
&%
&%
& %'%
&%
& %
& %' %
& %
$ %$ %
$ %$ %
$ %$ %
$ %
$ %$ %
$ %
$ %
$ %$ $
%
4% %
4 %3
%4%
4 %3 %4 %
4 %3 %
4%
4 %3 %
4 %
4 %
4 %3 4 3
3
%
3%
3%
3 %4%
3%
3 %
3 %4 %
3 %
2% %
2 %1%2%
2 %1 %2 %
2 %1 %
2%
2 %1 %
2 %
2 %
2 %1 2 1
1%
1%
1%
1 %2%
1%
1 %
1 %2 %
1 %
60
Mastering PostgreSQL Administration
Other Solutions
Mutli-master replication: pgcluster, bacardo
Pooling: pgpool
Mastering PostgreSQL Administration
61
Data Maintenance
VACUUM
(nonblocking), free space map
VACUUM FULL
ANALYZE
Mastering PostgreSQL Administration
62
Vacuum
Free Space Map
Table
Block #
Block #
Table
Block #
Block #
Table
Block #
Block #
Block #
DB oid
Relfilenode
Block #
Hashed
Shared Memory
Mastering PostgreSQL Administration
63
Vacuum Full
Original Heap
With Expired
Rows Identified
Move Trailing
Rows Into Expired
Slots
Truncate File
Mastering PostgreSQL Administration
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
E
X
P
I
R
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
E
X
P
I
R
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
A
C
T
I
V
E
64
Checkpoints
Write all dirty shared buffers
Sync all dirty kernel buffers
Recycle WAL files
Check for server messages indicating too-frequent checkpoints
If so, increase checkpoint_segments
Mastering PostgreSQL Administration
65
Automating Tasks
0 3 * * * root psql -c ’VACUUM FULL;’ test
0 3 * * * root vacuumdb -a -f
Autovacuum usually makes vacuum administration unnecessary.
Mastering PostgreSQL Administration
66
Monitoring Active Sessions
Mastering PostgreSQL Administration
67
ps
$ ps -Upostgres
PID TT STAT
2125
2142
2143
3341
3340
??
??
??
??
p6
Ss
S
S
I
I+
TIME COMMAND
0:00.26
0:00.03
0:00.06
0:00.07
0:00.03
Mastering PostgreSQL Administration
./bin/postmaster -i
stats buffer process
(postmaster)
stats collector process
(postmaster)
postgres test [local] idle (postmaster)
psql test
68
top
$ top
load averages: 0.56, 0.39, 0.36
18:25:58
138 processes: 5 running, 130 sleeping, 3 zombie
CPU states: 50.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 50.0% idle
Memory: Real: 96M/133M Virt: 535M/1267M Free: 76M
PID USERNAME PRI NICE SIZE RES STATE
23785 postgres 57
0 11M 5336K run/0
23784 postgres 2
0 10M 11M sleep
Mastering PostgreSQL Administration
TIME WCPU
CPU COMMAND
0:07 30.75% 30.66% postmaster
0:00 2.25% 2.25% psql
69
pgmonitor
Mastering PostgreSQL Administration
70
Query Monitoring
test=> SELECT * FROM pg_stat_activity;
-[ RECORD 1 ]-+-------------------------------datid
| 16384
datname
| test
procpid
| 2373
usesysid
| 10
usename
| postgres
current_query | select * from pg_stat_activity;
waiting
| f
xact_start
| 2009-01-27 12:12:59.917695-05
query_start
| 2009-01-27 12:12:59.917695-05
backend_start | 2009-01-27 12:12:33.422001-05
client_addr
|
client_port
| -1
Mastering PostgreSQL Administration
71
pgmonitor
Mastering PostgreSQL Administration
72
Access Statistics
pg_stat_all_indexes
pg_stat_all_tables
pg_stat_database
pg_stat_sys_indexes
pg_stat_sys_tables
pg_stat_user_indexes
pg_stat_user_tables
pg_statio_all_indexes
pg_statio_all_sequences
pg_statio_all_tables
pg_statio_sys_indexes
pg_statio_sys_sequences
pg_statio_sys_tables
pg_statio_user_indexes
pg_statio_user_sequences
pg_statio_user_tables
Mastering PostgreSQL Administration
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
view
view
view
view
view
view
view
view
view
view
view
view
view
view
view
view
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
postgres
postgres
postgres
postgres
postgres
postgres
postgres
postgres
postgres
postgres
postgres
postgres
postgres
postgres
postgres
postgres
73
Database Statistics
test=> SELECT * FROM pg_stat_database;
...
-[ RECORD 4 ]-+---------datid
| 16384
datname
| test
numbackends
| 1
xact_commit
| 188
xact_rollback | 0
blks_read
| 95
blks_hit
| 11832
tup_returned | 64389
tup_fetched
| 2938
tup_inserted | 0
tup_updated
| 0
tup_deleted
| 0
Mastering PostgreSQL Administration
74
Table Activity
test=> SELECT * FROM pg_stat_all_tables;
...
-[ RECORD 10 ]---+-----------------------relid
| 2616
schemaname
| pg_catalog
relname
| pg_opclass
seq_scan
| 2
seq_tup_read
| 2
idx_scan
| 99
idx_tup_fetch
| 99
n_tup_ins
| 0
n_tup_upd
| 0
n_tup_del
| 0
n_tup_hot_upd
| 0
n_live_tup
| 0
Mastering PostgreSQL Administration
75
n_dead_tup
last_vacuum
last_autovacuum
last_analyze
last_autoanalyze
| 0
|
|
|
|
Mastering PostgreSQL Administration
76
Table Block Activity
est=> SELECT * FROM pg_statio_all_tables;
-[ RECORD 50 ]--+-----------------------relid
| 2602
schemaname
| pg_catalog
relname
| pg_amop
heap_blks_read | 3
heap_blks_hit
| 114
idx_blks_read
| 5
idx_blks_hit
| 303
toast_blks_read |
toast_blks_hit |
tidx_blks_read |
tidx_blks_hit
|
t
Mastering PostgreSQL Administration
77
Analyzing Activity
Heavily used tables
Unnecessary indexes
Additional indexes
Index usage
TOAST
usage
Mastering PostgreSQL Administration
78
CPU
$ vmstat 5
procs
memory
r b w
avm
fre
1 0 0 501820 48520
3 0 0 512796 46812
3 0 0 542260 44356
4 0 0 539708 41868
4 0 0 547200 32964
4 0 0 556140 23884
1 0 0 535136 46280
page
flt re
1234 86
1422 201
788 137
576 65
454 0
461 0
1056 141
Mastering PostgreSQL Administration
pi po fr sr
2
0 0 3
12 0 0 0
6
0 0 0
13 0 0 0
0
0 0 0
0
0 0 0
25 0 0 0
disks
s0 s0
5 0
3 0
8 0
4 0
5 0
2 0
2 0
faults
in sy cs
263 2881 599
259 6483 827
286 5698 741
273 5721 819
253 5736 948
249 5917 959
261 6417 890
cpu
us sy
10 4
4 7
2 5
16 4
50 4
52 3
24 6
id
86
88
94
80
46
44
70
79
I/O
$ iostat 5
tty
sd0
tin tout sps tps msps
7 119 244 11 6.1
0 86
20
1 1.4
0 82
61
4 3.6
0 65
6
0 0.0
12 90
31
2 5.4
24 173
6
0 4.9
0 91 3594 63 4.6
Mastering PostgreSQL Administration
sps tps
0
0
0
0
0
0
0
0
0
0
0
0
0
0
sd1
msps
27.3
0.0
0.0
0.0
0.0
0.0
0.0
sps tps
0
0
0
0
0
0
0
0
0
0
0
0
0
0
sd2
%
msps usr nic sys int
18.1
9 1 4
0
0.0
2 0 2
0
0.0
2 0 2
0
0.0
1 0 2
0
0.0
4 0 3
0
0.0 48 0 3
0
0.0 11 0 4
0
cpu
idl
86
96
97
97
93
49
85
80
Disk Usage
play=# SELECT relfilenode, relpages * 8 AS kilobytes
play-# FROM pg_class
play-# WHERE relname = ’customer’;
relfilenode | kilobytes
-------------+---------16806 |
480
(1 row)
Vacuum required. dbsize available.
Mastering PostgreSQL Administration
81
TOAST Usage
play=#
play-#
play-#
play-#
play-#
SELECT relname, relpages * 8 AS kilobytes
FROM pg_class
WHERE relname = ’pg_toast_16806’ OR
relname = ’pg_toast_16806_index’
ORDER BY relname;
relname
| kilobytes
----------------------+---------pg_toast_16806
|
0
pg_toast_16806_index |
1
Mastering PostgreSQL Administration
82
Index Usage
play=#
play-#
play-#
play-#
play-#
play-#
SELECT c2.relname, c2.relpages * 8 AS kilobytes
FROM pg_class c, pg_class c2, pg_index i
WHERE c.relname = ’customer’ AND
c.oid = i.indrelid AND
c2.oid = i.indexrelid
ORDER BY c2.relname;
relname
| kilobytes
----------------------+---------customer_id_indexdex |
26
Mastering PostgreSQL Administration
83
Largest Tables
play=# SELECT relname, relpages * 8
play-# FROM pg_class
play-# ORDER BY relpages DESC;
relname
| kilobytes
----------------------+---------bigtable
|
3290
customer
|
3144
Mastering PostgreSQL Administration
84
Database File Mapping - oid2name
$ oid2name
All databases:
--------------------------------18720 = test1
1
= template1
18719 = template0
18721 = test
18735 = postgres
18736 = cssi
Mastering PostgreSQL Administration
85
Table File Mapping
$ cd /usr/local/pgsql/data/base
$ oid2name
All databases:
--------------------------------16817 = test2
16578 = x
16756 = test
1
= template1
16569 = template0
16818 = test3
16811 = floattest
$ cd 16756
$ ls 1873*
18730 18731
18732
Mastering PostgreSQL Administration
18735
18736
18737
18738
18739
86
$ oid2name -d test -o 18737
Tablename of oid 18737 from database "test":
--------------------------------18737 = ips
$ oid2name -d test -t ips
Oid of table ips from database "test":
--------------------------------18737 = ips
$ # show disk space for every db object
$ du * | while read SIZE RELFILENODE
> do
>
echo "$SIZE
‘oid2name -q -d test -o $RELFILENODE‘"
> done
24
18737 = ips
36
18722 = cities
...
Mastering PostgreSQL Administration
87
$ # same as above, but sort by largest first
$ du * | while read SIZE OID
> do
>
echo "$SIZE
‘oid2name -q -d test -o $OID‘"
> done |
> sort -rn
2048
19324 = bigtable
1950
23903 = customers
...
$ # show disk usage per database
$ cd /usr/local/pgsql/data/base
$ du -s * |
> while read SIZE OID
> do
>
echo "$SIZE
‘oid2name -q | grep ^$OID’ ’‘"
> done |
> sort -rn
2256
18721 = test
2135
18735 = postgres
Mastering PostgreSQL Administration
88
Disk Balancing
Move pg_xlog to another drive using symlinks
Tablespaces
Mastering PostgreSQL Administration
89
Per-Database Tablespaces
DB1
Disk 1
Mastering PostgreSQL Administration
DB2
DB3
Disk 2
DB4
Disk 3
90
Per-Object Tablespaces
tab1
Disk 1
Mastering PostgreSQL Administration
tab2
index constraint
Disk 2
Disk 3
91
Analyzing Locking
$ ps -Upostgres
PID TT STAT
9874 ?? I
9835 ?? S
10295 ?? S
TIME
0:00.07
0:00.05
0:00.05
COMMAND
postgres test [local] idle in transaction (postmaster)
postgres test [local] UPDATE waiting (postmaster)
postgres test [local] DELETE waiting (postmaster)
test=> SELECT * FROM pg_locks;
relation | database | transaction | pid |
mode
| granted
----------+----------+-------------+------+------------------+--------17143 |
17142 |
| 9173 | AccessShareLock | t
17143 |
17142 |
| 9173 | RowExclusiveLock | t
|
|
472 | 9380 | ExclusiveLock
| t
|
|
468 | 9338 | ShareLock
| f
|
|
470 | 9338 | ExclusiveLock
| t
16759 |
17142 |
| 9380 | AccessShareLock | t
17143 |
17142 |
| 9338 | AccessShareLock | t
17143 |
17142 |
| 9338 | RowExclusiveLock | t
|
|
468 | 9173 | ExclusiveLock
| t
(9 rows)
Mastering PostgreSQL Administration
92
Miscellaneous Tasks
Log file rotation, syslog
Upgrading
Migration
Mastering PostgreSQL Administration
93
Administration Tools
pgadmin
phppgadmin
Mastering PostgreSQL Administration
94
Recovery
Mastering PostgreSQL Administration
95
Client Application Crash
Nothing Required. Transactions in progress are rolled back.
Mastering PostgreSQL Administration
96
Graceful Server Crash
Nothing Required. Transactions in progress are rolled back.
Mastering PostgreSQL Administration
97
Abrupt Server Crash
Nothing Required. Transactions in progress are rolled back.
Mastering PostgreSQL Administration
98
Operating System Crash
Nothing Required. Transactions in progress are rolled back. Partial page
writes are repaired.
Mastering PostgreSQL Administration
99
Disk Failure
Restore from previous backup or use PITR.
Mastering PostgreSQL Administration
100
Accidental DELETE
Recover table from previous backup, perhaps using pg_restore. It is
possible to modify the backend code to make deleted tuples visible,
dump out the deleted table and restore the original code. All tuples in
the table since the previous vacuum will be visible. It is possible to
restrict that so only tuples deleted by a specific transaction are visible.
Mastering PostgreSQL Administration
101
Write-Ahead Log (WAL) Corruption
See pg_resetxlog. Review recent transactions and identify any damage,
including partially committed transactions.
Mastering PostgreSQL Administration
102
File Deletion
It may be necessary to create an empty file with the deleted file name so
the object can be deleted, and then the object restored from backup.
Mastering PostgreSQL Administration
103
Accidental DROP TABLE
Restore from previous backup.
Mastering PostgreSQL Administration
104
Accidental DROP INDEX
Recreate index.
Mastering PostgreSQL Administration
105
Accidental DROP DATABASE
Restore from previous backup.
Mastering PostgreSQL Administration
106
Non-Starting Installation
Restart problems are usually caused by write-ahead log problems. See
pg_resetxlog. Review recent transactions and identify any damage,
including partially committed transactions.
Mastering PostgreSQL Administration
107
Index Corruption
Use REINDEX.
Mastering PostgreSQL Administration
108
Table Corruption
Try reindexing the table. Try identifying the corrupt OID of the row and
transfer the valid rows into another table using SELECT…INTO…WHERE
oid != ###. Use http://sources.redhat.com/rhdb/tools.html to analyze
the internal structure of the table.
Mastering PostgreSQL Administration
109
Conclusion
Mastering PostgreSQL Administration
110