Managing Resources with PostgreSQL
FOSDEM PGDay 2015
Cédric Villemain cedric@2ndQuadrant.fr
30th January 2015
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
1 / 24
Cédric Villemain
PostgreSQL Expertise
Development, Support, Training
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
2 / 24
What resources ?
•
•
•
CPU
RAM
Disk
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
3 / 24
How ?
•
•
with .... Concurrency
with .... Quota or Limit
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
4 / 24
Where ?
Operating System
•
•
•
user process (limits.conf, ...)
container (cgroup, jail, ...)
virtualization (VMWare, Xen, ...)
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
5 / 24
Where ?
PostgreSQL
•
•
•
•
•
•
Main conguration (postgresql.conf)
per object
per role
per database
per role in database
pg_service
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
6 / 24
GUCs with restart
•
•
max_connections
shared_buers
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
7 / 24
GUCs without restart
•
•
•
•
temp_buers
work_mem
synchronous_commit
temp_tablespace
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
8 / 24
More GUCs without restart
•
•
•
temp_le_limit
statement_timeout
lock_timeout
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
9 / 24
Resources per object
•
•
•
•
FUNCTION
ROLE
DATABASE
SYSTEM
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
10 / 24
Function example
ALTER FUNCTION update_datamart()
SET temp_file_limit = -1;
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
11 / 24
Database example
ALTER DATABASE datawarehouse
WITH CONNECTION LIMIT 10; --hard limit
ALTER DATABASE datawarehouse
SET temp_tablespace TO dwh_tblspc;
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
12 / 24
Role example
ALTER ROLE web_user
WITH CONNECTION LIMIT 10; -- hard limit but ...
ALTER ROLE dba
SET work_mem = '128MB';
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
13 / 24
Role in Database example
ALTER ROLE ALL IN DATABASE devel
SET synchronous_commit TO off;
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
14 / 24
System example
ALTER SYSTEM
SET temp_buffers = '12MB';
SELECT pg_reload_conf();
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
15 / 24
Connection Limit
•
•
PostgreSQL ood & DoS
pooling & bouncing
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
16 / 24
PgBouncer
•
•
pool_size = 20
max_client_conn = 2000
my_db = user=web_user host=remote_host pool_size=4
•
bonus: PAUSE / RESUME
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
17 / 24
top-like
•
•
pg_activity (python, system only)
pg_top (C, PostgreSQL extension)
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
18 / 24
pg_proctab
•
•
•
•
pg_cputime()
pg_loadavg()
pg_memusage()
pg_proctab()
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
19 / 24
Execution depends on loadavg
if (select load1<1 from pg_loadavg())
then update_datamart();
end if;
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
20 / 24
What about eective_cache_size ?
select pg_size_pretty(memcached * 4096)
from pg_memusage;
-- don't forget to count shared_buffers
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
21 / 24
Are we writing a lot on disk?
select pg_size_pretty(wchar) as requested_write,
pg_size_pretty(writes) as really_written,
(writes * 100 / wchar) as percent_really_written
from pg_proctab()
where pid = pg_backend_pid())
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
22 / 24
And next ?
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
23 / 24
Questions ?
Now is the time to ask!
Cédric Villemain cedric@2ndQuadrant.fr () Managing Resources with PostgreSQL
30th January 2015
24 / 24