Varnish Administration Console
Installation Guide
doc# 110
vac 3.2.1, varnish-agent 3.0.1, 4.0.1, 4.1.0
Varnish Software AS, 22.06.2016
Page 1
Table of Contents
1 Introduction
5
2 Installation
6
2.1 Supported platforms
6
2.2 Overview
6
2.3 Ports required
6
2.4 Varnish Administration Console via Varnish Plus
7
2.4.1 Debian / Ubuntu install
7
2.4.2 Red Hat Enterprise Linux install
8
2.4.3 Installation on other platforms
9
2.5 Configuring Varnish Administration Console
9
2.5.1 Log4j
9
2.5.2 RRD
9
2.5.3 Adding a new RRD archive
9
2.5.4 Memory/Heap
10
2.6 Supported counters in Varnish Administration Console
11
2.7 Graphs
11
2.8 Installing varnish-agent on Varnish Cache servers
11
2.8.1 Debian / Ubuntu install
11
2.8.2 Red Hat Enterprise Linux install
12
2.8.3 Installation on other platforms
12
2.9 Configuring the varnish-agent
12
3 Getting started with the VAC
14
3.1 Concepts used in the VAC
14
3.2 Logging in
14
3.2.1 Upgrading from previous versions
14
4 Super Fast Purger API
15
5 Maintaining the VAC
15
5.1 Backup
15
5.2 Upgrade
16
6 Troubleshooting
17
6.1 Disabling firewall in RHEL
17
6.2 Mongo dump and restore
17
6.3 Client browser requirements
17
6.4 Getting help
17
6.5 General FAQs about using VAC
17
Page 2
Released with vac version 3.2.1 and varnish-agent version 3.0.1, 4.0.1, 4.1.0, document version 110.
Copyright 2011 - 2016 Varnish Software AS (http://www.varnish-software.com)
Page 3
Conventions used:
• The '''server_hostname''' string refers to the DNS name pointing at the vac server.
• Commands that are to be run as user root are prefixed with # (hash) like in the shell:
# apt-get install sl
1 Introduction
The Varnish Administration Console (vac) is a management console for groups of Varnish Cache servers.
It communicates with the Varnish Cache server(s) through the varnish-agent.
The vac is distributed as Debian/Ubuntu deb packages, Red Hat RPMs and a binary ZIP archive. It is
recommended to use the Debian or RPM packages.
The vac has modest hardware requirements. A virtual machine with 1GB RAM and 10 GB disk space is
sufficient for ordinary installations.
Note that vac 3.0.0 and above will require varnish-agent 2.2.0 and above.
** For Varnish 3.x installations the Vac will require varnish-agent 2.2.x or 3.x.x. **
** For Varnish 4.0.x installations the Vac will require varnish-agent 4.0.x and lower than 4.1.x **
** For Varnish 4.1.x installations the Vac will require varnish-agent 4.1.x or above **
** From version 3.0.3 and above the VAC requires Java Runtime Environment version 7 or above **
Earlier versions of the vac, such as 2.0.x, is dependent on varnish-agent 1.16.
Page 5
2 Installation
2.1 Supported platforms
We provide vac and varnish-agent packages for the following platforms:
• RedHat Enterprise Linux 6
• RedHat Enterprise Linux 7
• Ubuntu Linux 12.04 LTS (precise)
• Ubuntu Linux 14.04 LTS (trusty)
• Debian Linux 7 (wheezy)
• Debian Linux 8 (jessie)
2.2 Overview
Installing the vac involves:
• Installing the vac on a server.
• Installing varnish-agent on Varnish Cache servers.
• Configuring the varnish-agent.
• Configuring the vac.
To perform a successful installation you need:
• Access to Varnish Plus repository.
• A vac license file which can be obtained from support@varnish-software.com.
• A Varnish Cache server that an varnish-agent can control.
After a successful installation the vac will be listening on port 80 on the server. varnish-agent will listen on
port 6085 on the Varnish Cache server(s).
It is recommended that you first install the vac and varnish-agent on a test system before deploying or
upgrading it in production. See Troubleshooting below for common mongodb operations to backup and
restore.
2.3 Ports required
The communication between the VAC, the Varnish Agent and Varnish Cache server(s) requires a few
ports to be open. The port that need to be open on the Varnish Cache server(s) are:
• Port 80, open for the world for HTTP access to Varnish Cache
• Port 6085, for varnish-agent by default, open from the server where the VAC is installed
• Port 80 on the vac server from the management network, to access the vac
varnish-agent utilises the Varnish API internally for fetching and issuing commands to and from Varnish
Cache.
All ports are configurable.
Page 6
2.4 Varnish Administration Console via Varnish Plus
In order to install vac on either Debian/Ubuntu or Red Hat Enterprise, one would require access to the
Varnish Plus Software repository. With your welcome letter you should have gotten a document describing
how to set up the repository. Once the repository is set up you can follow the instruction below on how to
install the VAC.
Please get in touch via support@varnish-software.com for more information on Varnish Plus or if you have
any questions on how to set up the repository.
2.4.1 Debian / Ubuntu install
If you are installing on Debian or Ubuntu, use the prebuilt packages as per Varnish Plus VAC depends on
MongoDB to install.
2.4.1.1 MongoDB installation
To install MongoDB Community Edition on Debian/Ubuntu:
• Import the public key used by the package management system:
# sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927
• Create a /etc/apt/sources.list.d/mongodb-org-3.2.list file for MongoDB:
# echo "deb http://repo.mongodb.org/apt/debian wheezy/mongodb-org/3.2 main" |
sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list
• Reload local package database:
# sudo apt-get update
• Install the MongoDB packages:
# sudo apt-get install -y mongodb-org
For more reference on the MongoDB installation check
https://docs.mongodb.org/manual/tutorial/install-mongodb-on-debian/
2.4.1.2 Installing other dependencies
• You also need to run the following command to install the remaining dependencies:
# sudo apt-get install openjdk-7-jre openjdk-7-jdk unzip adduser ntp -y
2.4.1.3 VAC package installation
After installing MongoDB and the other dependencies you can proceed and install the VAC package:
# sudo apt-get install vac
# sudo update-rc.d mongod defaults
# sudo update-rc.d vac defaults
vac is now installed and you can now access it on http://server_hostname/. Proceed to install
varnish-agent on the cache server.
Page 7
2.4.2 Red Hat Enterprise Linux install
To install the vac on Red Hat Enterprise Linux or derivatives you need to install MongoDB and other
dependencies:
2.4.2.1 MongoDB installation
• Configure the package management system (yum).
Create a /etc/yum.repos.d/mongodb-org-3.2.repo file so that you can install MongoDB directly,
using yum.
Use the following repository file for the latest stable release of MongoDB:
[mongodb-org-3.2]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.2/x86_64/
gpgcheck=0
enabled=1
• Install the MongoDB packages and associated tools:
# sudo yum install -y mongodb-org
• Start MongoDB:
# sudo /etc/init.d/mongod restart
• Make sure MongoDB and start at boot with:
# sudo chkconfig mongod on
For more reference on the MongoDB installation check
• Add the MongoDB repository described below. Note that manual installation of MongoDB is not
required.
http://www.mongodb.org/display/DOCS/CentOS+and+Fedora+Packages
2.4.2.2 Installing other dependencies
• You also need to run the following command to install the remaining dependencies:
# sudo yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ntp -y
2.4.2.3 VAC package installation
• Add the vac Yum repository as per Varnish Plus instructions
• Install the vac:
# sudo yum update
# sudo yum install vac
• Make sure MongoDB and the vac start at boot with:
Page 8
# sudo chkconfig mongod on
# sudo chkconfig vac on
The vac software is now installed and you can now access it on http://server_hostname/. Proceed to
install the varnish-agent on the cache server.
2.4.3 Installation on other platforms
To install the vac unpack the distribution ZIP file. You can download it from the official distribution source
at repo.varnish-software.com . Contact support@varnish-software.com for further assistance.
Ensure that you have a valid
support@varnish-software.com.
license
file.
License
files
are
provided
by
contacting
• To install:
# bash install install
• To start, stop or restart the vac:
# /etc/init.d/vac start | stop | restart
• To uninstall:
# bash install uninstall
2.5 Configuring Varnish Administration Console
vac does not usually need to be configured. However, the main configuration files are
/opt/vac/etc/defaults, /opt/vac/etc/log4j.xml, and /opt/vac/etc/rrd_conf.xml.
2.5.1 Log4j
The log levels in vac can be tweaked in log4j.xml configuration file. The usual level of logging are TRACE,
DEBUG, INFO, WARN, ERROR or FATAL.
The namespace used by vac is prefixed with com.varnish. The log4j.xml file ship with a console appender
and a rolling file appender. By default the rolling file appender is used.
2.5.2 RRD
The rrd_conf.xml is used to configure the archive and datasource used in vac. A single cache in vac has
an associated RRD. This RRD is stored in memory based on the specified datasource and archives.
Similar to rrdtools, once the RRD is created for a single cache, its definition cannot be dynamically
changed. Therefore RRD changes are only applied to Varnish Cache servers added to vac after said
change.
2.5.3 Adding a new RRD archive
After you edit the rrd_config.xml file with the appropriate archives, you need to make vac aware of the
updated file in roder to make it choose the best exisiting archive to query. In order to do that you need to
follow the following steps:
• Delete all nodes from the vac - in order to trigger the deletion of all existing rrd config in mongo.
• Restart the vac - in order to pick up the new changes in the rrd_config.xml file
Page 9
• Restart the agents and make the node registration - in order for the nodes to re-register
themselves with the
new rrd configuration known by the vac.
2.5.4 Memory/Heap
A memory requirement of 2GB dedicated to vac is recommended. This is a very optimistic allocation as
the real consumer of memory in vac is the real-time RRD component. A rule of thumb would be to make
available 200MB ~ 300MB per cache for the RRD.
Out of the box, vac starts with the following:
max_heap_size=2048m
initial_heap_size=512m
# jvm max heap size
# jvm initial heap size
It is therefore recommended to monitor memory usage and ensure that enough memory is made available
for vac as the number of cache being managed by it increases.
See the configuration file, /opt/vac/etc/defaults, for more detail.
Page 10
2.6 Supported counters in Varnish Administration Console
Graph name
Counters
Hit Rate %
cache_hit, cache_miss, cache_hitpass
Request per second
cache_req
Hitrate
cache_hit, cache_miss, cache_hitpass
Hit vs Miss vs Hit pass
cache_hit, cache_miss, cache_hitpass, client_req
Request vs Connection
client_req, client_conn
Bandwitdh
s_hdrbytes, s_bodybytes
Bad things
backend_fail, n_wrk_drop, client_drop, n_wrk_queued,
n_lru_nuked
2.7 Graphs
With the help of varnish-agent, a snapshot of Varnishstat is pushed to VAC and aggregated in memory.
The end result is a time series for each key counter for each Varnish instance. The Real-time Statistic API
supports queries based on time range in seconds past epoch, and number of samples based on current
time. The API also supports the usual consolidation function, average, min, max, last, first, and total. vac
is a natural datasource for providing statistics to any existing monitoring tool.
The graphs provided in vac UI are in two forms of realtime and historical. The realtime graph indicates
time series data across a group as based on timestamp and usual consolidation function. The historical
graph represents timeseries data from and to specific ending time point across a cache group. The
counters of interest, and the sampling resolution and window are completely configurable within vac.
2.8 Installing varnish-agent on Varnish Cache servers
It is a requirement that varnish-agent to be installed on the same server as Varnish Cache itself. This is
mandatory for establishing the communication between vac and Varnish Cache through the RESTful
interface provided by varnish-agent.
varnish-agent utilises both Varnish CLI and Varnish API directly as the key mechanism for accessing the
internals of Varnish Cache. Therefore it is recommended that the following parameters are set on Varnish
to mitigate large amounts of redundant CLI traffic being written to syslog. See below:
syslog_cli_traffic=off
The aforementioned parameters can be changed through vac or via Varnish startup script.
The pre-made packages for the varnish-agent are available in the standard Varnish Cache repositories.
Please note that these repositories also includes Varnish Cache packages, something which may affect
(upgrade) older installed Varnish packages on the system.
2.8.1 Debian / Ubuntu install
varnish-agent is prepackaged in the package repository at www.varnish-cache.org.
Instructions on how to enable this repository is available online:
https://www.varnish-cache.org/installation/debian
After this repository has been added, install the varnish-agent by running:
Page 11
apt-get update
apt-get install varnish-agent
update-rc.d varnish-agent defaults
varnish-agent is now installed and can be configured.
2.8.2 Red Hat Enterprise Linux install
varnish-agent is prepackaged in the package repository on www.varnish-cache.org. The packages are
built for RHEL6, but also work without problems on newer releases.
Instructions on how to enable this repository is available online:
https://www.varnish-cache.org/installation/redhat
Please note that varnish-agent has dependencies in the EPEL repository.
For RHEL6:
# wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
# rpm -Uvh epel-release-6-8.noarch.rpm
For RHEL7:
# wget https://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm
# rpm -Uvh epel-release-7-5.noarch.rpm
After enabling both the Varnish Cache repository and the EPEL repository, the varnish-agent can be
installed with:
# yum install varnish-agent
varnish-agent is now installed and can be configured as described below.
2.8.3 Installation on other platforms
varnish-agent is an open-source C application. It is possible to install this on most systems compatible
with Varnish Cache.
Download the source distribution from:
http://github.com/varnish/vagent2
2.9 Configuring the varnish-agent
varnish-agent requires minimal configuration except to specify the callback URL of vac. The goal for the
callback is for varnish-agent to provide periodic varnishstat to the vac.
There are a varnish-agent secret file that is randomly generated upon installation. This file resides in:
/etc/varnish/agent_secret
Secondly, varnish-agent start-up parameters are loaded in the default configuration file. For
Debian/Ubuntu:
/etc/default/varnish-agent
Page 12
For RHEL:
/etc/sysconfig/varnish-agent
For varnish-agent to successfully register itself to vac, a callback URL is needed in DAEMON_OPT option
in the default configuration file. For example:
DAEMON_OPTS="-z http://vac_server/api/rest/register"
You can also use a hostname for the VAC callback URL.
When is this useful?
• You can use a domain name if the IP address is not reliable
• You can use a domain name or an IP address if the VAC is behind a proxy
For that you will need to pass a query parameter:
DAEMON_OPTS="-z http://vac_server/api/rest/register?hostname=agent_hostname"
After applying changes in configuration file restart is needed afterwards. Check varnish-agent man pages
for further configuration details. Note that if Varnish Cache secret file can only be read by root, then
varnish-agent is required to run as root. This can be achieved by specifying the -u root argument.
Outputs from varnish-agent is written to syslog.
When the vagent can not be reached the VAC will constantly log about it. If you do not want the agent to
log when the it is down, there is a flag agent_log in in var/opt/vac/log which is true by default. By setting it
to false we just get the state of the agent changes from up to down.
Page 13
3 Getting started with the VAC
3.1 Concepts used in the VAC
A cache group is a collection of Varnish Cache servers that have identical configuration. Attributes on a
cache group includes:
• One or more Varnish Cache servers
• Active Varnish Configuration Language (VCL) file
• Homogeneous parameter configuration across all servers in a group
A VCL file is a configuration file which describes control flow when Varnish Cache handles a HTTP
request. vac distributes and store VCL files for you.
A parameter set is a list of Varnish cache parameters. A parameter set can be applied to one or more
cache groups simultaneously, as long as the cache groups all consist of cache servers of the same
Varnish Cache version.
vac ships with a JSON-based RESTful API. All actions performed via the user interface can be replicated
with direct access to the API. This includes fetching all real-time graph data.
3.2 Logging in
The default user is "vac" and the default password is "vac". Please change the login password the first
time you log in.
You can access vac on http://server_hostname/ .
3.2.1 Upgrading from previous versions
Upgrading from previous versions should adhere to the following steps:
1.
2.
3.
4.
5.
Stop and upgrade |agent| individually
Stop and upgrade |vac|. Note the listening IP and port
Ensuring that |vac| and |agent| are not running, configure |agent| as specified above
Restart |vac|
Restart |agent| individually
Note that VCL names in 3.0.0 must now adhere to VCL naming syntax as imposed by Varnish Cache.
Therefore previous VCL name is prepended with "vac_" during the upgrade process.
In versions lower than 3.1.0, user roles where not supported by VAC. When upgrading to 3.1.0, existing
users inherit, by default, the Admin role. This role can be updated later. As in 3.1.0 version there are two
roles defined for VAC, "Admin" and "ReadOnly".
Page 14
4 Super Fast Purger API
The Super Fast Purger API is separate from the VAC API for the purpose of high performance purging. It
listens on a separate port (8088 by default,can be changed using the jdi_listening_port parameter)
and is not part of the usual servlet life cycle.
The SFP is actually a HTTP replicator that will fan out the messages you send to it to all the machine in a
cluster. Its first use is purging content from caches but it has other uses, like warming the cache of a
whole cluster.
To purge using SFP, you must first have a VCL that handles it (see the installation-guide for more
information). If it's the case you can simply issue the purge request to SFP instead of each individual
node.
To know what nodes should receive the request, SFP will read the "X-Cache-Group" header, so be sure
to include it in your request. For example, with curl:
curl sfpIP:8088/foo -X PURGE -H "X-Cache-Group: group1"
Be careful though, by default, the VCL will add the host to the hash key of the object, so it should match
the one use by client to retrieve content. Curl will add the host header if none is specified, so you probably
need to override it:
curl sfpIP:8088/foo -X PURGE -H "X-Cache-Group: group1" -H "host: you.com"
The Super Fast Purger will reply and concatenate the headers received from the nodes in its own
headers, as weel as some metadata, like so:
TODO example to be added once response formatting is sorted out.
If everything goes well, you should expect a status code 200, meaning all the nodes answer "OK". If not,
you should look at the collected headers to find out what went wrong. SFP will add a "X-Bad-Caches"
headers listing the nodes that failed. The following error codes are used:
• 200: successful purge
• 401: failed authentication TODO: the SFP requests are authenticated? How?
• 404: group not found
• 500: "VCL does not support purge" TODO: how do we know that
• 500: "fail on retries" TODO How do we distinguish two 500?
• 504: slow varnish
5 Maintaining the VAC
5.1 Backup
vac stores state in MongoDB, /var/opt/vac/ and /opt/vac/.
It it safe to back up the files on disk while vac is running. Use one of the methods described in the
MongoDB documentation to back up the database.
http://www.mongodb.org/display/DOCS/Backups
Page 15
5.2 Upgrade
The preferred method of upgrading is using the Debian / Red Hat packages.
Upgrading a manual install includes stopping the vac and re-running the installation script. Please see the
install section for detailed steps.
It is not necessary to do any manual database migration steps, or change anything on the Varnish Cache
server(s).
Page 16
6 Troubleshooting
For logging of exceptions or errors in vac, please review the log files when submitting support cases:
/var/opt/vac/log
The log files are rotated by vac init script on restart, and in normal operations the log volume is low.
6.1 Disabling firewall in RHEL
This is not recommended by default. However it could simplify installation a little.
• Disable the host-based firewall enabled by default:
# service iptables stop; service ip6tables stop
# chkconfig iptables off; chkconfig ip6tables off
6.2 Mongo dump and restore
vac stores all data to mongodb, and by default, it uses the db name vcc.
For example, to create a full mongo dump of vac's db:
mongodump -d vcc
To restore vac's db:
mongorestore -d vcc --dir PATH_TO_THE_DUMP_DIR
6.3 Client browser requirements
All recent versions of Firefox and Chrome work without problems.
6.4 Getting help
All VAC enquiries can be directed to support@varnish-software.com.
6.5 General FAQs about using VAC
1. I have a licensed VAC installed and it is running but I do not see any stats. Is some other
configuration needed?
In order for the VAC to work you need to install the Varnish agent in each instance of Varnish you want to
connect to the VAC. VAC is typically installed on a separate server to varnishd/varnish-agent; to register
the varnish-agent with VAC, the -z flag needs to be set either on the command line or in
/etc/defaults/varnish-agent, in the form
DAEMON_OPTS="-z http://vac-server-name/api/rest/register"
If this is successful, the varnish-cache instance should be listed in the VAC UI under the Configure tab. To
associate the cache with a cache group and start using it, drag and drop the instance from the list on the
left-hand side to the group entry.
2. Does VAC has support for Varnish 4.x?
Page 17
Since 3.0.3 the VAC supports both Varnish 3.x and 4.x. A Varnish agent is needed to match the Varnish
version. So for varnish 4.x the there is a Varnish agent 4.x The only thing they should be aware is that you
cannot group different versions of varnish in the same group.
3. All the cache nodes are shown by the IP only. How can we configure agent to return an instance
ID to the VAC?
If you are running a single instance per server and you are not specifying the instance name to varnishd
using the -n parameter (default behaviour), adding -n <hostname> to the DAEMON_OPTS parameter in
/etc/sysconfig/varnish-agent will add the hostname to the VAC UI. If you are specifying the instance name
to varnishd you will need to use the same value for the agent.
4. Can we specify a ban on a single cache through the web ui or api? Or we should create a group
first, add the cache and then apply the ban?
To specify a ban via the VAC or the API you need to have a group. You could, however, apply a ban to a
single cache accessing the Varnish CLI via the VAC. To do this go to the Configure tab and click on the
instance. This will take you to the CLI. After you are done type quit to close the session.
5. Is it also possible to manage both VCP3 and VCP4 instances from that same VAC environment?
It is possible to manage both as long the correct agent version is installed in each Varnish version. Please
note that different Varnish versions can not be deployed to the same VAC group.
6. What is the relationship between the VCLs on disk on the Varnish servers, and the VCL in the
VAC console?
In a Varnish server all the VCL's are kept in the agent, if you go to the agent and you issue vcl.list you can
see that one is active and other(s) there. VAC connect to the agents and makes sure that there is
coherence between Varnish instances that belong to the same group of caches. It is important to change
Varnish to use boot.vcl instead of default.vcl by default. So if the Varnish instance reboots we will have a
valid vcl that works for you. You can find the boot.vcl at: /var/lib/varnish-agent
7. If I start the Varnish servers with their default VCLs, can I display this in VAC?
At this time the VAC doesn't display the default VCL(boot.vcl) . What the VAC can do is to make sure that
the VCL in the VAC in a specific group will be active in the Varnish instance. So if the instance is restarted
the VAC will make sure the active VCL in the VAC will be deployed to all the Varnish instances that
belong to that group.
8. What is correlation between the CLI and the VAC GUI with VCL files?
Once the VAC is deployed and you started managing your VCL files with it, the VAC takes over the CLI. If
you do changes in the CLI they won't be seen by the VAC and vice versa. The only exception is when you
deploy a VCL from the VAC (as you would expect). There is a 1-to-1 correlation between Varnish
instances and agents - if you want to run multiple instances you will need one agent per instance. With
that said, what's the reason to have multiple instances? Varnish can handle several websites with just one
without issues.
9. How do I setup VAC with HTTPS?
The VAC supports HTTPS when used together with an SSL terminator. If VAC is configured with an SSL
terminator in front, it is important that this SSL terminator sets the X-FORWARDED-PROTO request
header properly. VAC will use this header to auto configure the communications channel from the
varnish-agent to the VAC. You can read more about the X-FORWARDED-PROTO header in rfc7239
[http://tools.ietf.org/html/rfc7239#page-7]
The following is an example of the X-Forwarded-Proto setup with nginx
server {
listen 443;
ssl on;
Page 18
# Certificate configuration goes here
location / {
proxy_pass http://localhost:8080;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
}
}
10. How the JVM looks when you try to run the vac?
In order to provide that log, in vac default file set jvmLogEnabled=true, restart the vac and you can check
the vac-stdout.log and vac-stderror.log from this path: /var/opt/vac/log. Please note that jvmLogEnabled is
set to false by default. The JVM log tends to grow in size quit fast so it is recommended to keep it to false
unless you would like to debug an issue in JVM.
Page 19