Toad® says RAC’em Up!
When someone asks if you’d like to shoot a game of pool (i.e. billiards), a common affirmative reply is
“Rack’em Up!” That means yes, you’re ready to play so setup the balls for a new game. The same can
now be said regarding whether or not to deploy Oracle databases using Real Application Cluster (RAC)
technology. When equipped with the “Toad® DBA Suite for Oracle RAC” bundle it’s now so easy to
manage, diagnose and optimize Oracle clustered databases that the answer becomes “RAC’em Up!”
Let’s begin by reviewing what RAC is. Figure 1 shows an example of deploying a 2 node RAC database
utilizing an iSCSI file server as your shared storage mechanism – a key RAC requirement. In simple terms,
RAC is merely a collection of “coordinated” instances that collectively appear to the application or user
as a single, large database. Some people deploy RAC databases for fault tolerance, meaning that the
database remains “up” as long as any of the nodes are working. Others deploy RAC for scalability, such
that the application or user load can be “spread” or “load balanced” across multiple database servers.
There have been numerous debates about which usage is more correct. But here let’s simply assume
that for whatever legitimate business or technical reason that you’ve chosen to deploy a RAC database
and now require the use of tools that are RAC “aware”.
Figure 1: Example Two Node RAC
Refer back to Figure 1 once again. Any RAC database regardless of node and instance counts possess a
much more complex process and memory architecture, with lots more “moving parts”, than the more
traditional singular Oracle instance. This fact is critical for two reasons. First, many database tools were
not designed to work against this model. For example Quest offers Spotlight for Oracle which can work
with each of the RAC instances individually, but not the cluster mechanisms nor the clustered database
as a whole. For that you need Quest’s Spotlight for Oracle RAC – a very highly specialized version of the
product that understands and works with this more advanced clustered architecture, and is part of the
“Toad® DBA Suite for Oracle RAC” bundle. The same holds true for other vendors’ database tools as
well – when working with RAC you really need the RAC “aware” versions of their tools. Second, I have
found that many initial or “proof of concept” (i.e. POC) RAC projects fail due to insufficient planning and
preparation. The DBA who’s new to RAC and without RAC “aware” tools is at a huge disadvantage. It’s
quite easy to experience or even to cause critical mistakes when using the wrong tool – try pounding a
nail with a wrench instead a hammer!
So let’s examine how the right RAC aware tools such as Toad and Spotlight can ease that pain. Notice
how Spotlight for Oracle RAC’s dashboard shown in Figure 2 looks similar to the basic RAC architecture
diagram from Figure 1. Figure 2 cleanly and clearly shows that there are two database instances (middle
of the screen), the private network or interconnect (top of the screen) and the shared IO subsystem
(bottom of the screen). Moreover, the dashboard’s simple color coding instantly communicates the
status of the overall cluster and all of its nodes. It is far more than just RAC aware – it is RAC friendly.
Figure 2: Spotlight for Oracle RAC
Can one work effectively with a RAC database without RAC aware tools – maybe. But why do things the
inefficient (e.g. costly) and much harder way when there are tools for such needs. While a RAC literate
and experienced DBA with many RAC databases under their belt might be able to do without such tools,
what about the rest of us who can make good use of such tools to work both effectively and efficiently?
And isn’t that the true mark of any good worker – work smarter by using tools rather than brute force?
Toad Connecting to RAC
In order to work with any database, one must connect to it – RAC databases are no different in that
basic respect. But remember the RAC architecture from Figure 1 with its “more moving parts”? That
picture and its parts also complicates the normally rather simple process of establishing a connection.
When using Toad’s login or connection screen with a RAC database there are two options: specifying the
specific instance (e.g. RAC1 or RAC2) or the generic, load balancing instance (e.g. RAC) as shown here in
Figure 3. In most cases you’ll want to select the generic instance (e.g. RAC). The only time you really
should use the specific instance (e.g. RAC1 or RAC2) is when you’re going to perform a DBA task that
absolutely requires your session on a specific instance. For example when adding a redo log file to an
instance, you have to connect to that specific instance. You need to know when to use which method.
Figure 3: Toad Connection Screen
A key requirement when connecting to a RAC database is being able to point to and connect with the
proper moving parts from Figure 1. That requires a syntactically correct and fairly complex” SQL*Net”
client file for database alias names (i.e. TNSNAMES.ORA) – which hopefully your DBA will provide. But
let’s examine how the RAC database TNSANMES.ORA file entries shown here in Figure 4 relate back to
Figure 1’s architecture diagram and Figure 3’s database login or connection screen. This is important.
RAC =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip)(PORT = 1521))
(LOAD_BALANCE = YES)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = RAC)
(FAILOVER_MODE =
(TYPE = SELECT)
(METHOD = BASIC)
(RETRIES = 180)
(DELAY = 5)
)
)
)
RAC1 =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = rac1)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = RAC)
(INSTANCE_NAME = RAC)
)
)
RAC2 =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = rac2)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = RAC)
(INSTANCE_NAME = RAC)
)
)
Figure 4: Example RAC TNSNAMES.ORA Entries
When the generic alias (i.e. RAC) is selected, the database listeners on the servers will connect you to
either the RAC instance on node #1 or node #2 depending on each node’s current work load since the
load balancing parameter is set. Note too that the generic alias (i.e. RAC) uses the virtual IP addresses.
The listener also provides fault tolerance by being smart enough to detect down nodes and reassign
their virtual IP addresses the same value as one that’s till up. Thus the generic alias (i.e. RAC) provides at
least two levels of abstraction or possible misdirection – as you can’t really be sure in advance on which
node it will select or where that node’s virtual IP address currently points. But for 95% of the database
tasks one is likely to perform, that’s quite all right. However when you really do need to perform a node
specific database administrative task, then the specific aliases (e.g. RAC1 and RAC2) point to the proper
node using the physical or real IP address. Once again, when in doubt use the generic alias – only use the
node specific aliases when you know why what you’re doing requires it.
Once you’ve successfully connected Toad to your RAC database, it will display both the alias name you
used during connection and the actual database instance number in its “Connection Toolbar” as shown
here in Figure 5. Therefore this session used the generic alias (i.e. RAC) and is currently connected to the
second database instance in the cluster.
Figure 5: Toad Connected to RAC
Toad’s RAC Enabled Screens
Where necessary, Toad’s screens and utilities have been “RAC enabled” or made “RAC aware”. But the
nature of those enablement or awareness depends upon the context of the screen or utility as to how it
behaves or works. Let’s examine a few examples that demonstrate that variety and should provide you
with sufficient familiarity such that you can master all of Toad’s RAC support.
Let’s start by looking at the Toad screen for Oracle configuration parameters – often referred to as the
init.ora or spfile parameters. These parameters are the basic configuration settings for each of the RAC
instances – so now this screen needs to be able to show multiple sets of values. In Figure 6 we see that it
can be done via tabs – the famous trademark design that Toad made popular and is so famous for. Thus
here we see that there are simply additional tabs across the top for each instance in the RAC setup. But
this screen style does not make it very easy to see and compare values across the nodes. It only makes
viewing a particular node’s values straightforward.
Figure 6: Toad init.ora Parameter Screen
Thus as shown here in Figure 7, we can change the display type to “Single Grid” which now simply lists
all the values sorted by parameter name and then by RAC node. Now it’s very easy to compare the same
settings across nodes when looking for any that might improperly vary. Of course now you may to scroll
more to bring things into views as your effectively multiplying the grid row counbt by the number of RAC
instances. Don’t forget to use the “Filter” text box to focus more easily on those parameters of interest.
Figure 7: Toad Instance Parameter Screen
Next let’s examine the Toad Redo Log Frequency Map screen. Remember that each RAC instance has its
own redo log files – that’s by definition of the RAC architecture. The RAC nodes must share the data and
control files, but each RAC instance must possess its own redo log files. That’s just the way it is. So how
does this Toad screen handle RAC? Figure 8 demonstrates how some Toad screens offer you an instance
selection drop down box when Toad knows that you’re connected to a RAC database (otherwise these
are hidden). Note also that in this case you can specify that you want to see the cumulative results for all
nodes in the cluster via the “ALL” selection.
Figure 8: Toad Log Switch Frequency Map Screen
Note too however as shown in Figure 9 below that not all Toad screens using this style offer the “ALL”
option. In the case of the Database Monitor screen it really doesn’t make too much sense to show the
cumulative values that one can monitor. For example knowing something like the total cumulative SGA
memory consumed by all the nodes across the RAC database is just not that compelling or useful a fact
to display. So Toad offers two different drop downs on screens using this method, either listing each
RAC instance and all of them cumulative or just each of the RAC instances without offering them all.
Figure 9: Toad Database Monitor Screen
Of course all these prior screen examples have been fairly simple – now it’s time to look at some of the
more complex examples. Figure 10 shows the Database Browser screen which offers numerous tabs that
behave differently. In some cases, such as the “Sessions” tab, Toad merely adds a column for the RAC
node instance ID (i.e. Inst_ID). That makes it very easy to sort by node and thus see the whole cluster
wide status of the metric in focus. While other tabs such as “Space Usage” and “Datafile IO” do not
show anything extra since both display information that’s database wide and not instance specific.
Figure 10: Toad Database Browser – Sessions
Finally there are some tabs such as “Instance” and “Database” that offer tabs along the bottom which
permit either the single record view or traditional grid style display of the RAC nodes’ information.
Figure 11: Toad Database Browser - Instance
The final screen worth examining is the “Session Browser”, arguably one of the most useful screens for
many DBA’s. One often needs to see who’s connected to the database and what their session is up to.
Remembering the basic architecture from Figure 1, we know that a user’s session must exist on a node.
But we also know that the users’ sessions can seem non-stationary due to load balancing and/or fault
tolerance. Thus you’ll see here in Figure 12 that the RAC node instance ID (i.e. Inst_ID) shows up as a
new column in both the top portion of the screen and optionally on some of the tabs in the bottom.
Furthermore, note that the bottom portion of the screen’s tabs offer the “Multi” and “Single” options
similar to Figure 11 – but now they are in reverse order.
Figure 12: Toad Session Browser Screen
Finally if your single instance and/or RAC databases are using Oracle’s Automated Storage Manager (i.e.
ASM), the n Toad offers the ASM manager screen. And while this screen has a drop down that seemingly
offers choices similar to prior examples, it is in fact quite different. When working with ASM storage you
can see some information via the database connection. But for more detailed information and for being
able to issue some commands you must instead connect directly to the ASM instance. That’s why you
see in Figure 13 just the two choices for “Database Instance” and “ASM Instance”. In this case there is
no need to offer drop down choices for each node in the RAC setup, but rather just use you existing
database connection or one specific to the ASM instance.
Figure 13: Toad ASM Manager Screen
Conclusion
We’ve now seen that the “Toad® DBA Suite for Oracle RAC” bundle offers numerous places where Toad
is both “RAC aware” and “RAC enabled”. Of course not every screen needs these additional options, and
based upon the context of the screen their display style and options can be different. But Toad does do
RAC and does it nicely. In fact with Toad’s DBA screens and Spotlight for Oracle RAC there is literally no
management task or problem detection that cannot be performed. Thus armed with such “proper” tools
most any DBA can tackle a RAC database. Since we know that the RAC architecture is more complex as it
has more “moving parts”, utilizing the right tools such as these can make a huge difference for most any
DBA. You can hit the ground running and appear like a super star even if this is your first RAC database!
Author Bio
Bert Scalzo is a Database Expert for Quest Software and a member of the TOAD dev team. He
has worked with Oracle databases for well over two decades. Mr. Scalzo’s work history includes time at
both Oracle Education and Oracle Consulting. He holds several Oracle Masters certifications and an
extensive academic background - including a BS, MS and PhD in Computer Science, an MBA, plus
insurance industry designations. Mr. Scalzo is also an Oracle ACE.
Mr. Scalzo is accomplished speaker and has presented at numerous Oracle conferences and user groups
- including OOW, ODTUG, IOUG, OAUG, RMOUG and many others. His key areas of DBA interest are
Data Modeling, Database Benchmarking, Database Tuning & Optimization, "Star Schema" Data
Warehouses, Linux and VMware.
Mr. Scalzo has written numerous articles, papers and blogs - including for the Oracle Technology
Network (OTN), Oracle Magazine, Oracle Informant, PC Week (eWeek), Dell Power Solutions Magazine,
The LINUX Journal, LINUX.com, Oracle FAQ, Ask Toad and Toad World.
Mr. Scalzo has also written six books: Oracle DBA Guide to Data Warehousing and Star Schemas, TOAD
Handbook, TOAD Pocket Reference (2nd Ed),Database Benchmarking: Practical Methods for Oracle &
SQL Server, Advanced Oracle Utilities: The Definitive Reference, Oracle on VMware: Expert Tips for
Database Virtualization, and Introduction to Oracle: Basic Skills for Any Oracle User.