SQL Query Tracing for RQM Developers
Overview
How you trace into SQL depends on what you are trying to discover.
If you would like to see all the SQL associated with a single RQM service call, the best option is
using log4j. With that, you can filter the output for a single service call made from the web UI.
If you are looking at all the SQL associated with a scenario, repodebug can be very useful. Once
you have a collection of SQL statements, you can focus on the ones that are most expensive (or
repeated a lot) from your scenario. You can then turn on stack tracing, and determine where the
SQL statement was called in your RQM code.
If you have RQM code that makes query and you want to find out what the SQL that is generated
is, you can put a breakpoint and see the specific value.
Using log4j
Turning this on will cause the console output from your RQM repository to spew massive
amounts of information. I find the best trick is to enable it, then quickly run the scenario, and
then turn it off again. There are 2 different ways to turn it on dynamically while the server is
running.
Enable log4j using jazz/admin
If you go to the URL http://host:port/jazz/admin?internal=true you will find under “Reload Log
Settings” under “Internal Tools” towards the bottom of the left hand side. If you click on “Reload
Log Settings” you will see a single button “Reload Log Settings”. Pressing this will cause the
server to re-load the log4j.properties file.
In the log4j.properties file, you should add these lines if they do not already exist:
# Turn on debugging of all SQL
log4j.logger.sqlTxLogger=DEBUG
Save the file, clear the console window, and then push the “Reload Log Settings” button. Massive
amounts of output will begin immediately. You should probably turn off the limit on the console
size before you do this, or some of the output could be lost. You can do this by using the mouse
to right-click in the console window, and then from the preferences section disable the “Limit
console output” checkbox.
I recommend that you have your web UI scenario queued up before you push the button. What I
actually do is change the setting to DEBUG and save the file, and then in the editor I change the
setting back to ERROR but do not save.
log4j.logger.sqlTxLogger=ERROR
Then I press the button, run my scenario quickly, hit save in the log4j.properties editor window,
and then hit the button one last time. After this, I copy and save the full SQL trace for editing and
analysis.
Enable log4j tracing using repodebug
If you go to the URL such as
http://host:port/jazz/repodebug
then click on
log->loggers
at the bottom you see the sqlTxLogger. Click on that, and you see the current setting for that
logger (presumably ERROR). Click on “editLevel”, and you can change the level to DEBUG. Again,
once you hit the Submit button, the logging will start, so again, be quick about running your
scenario and resetting the logger to ERROR.
Analyzing the SQL trace output
Several of the asynchronous tasks use a lot of SQL, which is why your output will have extra SQL
that is not explicitly from your scenario.
I typically will trim the trace output using something like:
grep "sql =>" trace.txt >sql.txt
Then, I edit the sql.txt file and look for the specific scenario, skipping over the asynchronous task
output. For example, a line of trace output for a “browse test case scenario” might look like:
17:26:14,588 [1071268113@qtp-1954570098-49 @@ 17:26 TestJazzAdmin1
<com.ibm.rqm.planning.home.actionDispatcher/viewTestCases@2b4ee1cc-35dc-475e-887204ed3bcdb39f>
/jazz/service/com.ibm.rqm.web.common.service.rest.ICompositeWebRestService/browseInitDa
ta] DEBUG sqlTxLogger
- ConnectionStatWrapper(type 4, 22737, dirty: true,
b(15919491)) sql => select t1.ITEM_ID, t1.STATE_ID from PLANNING.CUSTOM_ATTRIBUTE t1
where ((t1.PROJECT_AREA_ITEM_ID = ?) and (t1.SCOPE_COL = ?) and (t1.ARCHIVED = 0)) order
by t1.NAME_COL asc
You can see lots of interesting information, such as when the trace occurred, and what user
(TestJazzAdmin1) performed the action. After that, you see the marker that tells you the
scenario (viewTestCases) and a unique id such as
2b4ee1cc-35dc-475e-8872-04ed3bcdb39f
You can then use filter using that to get just the lines associated with the scenario
grep "2b4ee1cc-35dc-475e-8872-04ed3bcdb39f" sql.txt >browseTestCase.txt
You can also see the specific service call used by the web UI in each trace output line. In the
example above, see: ICompositeWebRestService/browseInitData
Sometimes I am looking at the entire scenario, other times I am looking at a single service call.
Once I decide, I will often trim the output one last time by stripping out the text using a regular
expression like:
.*sql =>
That gives me a more compact sequence of output, where I can focus on the SQL alone.
Using repodebug
You can get SQL tracing directly from repodebug. Again, go to a URL such as
http://host:port/jazz/repodebug
This time, go into
database->queryStats
Here you can start and stop SQL tracing. So let’s look at the same scenario, “browse test cases”.
Start the SQL tracing by clicking “start” and then again quickly run the scenario, return to the
repodebug page and click on stop.
This produces a somewhat nicely formatted table of output divided into sections for write, read,
and other. SQL statements are sorted by total time, and if the count of the number of times each
statement occurred is listed. In our scenario of “browse test cases” there are 10 rows displayed,
and we can see that certain SQL statements are run 10 times. This is a hint that such statements
occur once per row.
Unfortunately, many of our SQL statements are long, and this makes looking at the entire table a
bit tricky. You can scroll the table to the right, and eventually you will see other columns listing
performance statistics, as well as whether stack tracing is enabled for this statement.
Unfortunately the actual sequence of the tracing is gone, so the SQL from asynchronous tasks
may be intermixed. Some of these are easy to spot, such as:
SELECT T1.ITEM_ID, T1.STATE_ID FROM PRINT.PRINT_JOB…
SELECT T1.ITEM_ID, T1.STATE_ID FROM LM.RUNNING_JOB…
SELECT T1.ITEM_ID, T1.STATE_ID FROM LM.RUNNING_DISCOVERY…
Finding the stack traces for your SQL
There is a column in the output labeled “hash” and each entry is a clickable link. If you click on
such a hash value in a row, it takes you to information about that SQL statement. It also provides
an action at the top for “enableStacks”. Once you click that, stack tracing is on for this specific
SQL statement. You can then start tracing again, run your scenario, and stop tracing. Then, when
you click on the hash link for your statement, you will see that the output contains one or more
stacktraces where the SQL occurred.
Using the debugger
You can use the debugger to find the actual SQL being generated for a specific query. Setting a
breakpoint in the repository code generally does not work well, as the asynchronous tasks will
hit it pretty quickly. If you are OK with stopping the tasks, you can simply let the breakpoints
occur and not resume execution for the asynchronous tasks.
More commonly, if you know the RQM code where you want to find the SQL, you can set a
breakpoint in the RQM code. Then you can go to the code for
com.ibm.team.repository.service.internal.ExecuteDataQuery.doServerQueryData()
and look around line 92 where you will see
String trueSQL = aResult.query();
Once your breakpoint hits in RQM code, you can use the debugger option “Run To Line” to run
until the line 93 (after the trueSQL has been calculated). This will not set a full breakpoint, and
only your thread will stop at that line. For the trueSQL variable you may see something like:
select t1.ITEM_ID, t1.STATE_ID, t1.JZ_DISCRIMINATOR from (select * from
PLANNING.VERSIONABLE_ITEM z1 where z1.JZ_DISCRIMINATOR = 151) t1 left outer
join PLANNING.VERSIONED_TEST_PLAN_TEST_CASES j1 on
(t1.VERSIONED_TEST_PLAN_TEST_CASES = j1.JZ_PARENT_ID) left outer join
PLANNING.VERSIONABLE_ITEM t2 on (j1.ITEM_ID = t2.ITEM_ID) where
((t2.ITEM_ID = ?) and (t1.ARCHIVED = 0))
And if you are interested in the predicate used in the query, you can look at the variable
query.filter
where you can see something like
r.testCases().itemId()._eq(q.newUUIDArg())._and(r.archived()._eq(0))
(operator: AND)