Using the Visual Studio debugger

advertisement
Two bytes meet. The first byte asks, “Are you ill?”
The second byte replies, “No, just feeling a bit off.”
CP4349 - Debugging Autodesk® Product Plug-ins
Kevin Vandecar
Principal Developer Consulting Engineer – M&E Sparks Autodesk Developer Network
© 2011 Autodesk
bio: Kevin Vandecar
 @ Autodesk for over 18
 all customization related
years

Experience with AutoCAD, AutoCAD Architecture, 3d Studio (DOS),
Autodesk Revit
 Last five years as Software Engineer on AutoCAD Architecture API and
Revit API
 now focusing on 3ds Max customization in ADN Sparks team

Based in Manchester, New Hampshire, US

email: kevin.vandecar@autodesk.com
© 2011 Autodesk
Autodesk Developer Network

Access to almost all Autodesk software and SDK’s

Includes early access to beta software

Members-only website with thousands of technical articles
 Unlimited technical support
 Product direction through conferences
 Marketing benefits

Exposure on autodesk.com
 Promotional opportunities

One to three free API training classes

Based on user level
www.autodesk.com/joinadn
© 2011 Autodesk
What are we covering…

Most “plug-ins”, “add-ins”, “apps”, etc. for Autodesk applications are DLLs

Use mainly the C++ language to demonstrate debugging techniques in AutoCAD and
3ds Max
 Touch on managed debugging, using Revit as an example.
 Take a look at how CER mini-dump may be helpful for you

Look at the Visual Studio debugging tools

Nearly all of the techniques are the same between native (C++) and managed (C#,
VB.NET, etc.)
© 2011 Autodesk
Debugging…

…is fundamentally problem solving
"Intellectuals solve problems,
geniuses prevent them." - Albert Einstein
© 2011 Autodesk
What is Debugging?

Wikipedia:

Debugging is a methodical process of finding and reducing the number of bugs,
or defects, in a computer program or a piece of electronic hardware, thus
making it behave as expected. Debugging tends to be harder when
various subsystems are tightly
bugs to emerge in another.
coupled, as changes in one may cause
aka: Autodesk plug-in developers! 
Autodesk Application
plug-in 1
plug-in 2
plug-in 3
© 2011 Autodesk
What is Debugging?

For the “coder” much easier to understand at the implementation time

Code-time testing is important! aka Unit Testing
 document your code!

For the “inspector/debugger” much harder
You may be debugging other people’s code
 Or it may be code from your distant past
 This makes it much more difficult

© 2011 Autodesk
How to debug?

First verify problem and reproduce it…

No use setting break points, stepping through code, etc. without
knowing how to repro it.

helps to document steps
 smallest repro steps/sample is ideal

When communicating API problems beyond your code to Autodesk, it
is ideal to have the smallest repro sample code possible.
Swigert: 'Okay, Houston, we've had a problem here.'
Houston: 'This is Houston. Say again please.'
Lovell: 'Houston, we've had a problem. We've had a
main B bus under-volt.'
© 2011 Autodesk
How to debug?
 Then once you found the problem…
 Find the entry point into necessary code branch


If you have a complex plug-in, sometimes UI resource strings can
help to find how to enter the code branch
Start Debugging!

Use the tools you have
 The debugger itself is not the only
tool!
© 2011 Autodesk
How to Debug Autodesk Plug-ins? – different techniques

Coding Techniques

Additional Application specific options

Visually Inspect the Code

Use VS debugging tools

Reminder, VS2010 C++ Project Format used for 3ds Max and AutoCAD 2012 SDKs,
but targeting VS2008 SP1 for binary compatibility (V90 toolset).
 Revit is also using the VS 2010 managed compiler directly.

CER mini dump files
© 2011 Autodesk
Coding techniques

Simplest form: Use Output statements

Debugger output (OutputDebugString)
 Application output (“printf”, “cout”, and other routines)

Sometimes apps have their own output means




3ds Max: MaxMsgBox, MAXScript mprintf
AutoCAD: acutPrintf, acedAlert
Revit: TaskDialogs
Other Debug code statements

http://msdn.microsoft.com/en-us/library/ms679303(v=VS.85).aspx
© 2011 Autodesk
Coding Techniques

Assert



code safety to ensure conditions are always met
Useful during dev cycle and all testing
#if <NAME> Pre-Processor statements


Technique can be used for “asserting” or other debug output in production
code
Use project “variable” to set/unset, then very easy to enable/disable during
build process
demo (acad)
© 2011 Autodesk
Additional Application specific options


Exploit application specific tools
For example:

Revit has RevitLookup API sample and debugging tool

AutoCAD

has ArxDbg and ArxDbgMgd (also vertical versions for ACA and AME)
 Public symbol server (pdbs)

3ds Max

Create routines that work in MAXScript context
 has a full debug build with symbols and with some base source code
 Public symbol server (pdbs)
© 2011 Autodesk
Visually Inspect the Code

If you have a large team of developers, source control can help

Source Control differences between old and new versions of source code

Look at file history to find out who made changes
© 2011 Autodesk
Use Debugging Tools – Visual Studio

Visual Studio IDE


VS 2010 and VS2008 IDEs are very similar for debugging
VS “global” Debugger Settings

Tools, Options, Debugging
 Debug, Options and Settings

Consider Turning on Exception settings
demo
© 2011 Autodesk
VS Debug Project settings

Debugger “type”

Auto
 Managed
 Native Only
 Mixed-mode (very useful with C++/CLI together with C++)

Edit and Continue (useful especially when learning)
demo
© 2011 Autodesk
VS Debug Project settings

Caveat for x64 vs. win32 (x86) projects

Edit and Continue STILL not supported in VS2010 for x64 targets

x64 Mixed-mode debugging also not available for .NET Framework 3.5 and prior. Only
works in .NET Framework 4. The good news is that .NET Framework 4 is now
supported by many Autodesk apps, but a C++ project has to be modified manually.


See: http://adn.autodesk.com/adn/servlet/devnote?siteID=4814862&id=16002138&linkID=4900509
Can build x86 targets and run under WOW, but some Autodesk apps are not supported
as x86 on x64 Operating Systems (NOTE: 3ds Max is!)
© 2011 Autodesk
Using the Visual Studio debugger

Watch window

Variables

Autos
 Locals
 Watch



Can change variable data at runtime
Can use format specifiers to change the display of data
Step commands
Demo
(revit and acad)
© 2011 Autodesk
Using the Visual Studio debugger

Breakpoint

To set one:

Press F9
 Right Click->Set Breakpoint
 Click in the margin

Halts execution at a specific line
 Allows visualization of memory state at a given execution state


Step into, step over, step out, run, run to cursor
Right click -> Step into specific
© 2011 Autodesk
Using the Visual Studio debugger

Breakpoint “types”





Default == break every time
Location
Condition
Hit Count
Filter (Processes and Threads)


For example, break in process 1, but not in 2
When Hit (Uses trace points)

Print Message: myPtr Val: $(myPtr)
 Run macro

Edit Labels
Demo (revit)
© 2011 Autodesk
Using the Visual Studio debugger

Other useful windows


Call Stack
Output

Remember to turn off output you do not want to see.
 Can redirect to Immediate window.





Modules
Registers
Processes (useful in multi process apps/COM/etc.)
Threads
Memory
Demo (acad)
© 2011 Autodesk
Using the Visual Studio debugger – tips/tricks/demo

Use the breakpoint window

Helps you to see what break points and where
 You can also directly set breakpoints there
 Use labels with your break points

Disable, rather than delete breakpoints
 Call Stack break point
 Use Step into specific
 Run to Cursor (temp break point)
 Drag the execution pointer to another line
 Change values in the watch window
 Use the find tool in other windows (ie. it can help in output)
 Turn on/off output that may or may not be interesting for you debugging
session.
demo
© 2011 Autodesk
Symbols
 PDB
files are the key
 Contains
the debugging information
 Symbols also generated in “release” mode.
 Optimization can cause “tricky” debugging
 This is how it is possible to debug production software, using pdbs.

AutoCAD and 3ds Max now have public symbol servers!

http://symbols.autodesk.com/symbols
© 2011 Autodesk
Symbols

When using symbols, versions MUST MATCH

internally a GUID is present and must match
 Symbol servers simplify this task, but there are more “unknowns” when it doesn’t work



Likely we have not uploaded proper symbols, so let us know
Consider setting up a Symbol server internally for your own public builds
Great Blog article here about symbols:

http://www.wintellect.com/CS/blogs/jrobbins/archive/2009/05/11/pdb-files-what-everydeveloper-must-know.aspx
© 2011 Autodesk
Managed debugging

Managed APIs are often wrappers around native C++ APIs


Managed debugger shares many of the same features


This can sometimes cause strange debugging results
Revit
Mixed-Mode debugging can be very useful

AutoCAD
 3ds Max
demo
© 2011 Autodesk
Customer Error Report (CER)
 CER
= Customer Error Report.
 Key Information Collected
 Mini-dump files
 Snapshot of the memory
 Applications Counters
by CER
© 2011 Autodesk
Key Information Collected by CER
 System
Information
 Hardware information

Computer manufacturer
 Amount of RAM installed
 Number of processors
 Video
card model
 Operating
System
 Computer
Name

Video card driver version
 Configured video resolution
 Amount of video ram
© 2011 Autodesk
CER debugging – How to debug the minidump

Make sure to have the EXACT production install indicated in the CER data

Use the EXACT matching debug symbols if you have them

devenv <minidump>.dmp
© 2011 Autodesk
Code Structure tools

Intellisense

Code Model
 Dynamic help when coding and debugging
 Rewritten completely for VS2010 using an actual database (instead of NCB file)

VS2010 Ultimate has an Architecture feature

UML diagrams built in
 Dependency graph

Visual Assist

Spell checker!
 Plus many other useful helpers
© 2011 Autodesk
Other useful tools

Dependency Walker (depends)


SysInternals stuff


http://dependencywalker.com/
http://technet.microsoft.com/en-us/sysinternals/default.aspx
Reflector (for .NET Assemblies)

http://reflector.red-gate.com/download.aspx
 (no longer free )

WinDbg

http://msdn.microsoft.com/en-us/windows/hardware/gg463009.aspx
© 2011 Autodesk
How to Debug Autodesk Plug-ins? – Summary

Coding Techniques

Additional Application specific options

Visually Inspect the Code

Use VS debugging tools

CER mini dump files
© 2011 Autodesk
Developing Autodesk plug-ins

Autodesk has pages for most of the products that have APIs

The website entry for developing on a specific product is typically:

http://www.autodesk.com/develop<product name>

http://www.autodesk.com/develop3dsmax
 http://www.autodesk.com/developmaya
 http://www.autodesk.com/developmotionbuilder
 http://www.autodesk.com/developfbx

http://www.autodesk.com/developautocad
 http://www.autodesk.com/developrevit
 http://www.autodesk.com/developinventor
© 2011 Autodesk
Autodesk University Session Feedback

Your feedback is very important to Autodesk!

You can complete the session
survey on your mobile device,
PC, or at a survey station.

Each completed session survey
enters you in that day’s drawing
for a free AU 2012 pass.

You can help make AU 2012 better!
Complete the AU Conference Survey at a survey station and receive an AU 2011 T-Shirt.
© 2011 Autodesk
Questions?

Feel free to contact me later
Why do programmers always mix up Halloween and Christmas?
Because Oct 31 equals Dec 25!
© 2011 Autodesk
Download