Creating an Integrated Appraisal SW System in Excel Alan Gertner Tejas Appraisal And Software Alan Gertner Bio • Graduated from Colorado St University, BS and MS in Computer Science • In the high tech industry for 20 years, last 5 years working on a public data standard and data exchange between vendors • Appraiser in Austin, Tx for 12 years What is an Integrated SW System • SW within the same tool or set of tools, work well together • Same HI and usage wherever possible • Prefer using industry standard SW, primarily Microsoft • Some separate tools may be needed • Tight integration is preferred over loose integration Open System Philosophy • Plug and Play user selected tools • Allow user to select best in class tools (sketching, mapping, cost estimate, data source, etc.) SW Development Issue • SW Developer should attempt to limit the creation of new SW code • The SW maintenance cost is much higher than the development cost • Leverage off existing functionality and only develop new SW when needed Path to Becoming an Appraiser • Typical Appraiser path involves Trainee and Mentor • Another path is a Provisional License, no Trainee / Mentor • Provisional License possible but is a challenge and not recommended (TALCB discourages this option and is phasing it out) • Pursued Provisional License, started with nothing • Treated as an Engineering Problem, i.e. identify and evaluate the appraisal process and the required tools (software, hardware, data sources, field inspection equipment, camera, etc.) Appraisal Software Analysis • Appraisal Report Creation process appeared straight forward • Found a small number of Appraisal Software companies • Quick investigation into the companies to determine viable options • Requested demo copies from several companies First Appraisal SW Demo • Started with low end Appraisal SW • Created a 15 page Appraisal Report • Numerous Usability and HI issues (Subject address change requires editing each page in report, cut/paste/import photos were mangled, HI inconsistent with my other SW • After evaluation, not a viable option Second Appraisal SW Demo • Popular Appraisal SW, high end • Appeared to be a heavyweight installation and usage application (required lot of resources and system mods / requirements) • Overwhelmed by the HI and the duplication with other generic SW packages on my pc • Might work, but not a desirable option Third Appraisal SW Demo • Mid range Appraisal SW package • During the evaluation, I discovered the company was struggling and may cease to exist • Stopped evaluation • Not a viable option Appraisal SW Evaluation Results • Numerous issues with the available Appraisal SW packages • Human Interface • Complexity • Lack of needed functionality • Heavyweight installation / usage requirements • Did not fit my requirements, only reason to consider writing an Appraisal SW System Appraisal SW Requirements • Support the Appraiser workflow • Tightly integrated system which leverages off my existing software • Similar HI and usage as my other SW tools • Simple functionality with extended functionality available if needed Appraisal SW Tasks • Admin, business records, bookkeeping • Manage related parties (Appraisers, Clients, etc.) • Manage Assignments (everything related to an assignment) • Create Appraisal Report (Std Forms, Forms Library, Sketch, Maps, Photos, Import Data, customizable) • Deliver Report • Archival Professional Appraisal Report Contents • • • • • • • • • • • Cover page Table of Contents Cover Letter Std Form (1004, 1004C, 2000, etc) 1004MC Appraiser Comments Map Sketch MLS Listing History Photos (Subject, Comp, Field Inspection) Misc (Plat, Survey, Flood Map, Aerial View, etc) Appraisal Report Contents • See Example Appraisal SW Candidate • • • • • • Combination of MS Office Tools Access / Word / Excel Already have on pc May be a viable option How tightly integrated are they? May be too complicated and overkill for my usage Appraisal SW Candidate • • • • MS Word Should work well for report creation Not sure about other requirements May be able to use with another product to obtain required features • Not sure if best option Appraisal SW MS Word Issues • While attempting to use MS Word for Appraisal SW, I quickly realized MS Word is text centric and an Appraisal Report is data centric • Text centric, primarily text • Data centric, primarily data points • MS Word does not work well for an Appraisal Report with many data points Appraisal SW Candidate • • • • • • MS Excel Excel is data centric rather than text centric Text processing is available but has limitations Can be used as a simple db Has drawing capabilities but no 2d sketch May be a viable option MS Excel vs Appraisal SW Requirements • • • • • • Administration Manage Parties Manage Assignment Create Report Deliver Report Archive Yes Yes Yes (folder) ??? ??? Yes (backup) Excel Appraisal SW Design Overview VBA Data Source (MLS) Std Forms Customized Forms Appraiser Report Creation Assignment Admin Manager Assignment Instance Report Delivery Client Install Init Misc. Cost Data Sketch CAD, Map, Aerial, etc. Tejas Appraisal SW Folder Hierarchy C:/appraisal_1_0_1 addin db excel_forms_db new_assignment_init pending_assignment active archive main folder appraisal_manager.xlsm appraisal_install_option.xlsm Excel VBA addins data files use by appraisal sw std and customized forms customized report templates misc. notes on possible assignments active assignments completed assignments Tejas Appraisal SW Folder Hierarchy • See Examples Excel Appraisal Report Creation • Significant challenge • This is a “must have” requirement for an Appraisal SW System • This challenge had to be overcome for Excel to be a viable option Excel Appraisal Report Creation Requirements • • • • Create Report templates Create library of items to add to a report Able to insert library items into a report Library item formulas must be updated after insertion into report • Allow user customizations • Create report to be delivered to the client Tejas Appraisal Report Creation • All the Tejas Appraisal Reports are considered narrative reports. • Reports are created from scratch. • Reports may include GSE standard forms such as 1004, 1004C, 2000, etc. Forms Library • Standard Forms 1004, 1004C, 2000, etc. Need Fannie Mae approval, submit samples Freddie Mac approval is not required ERC (Employee Relocation Council) form requires license and fee to use • User must be able to created customized forms and load into reports later Excel Forms Library Implementation Notes • VBA forms were considered for forms library Increase complexity and costs • VBA forms were not needed for many items Easier and more efficient to use worksheet Use cells, name references and formulas • Leverage off existing functionality Tejas Appraisal Sketch Package • Initial implementation used an external popular 2d Sketch package • Several problems with the external 2d Sketch package: HI was different Lot of functionality, but difficult to use for the infrequent user Sketch was a large image file and fuzzy Tejas Appraisal Native Sketch • Second implementation was a native sketch package built on top of Excel drawing functions • Excel drawing objects may be customized • Simple with advanced features available if needed partition and label sketch build sketch from smaller blocks, then combine eraser temp measuring stick available cloak of invisibility Tejas Appraisal SW Sketch • See Examples • Most complex sketch drawn was a circular house in the shape of an Archimedean spiral Tejas Appraisal Cost Approach • • • • • Cost data from any source may be used Select cost options and perform table lookups Uses Excel cells, names and lookup functions Improved Cost Estimate Form 1007 Craftsman National Building Cost Manual Approx $30 / yr for pdf version 2-3 hours / yr to update the Excel cost data Tejas Appraisal Cost Approach • See examples Local MLS Issues • • • • • • • MLS data is Appraiser’s primary data source Desire to import MLS data into Appraisal SW Local MLS staff are not knowledgeable MLS changes are not disclosed Appraiser must work around MLS issues Appraiser must understand MLS and Appraisal SW Considered using RETS (R.E. Transaction Std) MLS Staff hostile to this approach Worried stealing MLS data MLS Import Issues • MLS Data must be normalized to Appraisal SW data model • MLS field names may change without notice Current Days on Market -> CDOM • MLS field values may change without notice “T” becomes “True” • MLS field default values and missing fields blank vs space, required field is missing MLS Import Issue • See examples Appraisal Report XML Creation • GSE UCDP Requirement • XML is a text file with Header info, Subject info and Comp info • Tejas SW uses two worksheets and VBA to create XML Header and Subject info Comp info, use loop and indirect references • Small VBA function to read the worksheets and write to xml file XML Creation • See examples Excel 2010 Notes • Excel 2010 not as stable as Excel 2003 • Excel 2010 may be slower than Excel 2003 Many more rows x columns 65k x 256 to 1 mil x 16k • Many Excel Names may cause slow performance, primarily deleting or redefining names, workaround it to turn off Excel AutoCalc during large operations Excel 2010 Notes • File size may become huge if many unused rows/columns are included in worksheet 1 mb -> 40 mb with 65k rows Use <ctl> End to find the last used cell in WS • Worksheet display scaled < 100% may impact other WS items such as cell drop down menu button (shrinks) and VBA AddPicture (picture loaded incorrectly, workaround is Select WS in VBA first) Excel 2010 Notes • Formula window is 1 line and does not auto expand, have to manually expand and shifts the worksheet down, Excel 2003 automatically adjusted the formula window based on the info typed • PropertyDB VBA update hangs (also in Excel 2003) Either a bug in Excel VBA or in my code Workaround is set VBA dbugger breakpoint and continue Excel Useful Features • Cell Data Validation Define list of values Specify the list as the valid values for a cell Creates a drop down menu for the cell • Conditional Formatting Highlight cells that meet certain condition(s) • Highlight items of interest in report • INDIRECT, MATCH and INDEX functions table lookup, useful with cost data and XML creation Excel 2010 Excel Addins and Utilities • Data Form Editor (better than Excel built in) • JKP Name Manager for Excel cell names • See Examples Useful Misc Tools • Custom UI Editor for MS Office Create ribbon XML commands • XML Notepad View XML file Use MISMO .xsd to validate field values UCDP validation tool may identify other issues • Diffpdf, compare pdf files • See examples MS Excel Resources • Online forums, search web • Search web for specfic Excel issues GSE UCDP Issues • Many Appraisal Reports must be submitted to UCDP and pass the GSE UCDP validation tool • GSE and UCDP staff limit their support to their selected vendors • If you contact the UCDP staff, they do not want to talk to you if you are not on their selected vendor list GSE UCDP Issues • GSE’s claim Lenders will assist Appraiseers and Appraisal SW Companies to test with UCDP and resolve issues. • GSE’s cannot identify any Lender who will do this. • GSE’s continue to make this claim even thought they know it is not accurate. GSE UCDP Issue • GSE’s and the GSE selected Appraisal SW Vendors claim the UCDP / UAD / MISMO specification documents are sufficient for Appraisal SW Developers to comply with the GSE UCDP requirements • The GSE selected Appraisal SW vendors are unable to comply with the GSE UCDP requirements using the documents • Require GSE UCDP testing and support GSE UCDP Issues • GSE’s have not provided any publicly available mechanism for the Appraisal User Community to determine if an Appraisal Report XML file will pass the UCDP validation test. • At the same time, the GSE’s will begin blacklisting Appraisers for non – compliance • GSE’s have created a similar publicly available test web site for the ULDD https://www.fanniemae.com/singlefamily/loandelivery-test-environment GSE UCDP Test Web Site • GSE’s MUST provide public access to the UCDP XML validation tool so anyone (SW Developer, Appraiser, Lender, etc.) can confirm if an XML file will meet the UCDP XML validation tool requirements. • Provide example XML files for each GSE Form (1004, 1073, 1075, 2055) • Provide Related Documents (links to MISMO, UCDP and UAD specifications) GSE UCDP Issue Examples • XML comment delimiter caused UCDP to reject file. • Blank line in XML caused UCDP to reject file. • UCDP was attempting to open the submitted XML file, extract the embedded PDF file and validate the PDF file. PDF file validation does NOT work. UCDP was NOT supposed to validate an embedded PDF file. GSE UCDP Public Test Web Site Needed • Needed by the entire Appraisal User Community Members • Allow UCDP XML file validation and testing before submitting XML files on “live” assignments • Without this, the GSE’s are creating a monopoly for the GSE selected vendors • There was a back room deal between the GSE’s and their selected vendors to limit UCDP access and support GSE UCDP Issues • See examples Current Issues • Need public UCDP Test Web site • Appraisal Validation Tools that do not work on all valid reports. Primarily pdf validation tools. • GSE’s overloading Appraisal Form fields with multiple values and special encoding (UAD values, driveway, sold and contract dates, etc.). Modify forms to create fields for required values. Current Issues (cont) • GSE’s blacklisting Appraisers for UCDP violations. GSE’s refuse to make UCDP validation tool publicly available. GSE’s require Appraisers to maintain local property db, primarily for Quality and Condition ratings. Exposes underlying issue of using subjective property attributes for blacklisting Appraisers. Current Issues (cont) • Multiple Report Formats (MISMO GSE, AI Ready, ACI, pdf conversion, etc.). Must use industry standard (MISMO GSE) and drop the rest For example, AI Ready has 4 versions of report (xml, pdf, mismo gse and embedded pdf) which may be different Current Issues (cont) • Multiple MLS Systems covering same areas Need a single MLS db and allow each local Board of Realtors select which MLS system they use. User decides which local BOR he wants to join, single membership rather than many memberships. National MLS db is best, state db is next best. Very political, the MLS is a cash cow for the local Board of Realtors. Tejas Appraisal SW Summary • Address the Appraiser work flow tasks • Leverage off existing functionality • Avoid writing many lines of code which need to be maintained, approx 10k lines of VBA • Originally implemented on Excel 2003 • Converted to Excel 2010 HI Ribbon Few VBA functions changed Worksheet spacing modified slightly Excel 2010 not as stable as Excel 2003 Tejas Appraisal SW Best and Worst • Best Feature Wide Open • Worst Feature Wide Open Tejas Appraisal SW Future ? • • • • • • Shareware Release Commercial Release of entire system Commercial Release of subsystem(s) Commercial Appraisal Support Apply to other problem domains Other (licensing, joint venture, etc.) Tejas Appraisal SW Possibilities Excel Functionality Available Excel Functionality Used DEMO