Wichita/Sedgwick County HMIS
Evaluation Guide
August 2006
David Ediger, HMIS Coordinator
United Way of the Plains
245 North Water
Wichita, Kansas 67202
(316) 267-1321
TABLE
OF
CONTENTS
About This Guide ..................................................................................................................................... iii
Objective .............................................................................................................................iii
Audience ...............................................................................................................................iii
Organization ......................................................................................................................iii
Definition Phase ................................................................................................................................... 1-1
Objective ...........................................................................................................................1-1
Tasks ....................................................................................................................................1-1
Define the Project Scope ................................................................................1-2
Define and Rank Objectives for the New System ..............................1-6
Define Specific Requirements for the New System .........................1-7
Assessment Phase ................................................................................................................................... 2-1
Objective ...........................................................................................................................2-1
Tasks ....................................................................................................................................2-1
How the Current System Works ......................................................................2-2
Analyze the Current System ...........................................................................2-7
Selection Phase...................................................................................................................................... 3-1
Objective ...........................................................................................................................3-1
Tasks ....................................................................................................................................3-1
Identify System Candidates .............................................................................................3-2
Solicit Vendor Proposals .................................................................................................3-7
Validate Vendor Claims and Performance .......................................................................3-7
Table of Contents
i
Evaluate Vendor Proposals ..............................................................................................3-7
Negotiate Vendor Contract ............................................................................................3-11
Implementation Phase ................................................................................................................................ 4-1
Objective .................................................................................................................................4-1
Tasks .......................................................................................................................................4-1
Prepare the Implementation Plan .....................................................................................4-2
Install Hardware and Software Infrastructure ..................................................................4-2
Build a Test Environment ................................................................................................4-2
Conduct System Testing ..................................................................................................4-3
Train Users ......................................................................................................................4-3
Prepare the Changeover Plan ...........................................................................................4-3
Transition to the New System ..........................................................................................4-4
Perform a Post-Implementation Review ..........................................................................4-4
ii Wichita/Sedgwick County HMIS Evaluation Guide
About This Guide
Objective
This publication is a guide for the definition of requirements
for
an
operationally
serviceable
and
successful
HMIS
implementation, evaluation of the current Wichita/Sedgwick County
HMIS implementation, and comparison of alternate HMIS systems.
Audience
This publication is designed for a group to evaluate the current
HMIS system and potentially recommend implementation of a new
Wichita/Sedgwick County HMIS System. The outline utilized is not
unique and incorporates standard system and project management
tools, beginning with needs identification and progressing
through the successful implementation of a new system.
Organization
The remainder of this publication is divided into four sections
which include:
1. Definition Phase
2. Assessment Phase
3. Selection Phase
4. Implementation Phase
About This Guide
iii
PHASE
1
Definition Phase
Objective
The objective of the
operational requirements
will be used to assess
compare candidates for a
definition phase is to identify the
of the HMIS system. These requirements
the current system and to identify and
new system to the current system.
Tasks
The specific tasks of the definition phase are:

Define the scope of the HMIS project.

Define and rank objectives for the new system.

Define specific requirements for the new system.
Wichita/Sedgwick County HMIS System Evaluation Guide
1-1
Definition Phase
Define the Project Scope

What functions are supported by the HMIS system?



The primary function is to meet HUD data collection and reporting
requirements of homeless person demographics and aggravating
conditions. The supporting federal laws are: the McKinney-Vento
Homeless Assistance Act (42 USC 11381), HUD Supportive Housing
Rule 42 CFR 583, HUD HMIS Data and Technical Standards (69 CFR 146
and 69 CFR 201), initial HUD Notices of Funding Availability
(NOFA) 66 FR 12207 and 66 FR 27987, and renewal HUD NOFA’s 69 FR
26942 and 69 FR 27497. The initial HMIS grant HUD project number
was KS01B102002 and the first renewal grant HUD project number is
KS01B402004.
 The immediate local impact of the HUD required data
collection is to provide statistical information on the
number and types of persons/family units affected by
homelessness, the demand and utilization of bed-night
capacity, and length of stay data.
Secondary functions of the HMIS system within the local CoC
include:
 Increase the service referral capability between local
homeless service providers.
 Provide a case management system for homeless service
provider use that expands on the baseline HUD required data
collection.
How does the current system interface with organizations
within the Wichita/Sedgwick County CoC?

The following agencies have historically been primary HMIS
participants:
 Salvation Army
 YWCA
 United Methodist Urban Ministries
 Inter-Faith Ministries
 Step Stone
 Catholic Charities
 FEMA Emergency Assistance Providers (Catholic Charities,
Red Cross, Center of Hope)
1-2 Wichita/Sedgwick County HMIS System Evaluation Guide
Definition Phase





What features of the current system are critical to the
success of HMIS?



COMCARE and Catholic Charities shelters would be a participants
through data sharing.
An additional “organization” is configured to allow data
collection for Internally Displaced Persons (IDP) following major
disaster events.
United Way of the Plains functions as the host agency for the
HMIS, providing staff and network access/equipment, and software
hosting.
In general, any organization receiving federal McKinney-Vento
funding has an HMIS participation requirement. It should be
noted, however, that HMIS participation is not restricted to those
organizations, creating a large potential base for HMIS
participation.
There is presently 18 months of entered data for a single agency
that are critical to the success of HMIS.
As further use of the system is currently being restricted due to
security and privacy concerns, there are no further features of
the current system that are critical to the success of HMIS.
What opportunities exist for improving upon the current
system?





Adherence to HUD HMIS Data and Technical standards (69 CFR 146 and
69 CFR 201).
 Provides outline of HUD Universal and Program-Specific data
elements and response values.
 Provides baseline privacy and security standards and
additional recommended privacy and security practices.
Increase confidence and participation in the HMIS implementation
through the preceding item.
Provide opportunity for efficient client intake through:
 Data sharing of client demographics where permitted by
client approval and agency function.
 Concise user interface for client intake.
Incorporate a service referral mechanism utilizing AIRS taxonomy
to standardize service utilization reporting and to utilize data
from 2-1-1 referral services.
Improve access to the HMIS system and program functions by
encouraging support for alternate input methods (bar code/mag-
Wichita/Sedgwick County HMIS System Evaluation Guide
1-3
Definition Phase


Does this project have any constraints?



stripe) and alternate computer platforms (pen notebooks,
portables).
Leverage HMIS technology to facilitate CoC-wide collaboration
through a website for communications, CoC statistics, and news and
updates. For example, the Tarrant County, Texas CoC site at
www.tcaccess.org.
The HMIS project is funded entirely through a federal grant and
local matching funds. Any budget impact must be considered within
the limits of those funds.
The HMIS project, as of August 2006, is completing its fourth
overall year under the federal grant and its first year under a
three year renewal. Future renewals are unequivocally dependent
on improving participation in HMIS.
What resources are needed to successfully implement a system?



Personnel:
 HMIS Coordinator – To perform appropriate technical
evaluations, physical implementations of agency
configurations, agency end user training, system operation
and maintenance, technical assistance.
 Agency Executive Director – To provide agency leadership in
participation in HMIS.
 Agency Liaison – To provide direct liaison with HMIS
Coordinator in agency implementation.
Funding:
 Implementation of the HMIS requires initial purchase costs,
user licensing, and continuing annual maintenance costs.
Existing equipment and network access is sufficient with
minimal improvements until a larger active user base is
established.
Equipment:
1-4 Wichita/Sedgwick County HMIS System Evaluation Guide
Definition Phase






The HMIS servers commonly provide the functions of user
access and security, database hosting, application hosting,
Internet/remote access and security, and web hosting if
required by the software.
The current software implements the functions of user
access and security on one server, with the functions of
remote access, database hosting, and application hosting
combined on a second server.
Remote access to the HMIS is provided through agency
internet connections into the HMIS remote access server.
Separate router and firewall devices configured for
specific access profiles protect the HMIS servers from
general internet accessibility.
Anti-virus and other anti-malicious utilities are utilized
on the HMIS servers.
Regular data and system backup procedures are utilized on
the HMIS system.
Wichita/Sedgwick County HMIS System Evaluation Guide
1-5
Definition Phase
Define Objectives for the System
Objectives for the HMIS have been developed utilizing HUD HMIS
technical assistance materials and local CoC criteria outlined in
the Project Scope section previous. These are:

Case Management Needs


Technical Details


Appropriate vendor implementation and support of technical
requisites, particularly in distributed access and information
security is necessary. Also important are operating system and
database platform considerations as they impact the need and
availability of technical expertise for implementation and
maintenance.
Security/Privacy


Beyond basic HMIS data collection, offering service providers a
hosted case management function avoids duplication of resources in
acquiring and maintaining separate systems among providers and
leverages the data collection process required for HMIS.
The presence of Personally Protected Information (PPI) in the
HMIS, as well as the participation of domestic violence shelters,
mandates the need for appropriate security controls and privacy
considerations.
Pricing Model

The ability of the CoC to fund the acquisition, implementation,
and operation of the HMIS is also a primary consideration. The
CoC has already experienced one transition due to vendor licensing
increases beyond the ability of grant and matching funding
available.
1-6 Wichita/Sedgwick County HMIS System Evaluation Guide
Definition Phase
Define Specific Requirements for the New System
Utilizing the above Objectives as a framework, and again
utilizing HUD HMIS technical assistance and CoC criteria, the
following Specific Requirement categories have been developed:

Case Management Needs

Integrated HMIS – Data Collection Integrated with Intake Process
All evaluated packages support HMIS data collection and
reporting. Integration of HUD HMIS-specific data collection
with the intake process rather than as a stand-alone module
reduces intake workload and duplication of data while
increasing consistency of data.

Independent Agency Support
Support for multiple, independent agencies by keeping client
data segregated by agency, permitting HMIS statistical deduplication across agencies, and permitting client data sharing
across agencies where allowed.

Intake per Agency/Workgroup
Presentation of an agency- or program-specific intake form to
the user rather than a single intake form applied to all
agencies.

Guided Workflow
Ability to create and utilize guided workflow for the user to
follow rather than an intake form with many unneeded data
elements available. Also the ability to link input elements to
prevent contradictory or invalid responses.
This item can
increase user efficiency, reduce training and retraining, and
improve data consistency and completeness.

User Role Support (Intake, Case Manager, Director, Administrator,
etc.)
Ability to present functions and restrict data access based on
a user’s agency, program, or organizational role.

Domestic Violence Shelter Support
Wichita/Sedgwick County HMIS System Evaluation Guide
1-7
Definition Phase
Ability to discretely handle DV clients through strict privacy
and
sharing
control,
user
restrictions,
and
client
anonymization.


AIRS/Referral Support
Ability to scan services offered within community agencies for
potential
eligibility
and
referral.
Ability
to
link
services/referrals
with
AIRS
taxonomy
for
query/report
standardization and accurate referral capabilities. This item
reflects the potential for linkage to 2-1-1 and other community
resources for reporting or point-of-contact external referrals.
Technical Details

Technical Standards
Conforms to HUD Data and Technical Standards (69 CFR 146 and 69
CFR 201) in general and data storage and integrity in
particular.

Platform/Delivery
Technical architecture of the system. Client/Server, hosted
ASP, standard application, browser presentation, or other.
Impacts the operational implementation of hardware and system
access as well as the needs and/or availability of technical
expertise to implement and maintain.

Alternate Delivery
Ability to utilize differing access environments.
Primary
access is assumed to be via an internet-enabled workstation,
either through a remote access program like Terminal Services
or a web browser. Alternate access would indicate the ability
to utilize a pen notebook computer or PDA style device for
example.

Alternate Input
Support for alternate means of input such as a bar code reader
or mag-stripe reader. Useful for processing repetitive clients
or services as with the UMUM Drop in Center.

Database Engine
As with Platform/Delivery, indicative of requirements for local
expertise to implement and maintain.
Commercial SQL products
such as Microsoft and Oracle are more mature, widely supported,
and intended for distributed access environments. Open source
databases are less widely supported, while dBase-compatible
products are intended for local stand-alone or limited user
count applications.
1-8 Wichita/Sedgwick County HMIS System Evaluation Guide
Definition Phase



Self-Host Capable
Vendor fully supports self-hosting by the local CoC. This item
is necessary for DV shelter participation in order to retain
full control over critical PPI information.

Supports custom reporting tools
Vendor supports local custom report development through the
availability of a maintained database schema and standard data
access interfaces.

Integrated Data Interchange via HUD XML/CSV format
Vendor supports or implements within the software import/export
in the HUD HMIS XML/CSV format.
Security/Privacy

Encryption of Personally Protected Information (PPI)
Encryption of Personally Protected Information (PPI) within the
database per HUD Data and Technical Standards (69 CFR 146 and
69 CFR 201).

Encryption of Transactions involving PPI
Encryption of transactions between workstation and server per
HUD Data and Technical Standards (69 CFR 146 and 69 CFR 201).

Per User Credentials and Configuration
Software
supports
per-user
login/access
credentials
and
implements appropriate data access and software functionality
based on such credentials/restrictions.
Pricing Model

Software Acquisition Costs, Agency/User Licensing, and
Maintenance/Support Fees.
Outline of vendor’s pricing model to allow comparison
expected costs at acquisition and over time.
Wichita/Sedgwick County HMIS System Evaluation Guide
of
1-9
Definition Phase
1-10 Wichita/Sedgwick County HMIS System Evaluation Guide
PHASE
2
Assessment Phase
Objective
The objective during the assessment phase is to understand the
current system’s operational requirements.
This understanding
provides a baseline for comparing the operational requirements of
different systems.
Tasks
The specific tasks of the assessment phase are:

How the current system works.

Analyze the current system.
Assessment Phase
2-1
Assessment Phase
How the Current System Works
A detailed study of the current system including fact finding documentation and
a background analysis.
Fact Finding
Fact finding documents factual data about the current system.
sources were utilized in the preparation of this evaluation:
 Correspondence and questionnaires:







HMIS agency executive directors and users.
Simplicity provided references from 2004 were re-contacted.
HUD provided Simplicity HMIS CoC’s administrators were contacted.
Observations included:




ABT Associates, HUD’s HMIS Technical Assistance contractor.
HMIS Coordinator files.
Correspondence with Allen, Gibbs & Houlik, L.C., a consulting firm
utilized by United Way of the Plains.
Correspondence with the current software vendor.
Correspondence with potential software vendors.
Correspondence with ad-hoc national HMIS system administrator
groups.
Personal interviews:




The following
System configuration and operations.
Overall agency acceptance and adoption.
End user acceptance and adoption.
Research:


HUD grant applications and acceptance documents.
HUD Data and Technical Standards documents.
2-2 Wichita/Sedgwick County HMIS Evaluation Guide
Assessment Phase

History of the local HMIS effort under the HUD grants.
Assessment Phase
2-3
Assessment Phase
Background Analysis
Background analysis allows understanding and documentation of the functional
organization and structure. A complete background analysis should enable
identification of all the people who use or are affected by the current system.

In approximately 1998, the agencies and organizations that serve
the Sedgwick County area’s homeless population recognized the need
for a management information system. United Way of the Plains
began data collection on the homeless services in 1999. Among the
needs identified were:
 Improve service to the area’s homeless population.
 Identify duplicate requests for services.
 Facilitate information collection and data exchange among
the area’s homeless shelters.
 Provide accurate statistics for grant writing to secure
future funding.
 Identify trends and gaps in services for community
planning.

The generalized structure of an HMIS implementation is:
 Interaction of the HMIS Coordinator with an agency
executive director in determining HMIS implementation goals
and objectives
 Interaction of the HMIS Coordinator with an agency liaison
for meeting specific HMIS implementation objectives.
 Interaction of the HMIS Coordinator with an agency end user
for HMIS training and, with agency liaison approval,
technical support.

The current HMIS implementation represents the second software
implementation under the HUD grant.
 Prior to initial HUD grant funding, the local CoC utilized
an in-house developed Microsoft Access database entering
copied intake forms submitted from agencies.
 The initial HUD-funded system was ServicePoint by Bowman
Internet Systems, based on a first generation web user
interface attached to a SQL database.
 System configuration generally progressed through Q4 of
2002 with agency implementations going active during 2003.
2-4 Wichita/Sedgwick County HMIS Evaluation Guide
Assessment Phase



Notice was received during Q2 of 2004 that Bowman would be
significantly increasing licensing fees and switching
databases to discourage local hosting; significantly
increasing costs beyond what was provided under the HMIS
grant.
Usability issues also became significant through
limitations of the vendor’s browser implementation.
A new software package, Simplicity, from Simplicity Computing
Solutions, was selected during Q3 2004.
 Screening criteria focused on acquisition cost,
customization capability to accommodate various agency
intake processes, and avoidance of browser based form
input.
 References provided by Simplicity were single-agency social
service organizations, not participating in HMIS or
utilizing the software in the multi-organization role
required by the local CoC.
 The Simplicity software utilizes a Windows application
connected through a Borland Database Engine to file-based
individual data tables. Access by the end user is through
Windows Terminal Services sessions.
 System configuration and customization proceeded through
the remainder of 2004 and throughout 2005. An initial
agency was active in Q1 2005 with other partial
implementations configured continuing from that time period
to the present.
 The HMIS Coordinator resigned at the end of June 2005 and
was re-staffed in September 2005.
 Data privacy/security concerns became an issue during Q4
2005 to the present as additional agencies were being
implemented.
 During the same time period, report inconsistencies were
noted and investigated.
 Upon inquiry from HUD’s HMIS Technical Assistance
contractor, it was determined that the Wichita/Sedgwick
County CoC is the lone remaining user of Simplicity for
HMIS functions out of three who had acquired the
application for HMIS.
 Further inquiry with thse organizations revealed that the
local CoC, along with one other, where the initial
Assessment Phase
2-5
Assessment Phase
implementers of the multi-agency layer in the software.
The third CoC never reached an implementation phase.

Ongoing system issues have culminated in this system evaluation.
2-6 Wichita/Sedgwick County HMIS Evaluation Guide
Assessment Phase
Analyze the Current System
The following framework1 assists in the process of systematically analyzing the
current system. This framework consists of six categories; each beginning with
a letter that forms the acronym PIECES.

Performance

Information

Economy

Control

Efficiency

Services
Performance
Performance problems occur when a system is not responding fast enough to meet
business needs. When examining the performance category, determine if the current
system provides adequate throughput and response time.
1

Performance of the current system is adequate and is not
limited by current usage levels or hardware/network
capability.

Performance with the Microsoft Terminal Services presentation
is primarily dependent on outgoing server capabilities,
outgoing server bandwidth, remote location bandwidth, and
remote location workstation capability.
WETHERBE, JAMES C., Systems Analysis and Design (2nd ed.). St. Paul, MN: West Publishing Company, 1984.
Assessment Phase
2-7
Assessment Phase
Information
Information problems occur when a system is not providing useful information to
users. Examining the information category determines if the current system provides
timely, pertinent, accurate and usefully formatted information.

Without CoC-wide implementation, the informational value of
the HMIS software is very limited to non-existent.

Existing, vendor supplied reports from the system have been
determined to utilize different source tables, resulting in
values that vary between reports.

The reporting engine does not utilize user or agency
parameters, resulting in reports that must be manually
duplicated for each agency and hard coded with an agency
filter.

The only filter and grouping options available through the
report interface are beginning/ending date parameters.
Economy
Economy problems occur when a system is not providing the ability to track or
reduce costs. Examining the economy category determines if the current system
offers adequate service level and capacity to provide informational value.

CoC agency funding is at risk with further delayed utilization
of the HMIS. This approximate annual amount is $2 million.

Intense involvement of local staff, both the HMIS Coordinator
and the agency liaisons has been, and will foreseeably be
required to ensure agency data segregation and report
consistency.

Vendor does not appear to operate proactively. Data
segregation issues discovered by another Simplicity CoC
2-8 Wichita/Sedgwick County HMIS Evaluation Guide
Assessment Phase
implementation were not addressed to other Simplicity CoC
implementations such as ourselves until uncovered locally.
Software patches have only been implemented when initiated by
the HMIS Coordinator. The vendor despite annual maintenance
fees identified for this purpose has implemented no general
version upgrades; the vendor has revealed no future upgrade
schedule either.
Control
Control problems occur when a system is not providing information accuracy,
integrity and security. Examining the control category determines if the current
system offers adequate controls to ensure against these problems. Note that a
system can suffer from too many controls as easily as too few.

System controls are a very serious issue:



Windows Terminal Services restrictions are implemented to prevent
invocation of more than one active Windows session per user.
Additional Windows interface restrictions are available, and have
been implemented to prevent casual user manipulation of the
Windows environment except as noted below.
Simplicity operation relies on the presence of a visible
read/write enabled mapped drive for its programs to access various
data tables and reports. During an operating session, this drive
letter is available for any user to browse without restriction.
Individual table files can be viewed, copied, or deleted without
restriction. Similarly, report files are available from the file
interface for perusal without restriction. As reports do not
check user restrictions, this item exposes data irrespective of
agency data access/sharing restrictions.
Although Simplicity does employ an initial Username/Password logon
sequence (in addition to the Windows Terminal Services logon), the
functional application of user restrictions is haphazard
throughout the program. Various data exposures have been
discovered where user/agency restrictions have not been applied
and client data has been universally viewable. Although the
vendor as we have called their attention to them has addressed
Assessment Phase
2-9
Assessment Phase






these, the issues demonstrate a lack of a culture of project
management and the reactive nature of their general responses.
Contrary to HUD Data and Technical Standard 4.3.2 (binary data
storage format), data files are directly human readable via
numerous basic file access tools, ie: command prompt TYPE command,
Notepad application, Excel application.
Contrary to HUD Data and Technical Standard 4.3.1 and 4.3.2 (user
authentication), Simplicity does not prohibit users from signing
on to more than one instance of the program or prevent itself from
invoking more than one instance.
An administrative data manipulation application is available from
the file interface without restriction. A user can browse, copy,
and delete data or change Simplicity user-specific access
permissions without restriction.
Contrary to HUD Data and Technical Standards 1.4 and 1.5, client
Personal Protected Information (PPI) is unencrypted and available
in clear text through means noted previously.
Contrary to HUD Data and Technical Standards 1.4 and 1.5, client
Personal Protected Information (PPI) is highly duplicated between
data tables, allowing ready association of client PPI with
services offered through means noted previously without
restriction.
Data integrity is similarly a tenuous situation:


Due to the individual file-based data tables, referential
integrity can only be maintained programmatically within the
application code. The particular database engine employed does
not support referential integrity as part of the database; it must
be appropriately programmed and maintained within the code of the
application itself.
Client information is duplicated more than reasonably necessary
within the data tables. This issue is the primary source for the
report issues noted under the Information section above.
Efficiency
2-10 Wichita/Sedgwick County HMIS Evaluation Guide
Assessment Phase
Efficiency problems occur when a system uses resources wastefully. These resources
can include people, equipment, materials and money. When examining the efficiency
category, determine if the current system makes maximum use of available resources.

From the standpoint of hardware and network efficiency, the
current software system performs well.

Personnel efficiency within the local CoC has been taxed as
follows:



The HMIS Coordinator has been required to perform numerous
security reviews of the system culminating in this evaluation. The
findings of these reviews do not indicate a mature or even a
continuously improved software system, rather a system that is
only addressed at the need of the client.
Future version usability enhancements of the system are in doubt;
as of the date of this evaluation (August 2006) it is six months
past the implementation of an annual upgrade. Conversations with
other Simplicity agencies indicate that none are operating with
the 2006 updates; there did not indeed even appear to be a common
version of the system between those contacted.
End user efficiency is impacted as follows:
 An extreme reliance on drop-down controls with no default
values requires a high degree of mouse to keyboard to mouse
movements. The ordering of tab keying between form inputs
is inconsistent.
 Grid displays of multiple clients have fixed layouts
presenting clients in name order only; family members with
different last names become scattered throughout the grid
with no way to logically group them together. Family
members sharing a common last name will become scattered in
the display listing due to the large number of individuals
with that name (i.e.: individual clients John Smith and
Paul Smith will be inserted in the display between family
members Joe, Mary, and Sally Smith). Identification of
individuals with first and last names in common is trial
and error as no further identifying information other than
first and last name is available on the grid display.
 Because of the previous, misidentification of an existing
client’s record is common.
 “Customizable” form fields are implemented using generic
text box controls with no provision for value type checking
Assessment Phase
2-11
Assessment Phase





or bounds checking. For example, it is possible to enter,
and the system will accept, “The Quick Brown Fox” in the
Client SSN field. As this item includes several HUD HMIS
data fields, this is contrary to HUD Data and Technical
Standards 1.4 and 1.5 (Data Elements and Quality).
Similarly, there is no means to designate required input
fields other than those coded by the vendor. As this item
includes several HUD HMIS data fields, this is contrary to
HUD Data and Technical Standards 1.4, 1.5 (Data Elements
and Quality) and 5.1.5 (Data Completeness).
Entry form customizations are created by re-using an
existing input field and database field as needed; i.e.:
The client table column “religion” actually stores the
client’s primary phone number. It is even be possible to
re-use the same field differently between agencies. Using
the preceding example, it is possible assign the “religion”
field to actually store a religion value for one agency, a
phone number for a different agency, and spouse name for a
third agency.
Entry form customizations are also created by hiding
unneeded input fields when not needed, creating “holes” on
entry screens that interrupt the visual flow as entry
progresses from field to field. The resulting screens give
little sense of a coherent workflow process that is to be
followed as many screen forms are cluttered with unneeded
optional fields, re-named fields that are out of place
related to their data, and holes created by hidden fields.
Custom input “forms,” such as agency specific intakes and
the HUD HMIS specific fields, require the user to click on
a question item to activate that item, click again within a
pop-up form to enter the response in either the generic
text control or use the mouse to select from a drop down
response, click to save the response, and finally click to
select the next item. Entered responses are not presented
on the screen for review; the process must be repeated to
review a response. Corrections are not possible without
administrative intervention.
Logical functional groupings are disjoined in the system.
For example, HUD HMIS data collection requires the user to
visit as many as 4 tabs in order to complete the process during which they will have entered duplicate client
demographic and PPI information several times. Client
family member entry does not offer a means of duplicating
the primary member’s demographics if desired, and does not
2-12 Wichita/Sedgwick County HMIS Evaluation Guide
Assessment Phase
offer a means of quickly creating the family unit without
interrupting the primary member’s record entry.
Services
Service problems occur when a system is not providing adequate service to the
business and/or its customers. Examining the service category determines if the
current system provides desirable and reliable service. The flexibility,
expandability, ease of use and coordination of the system should also be examined.

The reliance on generic text fields and re-used fields within
the system severely inhibit the straightforward and timely
retrieval of useful data:


At the most basic level, this means that custom reports must be
created for every agency unless unified intake and reporting
processes are imposed on CoC agencies.
When reporting on data that involves a custom data field,
particularly when totaling or summarizing, the reporting engine
becomes burdened with the duties of type checking and type
conversion. The HUD HMIS income values and the FEMA emergency
assistance amount values for example, are stored within such
generic text fields.

Additionally, there is no means to aggregate or summarize data
across CoC agencies if that data involves re-used table
fields. An example would be to implement a report comparing
service referrals between agencies or totaling service
referrals across the CoC. Such a report requires utilization
of additional software, such as Access or Excel, to create
appropriate summary queries.

The above outlined Efficiency items have a direct negative
impact at the points of contact with clients by impeding
rather than assisting the collection of data and the ability
to provide case management services. This impact affects both
the agency end user staff and the clients being served.
Assessment Phase
2-13
Assessment Phase

Further implementations, security and privacy issues aside,
rapidly create an inverted pyramid shape in the effort
required to:




Maintain the integrity of the data within
Ensure the quality of the data within the
Implement new or expanded CoC agencies or
Provide accurate and informative analysis
system.
2-14 Wichita/Sedgwick County HMIS Evaluation Guide
the system;
system;
services;
of the data in the
PHASE
3
Selection Phase
Objective
The purpose during the selection phase is to identify candidates
and select a new system. This process is based upon the specific
requirements developed during the definition phase.
Tasks
The specific tasks of the selection phase are:

Identify system candidates.

Validate vendor claims and performance.

Evaluate vendor proposals.

Negotiate vendor contract.
Selection Phase
3-1
Selection Phase
Identify System Candidates
HMIS packages were selected and screened from HUD’s database on the broad criteria
of not existing as an add-on module to a larger social or medical case management
system that would need to acquired separately as a pre-requisite. From this
selection, seven packages were identified – the current application plus six
additional. Of the six additional, two applications (in addition to the current
implementation of Simplicity) were re-evaluated from the 2004 selection process.
Each of the packages are summarized and then presented in a table comparing their
functionality with the System Requirements presented in the Definition Phase of
this guide.

Simplicity – Current system.

HousingWorks.Net – Hosted ASP with Acrobat Forms interface.
Windows desktop application delivered
through Terminal Services. Borland dBase database. One time fee per
agency with annual support fees per agency. Custom report support.
Requires workstation install of Adobe Acrobat. Local hosting would
be a custom implementation and has not been performed elsewhere.
Oracle database. Priced with one time configuration fee per agency
with annual subscription. Adobe Acrobat license required per
workstation. Reports developed by the vendor for extra fees.
Previously evaluated in 2004.

Tapestry – Hosted ASP with browser access, no self-hosting
provisions. Open source PostreSQL database. Priced with one time
acquisition fee and per month user fees. Vendor provides report
customizations for fees. Multi-Agency support but single intake
method.

Radii-Advanced Client Management – Windows application with
Firebird open source database. Self-host capable. Multi-agency
capable but single intake methodology. HMIS not integrated with
intake. Priced per one time acquisition and per-workstation fee.
Custom report support. AIRS support.

ClientTrack.Net – Microsoft.Net program delivered via workstation
install or remotely through browser. Microsoft SQLServer 2000/2005
database. Priced with one time acquisition fee, one time per user
license fee, and annual support fee. Custom report support. AIRS
support.
3-2 Wichita/Sedgwick County HMIS System Evaluation Guide
Selection Phase

Service Point – Software utilized prior to 2005.

Awards – Hosted ASP with open source database/browser delivery.
Discontinued due
to prohibitive licensing costs and hosting restrictions. Open source
server and Oracle database. No local hosting. Multi-agency capable.
Priced per one time acquisition, monthly per user fee. Custom
reports available; require additional per month user fee.
No
self-hosting. Multi-agency, custom intakes possible – implemented as
extra cost customization per agency by vendor. Priced per one time
acquisition, monthly per agency and per user fees. Custom reports by
vendor only. Previously evaluated in 2004.
Case Management
For complete
descriptions, refer to
the Definition Phase of
this Guide.
HMIS Application Comparisons
Simplicit
y
Housing
Works.Net
Tapestry
Radii
ClientTrac
k. Net
Service
Point
Awards








Integrated HMIS

Integration of HMISspecific data within
the intake form to
reduce workload and
increase data
consistency.
Independent
Agency Support
Multiple agencies
supported to allow data
sharing and HMIS data
de-duplication.
Intake per
Agency/ Workgroup
Agency or program
specific intake forms
rather than a single
form for all agencies.

1




2
Guided Workflow
Allows efficient
completion of intake
tasks while providing
checks to prevent
invalid or
contradictory
data
User
Role Support
entry.
Reduced or enhanced
functionality based on
a user’s agency,
program, or
organizational
affiliation.
DV Shelter
Support
Specific support for DV
shelters through strict
privacy controls,
sharing restrictions,
and client
anonymization.













Selection Phase
3-3
Selection Phase
Referral Support
Listing of services
offered within
community agencies for
eligibility screening
and referral. AIRS
taxonomy support.

Note
s:
1 - Incomplete implementation.
2 – Incomplete implementation.



Screen forms can display data from other agencies.
User restrictions are inconsistent throughout application.
3-4 Wichita/Sedgwick County HMIS System Evaluation Guide
Selection Phase
Technical
Details
For complete
descriptions, refer to
the Definition Phase of
this Guide
Technical
HMIS Application Comparisons
Simplicit
y
Housing
Works.Net
Standards
Conforms to HUD Data
and Technical Standards
in general and data
storage and integrity
Platform/
in particular.
Delivery
Client/Server, hosted
ASP, desktop
application, browser
presentation, or other.
Desktop
Application;
Terminal
Server
Remote
Access
Adobe
Acrobat
Forms;
Internet
Access
Tapestry
Radii
ClientTrac
k. Net
Service
Point
Awards





Linux
Server;
Browser
Access
Windows
Client/Serve
r; Desktop
Application
Linux
Server;
Browser
Access
Linux
Server;
Browser
Access
Alternate
Delivery

Suitability for
adoption on a pen
notebook or PDA style
portable device.
Windows
Client
/Server
Microsoft.NET
;
Workstation
or Browser
Access

Alternate Input


Open Source
Firebird
Microsoft
SQLServer
2000/2005
Oracle SQL









Capability to use a bar
code or mag-stripe
reader for repetitive
functions.
Database Platform
Commercial SQL
(Microsoft or Oracle),
Open Source SQL, or
desktop database.
Borland
dBase
compatible
Oracle SQL
Open Source
PostgreSQL
Open Source
PostgreSQL
Self Host
Supports self hosting
by the local CoC.
Custom Reporting
Custom reporting
possible with vendor
database schema and
data access interfaces.
Integrated Data
Interchange
Data interchange
capable using HUD HMIS
XML/CSV formats.

Selection Phase
3-5
Selection Phase
Security/Privacy
For complete
descriptions, refer to
the Definition Phase of
this Guide
HMIS Application Comparisons
Simplicit
y
Housing
Works.Net
Tapestry
Radii
ClientTrac
k. Net
Service
Point
Awards















PPI Encryption
Encrypts Personally
Protected Information
(PPI) within the
database per HUD Data
and Technical
Standards.
Encrypts
1
Transactions
Encrypts transactions
between the workstation
and server per HUD Data
and Technical Standards
Per User
Credentialing and
Configuration
Provides for dynamic
user access and
functionality
restrictions.


2
Note
s:
Description
Describes vendor
pricing for
acquisition, licensing,
and support.
3 Year Budget
Impact
1
1
1 - Assumed as vendor is a hosted ASP and maintains control over encryption.
2 – Incomplete implementation. User can bypass menu restrictions via a file explorer interface.
Pricing Model
For complete
descriptions, refer to
the Definition Phase of
this Guide
1
HMIS Application Comparisons
Simplicit
y
Housing
Works.Net
Currently
Owned;
Per Agency
Implementati
on
Fee;Annual
Support Fee.
One time
configuratio
n and annual
subscription
; Acrobat
per
workstation.
Tapestry
Radii
ClientTrac
k. Net
One time
acquisition
fee; Monthly
user fees.
One time
acquisition;
per
workstation
fees.
One time
acquisition;
one time per
user; annual
support.
Estimate of 3 year
budget impact.
3-6 Wichita/Sedgwick County HMIS System Evaluation Guide
Service
Point
Awards
One time
acquisition;
per month
user fee;
per month
per user
report
writer.
One time
acquisition;
per month
per agency;
per month
per user.
Selection Phase
Solicit Vendor Proposals
Once you have acquired a basic understanding of the available candidates and their technology
implementations, you should request vendor proposals. These requests should state both the
technical and business requirements of the new system. Following is a suggested format for the
request.
Validate Vendor Claims and Performance
As you receive responses to your requests, you should confirm that the vendor proposals meet the
specified requirements. For software components, you should request that each vendor provide a
full-featured demonstration of their product’s capabilities. For hardware, you should ensure that
all equipment meets your compatibility, reliability and scalability requirements. It is also advisable
to perform background checks on all vendors. Background checks can be conducted by contacting
vendor references and by using traditional business analysis services such as Dun & Bradstreet.
Evaluate Vendor Proposals
Your next task is to evaluate the validated proposals. You should compare the feasibility of each
alternative being considered. From this analysis, you can identify viable alternatives that meet the
operational requirements of the new system.
When evaluating the feasibility of a solution, you should address three categories:

Operational feasibility

Technical feasibility

Economic feasibility
Selection Phase
3-7
Selection Phase
Operational Feasibility
Operational feasibility measures acceptability. Two major points are considered in this
measurement. Will the solution being evaluated operate proficiently? And, how well will the
solution be accepted by users and management?
When trying to determine proficiency, use the PIECES framework. When trying to determine
acceptability, ask yourself the following questions:

Does management support the new system?

How do users view their interaction with the system?

What users and managers are going to resist change?

What can be done to minimize resistance to change?

How well can users and managers adapt to change?
Technical Feasibility
Technical feasibility measures reasonableness and practicality. Four categories are generally
considered in this measurement:

Is the technology of the alternative solution practical? In other words, is it mature and
proven?

Do we currently have the necessary technology, and if not, can we obtain it?

Do we currently have the necessary technical expertise?

Is the time schedule for implementation reasonable?
Economic Feasibility
Economic feasibility measures cost-effectiveness. It determines if and when the lifetime benefits
of a solution will surpass the lifetime costs. First, you must itemize the benefits and costs of each
alternative solution. Then, you can apply the three standard cost/benefit analysis techniques:
return-on-investment analysis, present value analysis and payback analysis. Return-on-investment
and present value analysis determine the profitability of a solution, and payback analysis
determines how long it will take for a solution to pay for itself. The key to this task is accurately
3-8 Wichita/Sedgwick County HMIS System Evaluation Guide
Selection Phase
defining the costs and benefits of a solution; therefore, following is a discussion of these two
categories.
Costs
System costs are generally broken into two categories: development costs and operating costs.
Development costs of an alternative solution will include:

Personnel

Hardware and software

Training

Supplies and equipment
Selection Phase
3-9
Selection Phase
Operating costs of a system are usually either fixed or variable. The fixed operating costs include:

Lease or loan payments

Salaries of information system personnel
The variable operating costs of a system include:

Supplies and maintenance

Overhead such as utilities, maintenance and communication expense
Benefits
Benefits, which are normally characterized as either increasing profits or decreasing costs, are
generally divided into two categories: tangible benefits and intangible benefits.
Tangible benefits are usually measurable and include:

Increased sales

Reduced expenses

Faster turnaround

Fewer processing errors

Increase throughput

Decreased response time

Elimination of job steps
Although they are not as easily measured, intangible benefits are just as important and include:

Better management reporting and support

Better decision reporting and support

Better customer relations

Improved employee morale

Improved employee job satisfaction
3-10 Wichita/Sedgwick County HMIS System Evaluation Guide
Selection Phase
Negotiate Vendor Contract
Once the proposals have been evaluated and a particular system has been chosen, you should gain
management approval to negotiate a contract with the vendor. Contract negotiations include, but
are not limited to the following considerations:

Purchase price and payment terms

Upgrade and maintenance costs

Training costs

Licensing

Terms of acceptance
Selection Phase
3-11
PHASE
4
Implementation Phase
Objective
The objective during the implementation phase is to transition to the new system.
Tasks
The specific tasks of the implementation phase are:

Prepare the implementation plan.

Install hardware and software infrastructure.

Build a test environment.

Conduct system testing.

Train users.

Prepare the changeover plan.

Transition to the new system.

Perform a post-implementation review.
Implementation Phase
4-1
Implementation Phase
Prepare the Implementation Plan
After the selection phase is approved, you must prepare an implementation plan. The
plan should include:

A plan for installing the hardware and software infrastructure

A plan for testing applications individually and as an integrated whole

A plan for training users

A plan for converting procedures, related applications and data preparatory to the
actual changeover

A preliminary changeover plan
Install Hardware and Software Infrastructure
If new hardware and software has been selected, you will install it during the
implementation phase. This task is generally a two-step process. First, you must prepare
the site. Site preparation usually involves preparing for certain environmental
requirements such as special locations, furniture and security. Once the equipment
arrives, you must then manage the actual installation of the hardware and software,
which may or may not be done by the vendor.
Build a Test Environment
You must build a test environment before implementation can take place. The test
environment should be structured using information representative of a production
environment.
4-2 Wichita/Sedgwick County HMIS System Evaluation Guide
Implementation Phase
Conduct System Testing
After the new system has been installed and the test environment built, you must test the
system as an integrated whole. This is done to assure that all aspects of the operational
requirements are being met properly. System testing should involve those responsible
for using and/or maintaining the new system.
Train Users
Using the reference manuals supplied with the system and any supplementary materials
developed during this project, you should train the operating and user personnel. You
should coordinate your training schedule with the schedule for completing the
implementation phase. All training should be completed before the implementation
phase report is presented for acceptance at the end of the implementation phase.
Prepare the Changeover Plan
You must update the preliminary implementation plan, which was developed in the
beginning of this phase. The resulting changeover plan should specify the method of
system changeover (abrupt cutover, parallel conversion, location conversion or staged
conversion). This plan should also detail the schedule of activities and the responsibility
of all personnel involved in the changeover.
Implementation Phase
4-3
Implementation Phase
Transition to the New System
Using the changeover plan, you should now convert to the new system. The primary
focus of this task is to ensure a graceful transition. Attention should be given to the
acceptability of the new system and errors or omissions in system design. Next, you
should measure the validity of the new system by considering the following categories:

System performance

Peak workload processing performance

Human engineering

Methods and procedures

Backup and recovery
Perform a Post-Implementation Review
Once the system is operational, you should conduct a post-implementation review. This
review should be conducted approximately two to three months after the system is
declared operational. The purpose of this review is to determine how well actual
performance compares to promised performance. Individuals who are not directly
involved in the system’s implementation or operation should perform this review. They
should complete a system performance evaluation report in the following format:
System Performance Evaluation Report
I.
System Name
II.
Specific Performance Objectives
III. Method of Evaluation
IV. Evaluation Results
V.
Recommendations
4-4 Wichita/Sedgwick County HMIS System Evaluation Guide