Reevaluating the Performance Appraisal for Software Engineers
Reevaluating the Performance Appraisal for Software Engineers
Lonnie S. Jones
A Prospectus Presented to the Information Technology College Faculty
of Western Governors University
in Partial Fulfillment of the Requirements for the Degree
Master of Science in Information Technology Management
12/14/16
1
Performance Appraisal Method for Software Engineers
2
Abstract
The problem was producing a method that addressed objective and subjective
perspectives that satisfied all major stakeholders, identified bottlenecks in performance, and
demonstrated impact. The methodology ultimately improved the performance of the employee,
supervisors, and the product.
The plan separated the processes of termination, retention, and promotion from
performance evaluation reviews. The departments utilized an innovative performance appraisal.
It measured the employees’ impact on each major stakeholder. The appraisal method was
dependent on the office emphasizing employee development and implementing office reform
policies. After there was sufficient data, directors generated reports for determining performance
bottlenecks.
The primary stakeholders were the engineers and department directors. The project’s
other stakeholders were customers, other employees, investors, and potential recruits.
The process of implementation relied on seven major milestones.
1. Create and implement specific criteria for promotions and terminations.
2. Enforce new office policies to support the new appraisal method.
3. Allow directors to enforce the new appraisal process.
4. Provide the finance and accounting department with sufficient time to adjust
compensation packages.
5. Allow the data scientist to create the reports for the supervisors and allow employees
to capture impact moments of other engineers.
Performance Appraisal Method for Software Engineers
6. Create a dashboard for directors, engineers, and other key stakeholders access to the
reports. With the supplied performance data, supervisors can utilize the reports to create
performance indexes on the departments.
7. Hold an impact conference to celebrate the victories of employees.
The outcome aimed to provide employees and supervisors with explicit data and
information regarding performance on a quick and regular basis. Engineers will have specific
criteria for promotions and terminations. Provide supervisors a transparent environment to
create strategies.
3
Performance Appraisal Method for Software Engineers
4
Table of Contents
Reevaluating the Performance Appraisal for Software Engineers..................................................1
Abstract............................................................................................................................................2
Table of Contents.............................................................................................................................4
Capstone Highlights.........................................................................................................................8
Needs Analysis Summary...........................................................................................................10
Background Information.........................................................................................................10
Problem Statement..................................................................................................................15
Consequences.........................................................................................................................16
Problem Resolution Summary...................................................................................................17
Problem Resolution Summary...................................................................................................18
Project Plan Summary................................................................................................................23
Quality Assurance Summary......................................................................................................24
Post Implementation Summary..................................................................................................25
Needs Analysis...............................................................................................................................26
Need for the Solution Analysis...................................................................................................28
Problem Statement.....................................................................................................................30
Problem Causes..........................................................................................................................31
Consequences.............................................................................................................................33
Stakeholder Impacts...................................................................................................................34
Performance Appraisal Method for Software Engineers
5
Problem Analysis........................................................................................................................36
Cost.........................................................................................................................................36
Risk.........................................................................................................................................38
Problem Resolution........................................................................................................................43
Best Approach............................................................................................................................43
Approach Justification................................................................................................................53
Alternative Solutions..............................................................................................................54
Consequences.........................................................................................................................56
Risk Assessment.........................................................................................................................56
Cost/Benefit for each risk.......................................................................................................60
Risk Mitigation...........................................................................................................................62
Project Design and Requirements..................................................................................................63
Execution Requirements............................................................................................................63
Objectives...............................................................................................................................64
Existing Gaps.............................................................................................................................66
Project Plan....................................................................................................................................68
Project Development Plan..........................................................................................................69
Scope......................................................................................................................................69
Assumptions...........................................................................................................................71
Important Milestones..............................................................................................................72
Performance Appraisal Method for Software Engineers
6
Timelines................................................................................................................................73
PERT Analysis........................................................................................................................78
Resource Requirements..........................................................................................................80
Deliverables............................................................................................................................82
Project Implementation Plan......................................................................................................82
Strategy for the Implementation.............................................................................................82
Phases of the Rollout..............................................................................................................83
Dependencies..........................................................................................................................84
Deliverables............................................................................................................................84
Training plan for users............................................................................................................85
Quality Assurance..........................................................................................................................86
Quality Assurance Approach......................................................................................................87
Solution Testing..........................................................................................................................87
Revisions....................................................................................................................................91
Summative Evaluation Plan.......................................................................................................91
Disseminating Results................................................................................................................92
Post Implementation Support and Issues.......................................................................................93
Post Implementation Support.....................................................................................................93
Post Implementation Support Resources....................................................................................93
Maintenance Plan.......................................................................................................................94
Performance Appraisal Method for Software Engineers
7
Conclusion, Outcomes, and Reflection..........................................................................................95
Project Summary........................................................................................................................95
Deliverables................................................................................................................................96
Outcomes....................................................................................................................................97
Reflection...................................................................................................................................98
Post-Mortem Directions.............................................................................................................99
References....................................................................................................................................100
Appendix A: Employee Hierarchy...............................................................................................111
Appendix B: Organizational Alignment.......................................................................................112
Performance Appraisal Method for Software Engineers
8
Capstone Highlights
is a startup company that opened
company
ago. The CEO had founded the
with two programmers and an office administrator. The organization
developed a mobile application (app) that specialized in localized alternative currency. The app
was built on Blockchain technology, the same technology found in Bitcoin. The CEO had
outsourced customer support to eastern Europeans, payroll and benefits to a local bookkeeping,
legal department to several local companies as needed, and marketing to a friend. Over
the organization had grown to a 40 person company that is primarily comprised of 10 types
of software engineers and two directors. The development and evaluation methods for engineers
had not remained consistent for each year.
To accurately measure the engineers’ skill required a standard definition of engineering
and metrics. The definition from ISO/IEC/IEEE Systems and Software Engineering Vocabulary
(SEVOCAB) defined software engineering as “the application of a systematic, disciplined,
quantifiable approach to the development, operation, and maintenance of software; that is, the
application of engineering to software)” (IEEE Computer Society, 2014). Historically, there had
not been enough managers with programming experience programming to quantifying their
employees’ performance. The situation was challenging. Many managers had utilized a metric
comparable to data entry. Specifically, the number of lines (of source code) per hour. The
industry standard had historically been 10 lines of code per day (Brooks, 1995). “A study by the
National Institute of Standards and Technology (NIST) in 2002 found that software developers
spend approximately 80 percent of their development costs on identifying and correcting
software defects or bugs, resulting in an estimated $59.5 billion loss to the U.S. economy.
According to Carnegie Mellon University’s CyLab Sustainable Computing Consortium, released
Performance Appraisal Method for Software Engineers
9
commercial software typically has 20 to 30 bugs for every 1,000 lines of code. Even highly
critical code such as operating system code follows a similar pattern. In Fig. 1, we show a
Bugzilla bug report for the Fedora Core operating system, with bugs grouped by severity (Zaks,
2007).”
Figure 1
Non-technical managers do not verify the work produced other than a screenshot, video,
or user interface. The situation creates an environment where management ignores skilled
programmers and rewards fast typists. An engineer could "game the system" since the supervisor
would only notice bugs with visual (front end) or severe server (back end) issues. The
implications of traditional performance appraisals hold data entry, minimum viable product
results, and a popularity contest as the foundation of performance. Managers may delegate
verification to testers or other engineers for some accountability but still neglects the actual
metrics of the position.
Performance Appraisal Method for Software Engineers
10
The purpose is not to criticize traditional management, but identify areas of improvement.
Every engineer and supervisor has specific areas of improvement. Engineers often neglect soft
skills in favor of technical elitism. Engineers could struggle to interact with employees outside
of the development and QA departments. The general traits of professionalism often outnumber
the required skills involved for software engineering. These areas are not usually discussed and
addressed systematically within organizations.
Needs Analysis Summary
The problem was the organization needed to produce an innovative method that
addressed objective and subjective perspectives that pleased directors and employees, identified
bottlenecks in performance, and isolated and correlated performance to impact. The
methodology would ultimately improve the performance of the employee, supervisors, and the
product.
The primary cause of the problem was most appraisal methods were designed nearly a
century ago. They belonged to managing and maintaining industrial methodology such as
railroads. These methods had no clear way to quantify or qualify software engineers’
performance and impact. Without a clear methodology to evaluate the employees’ performance,
it was not feasible for directors to appraise the engineers’ performance. Without transparency in
performance in the employees, it made it equally impossible to measure the performance of the
supervisor.
Background Information
The company had anticipated a moderate success that would take years to develop,
market, and release to the public. The organization met with extreme success within a short
period. The first year’s downloads, profits, and total users were more than ten times the best
Performance Appraisal Method for Software Engineers
11
case scenario estimations. There were rapid changes that were required since the technology was
designed to support 50,000 simultaneous users.
needed to take shortcuts for hiring and
promoting since many employees would work 14-18 hour days. The CEO was working more
than 120 hours per week to maintain product performance. Many employees were showing signs
of burnout and some started to question their purpose.
Within the
the organization grew from four employees to more than 40
employees. Development (Dev) and quality assurance (QA) were the two departments that had
software engineers. There was a director and five employees for each department. Most of the
employees had come from high-profile organizations. Since the organization was still small, the
directors and the CEO also contributed to the source code. Employees were paid well compared
to other large organizations, but the benefits package and amenities were lacking.
The app utilized bleeding edge technology which can often result in a volatile (platform)
environment. In a traditional environment it is difficult to assess how employees are performing.
But in a highly volatile technology environment it is incredibly challenging. There were limited
opportunities for training. The only option for many high performers was self-evaluation. So
when a conference or training seminar was available, it was critical to take unless the
organization opted to hire elite consultants or coaches.
The team dedicated the first year to creating a stable app and marketing to locals within
the greater
area. Initially, the app was liked, but many customers felt uncertain about the
security. Developing an alternative currency required the organization to exceed security
compliances to ensure customer trust. A traditional model required a few compliances. The
security analyst had developed a proprietary security standard that exceeded the compliances and
was verified from a third party. Customers and businesses were requesting new features that fell
Performance Appraisal Method for Software Engineers
12
outside of the 1.0 release. After expanding the scope, the app performance decreased because of
additional processes. The team improved the performance to load in less than 0.7 seconds. The
performance improvement resulted in the app instability. It was after finishing the feature that
the QA director eliminated all critical bugs that caused crashes.
For the first year, the company used the BARS method to rate employee's performance.
The BARS method focuses on the specific behavior as indicators of effective and ineffective
performance (PerformanceAppraisal365, N.D.). It is based on identifying the most important
dimensions of a particular job and then coming up with behavioral descriptions of the job to
coincide with a rating scale. So if a director utilized a rating scale from 1 to 5 then, there would
be five descriptions of performance. For example, if a customer service representative is rated a
5 (highest) if he “answered the phone on the first ring, spoke in a clear voice, gave the proper
greeting, established rapport with the customer, and used a friendly tone.” At the other end of the
scale, a 1 rating might be “answered the phone after three or more rings, mumbled, didn’t greet
the customer, and was not friendly.” Ratings of 2– 4 are described in a similar fashion
(CALDWELL, 2002).
It was a basic form including a series of questions that quantified traits into numbers.
The results of everyone were above average, but some criticized the performance appraisal since
it did not include subjective information or details. Some people also were provided scores they
felt were unjustified, both low and high numbers. Supervisors would explain their perspective,
but not the view of others since the reviews were anonymous. Evaluators average each metric
group and the overall metric average of the employee from the provided scores.
The second year was about reviewing employees to ensure that the required work was
completed to refine the product. 2014 was the year of the hack. Many large profile
Performance Appraisal Method for Software Engineers
13
organizations had their system penetrated causing billions of dollars in damages. Small
businesses were no different. The security analyst, Lawls, developed a system to address most of
the incoming attacks, the ops engineer assisted him by automating 27% of his workload, and the
director of QA added fault-tolerance and high-availability with the disaster recovery system.
The directors had requested the appraisal method change to a 360 review for the second
year performance appraisal. Ward (1997) defined the 360-degree feedback as: ‘The systematic
collection and feedback of performance data on an individual or group derived from some the
stakeholders on their performance.’
Ward (1997) continues, “Personnel fed the data back into the form of ratings against
various performance dimensions. 360-degree feedback is also referred to as multi-source
assessment or multirater feedback. Performance data in a 360-degree feedback process can be
generated for individuals (as shown in Figure 2) from the person to whom they report, their
direct reports, their peers (who could be team members and/or colleagues in other parts of the
organization) and their internal customers.”
Figure 2
Performance Appraisal Method for Software Engineers
14
The range of feedback could be extended to include other stakeholders – external
customers, clients or suppliers (this is sometimes known as 540- degree feedback). A selfassessment process may also be incorporated using for comparison purposes the same criteria as
the other generators of feedback.
Feedback can be initiated entirely by peers (in a team setting) or by both peers and the
team leaders. It can also take the form of 180-degree or upward feedback where this is given by
subordinates to their managers. Feedback may be presented directly to individuals, or to their
managers, or both. Expert counseling and coaching for individuals as a result of the feedback
may be provided by a member of the HR department or an outside consultant.”
After the second meeting, the CEO trusted their decisions to run their departments their
way. The company had utilized a 360 review with three peers and a supervisor.
The 360 review form included the traits documenting performance, programming
performance, communication, teamwork, professional etiquette, and adheres to policies &
procedures. The reviewer would rate each trait from one to six with a brief description next to
the rating. The reviews were anonymous except for the supervisors’ reviews.
After the second review, it affected all the engineers harshly. Resentment soared, trust
disappeared, and teamwork plummeted (Leviticus, N.D.). Many feared they were going to be
terminated and felt the reviews coming from the director of development and the director of
quality assurance (QA) were arbitrary. Three employees almost left the company as many of
employees were regularly offered higher paying jobs. Employees who had lower
performance scores were demoralized into almost quitting, and high performing employees had
said they were unsatisfied with the work environment. Many subordinates considered the review
process “Dilbertesque” and drastically contrary to the company culture. The review failed to
Performance Appraisal Method for Software Engineers
15
show personal development and recommend development. Others were also complaining since
all of the business objectives were hit ahead of schedule with thorough documentation. The
directors had justified their decisions with low performing employees because their
communication came across as arrogant and confrontational to the directors and other
employees. Employees countered the directors’ reviews by showing the CEO their work in
private meetings. One employee asked the CEO three questions:

What would he have to do to keep his job?

What would he have to do to be fired?

What would he need to be promoted?
He showed that he was one of the best competitive programmers in the world, he had published
papers, and had excellent reviews from his teammates. He did not understand why the director
had given him a negative review.
After the meetings with employees, the CEO had to reconsider the appraisal process. The
organization had generated large amounts of money. The team had many famous and talented
engineers.
Problem Statement
The problem is to produce an innovative method that addresses objective and subjective
perspectives that will please management and employees, identifies bottlenecks in performance,
and isolates and correlates performance to impact. The methodology will ultimately improve the
performance of the employee, supervisors, and the product.
Performance Appraisal Method for Software Engineers
16
Consequences
The consequences to the appraisal were serious. For employees who were looking to advance
their career had no references for promotions and employees who did not realize they were
struggling did not know what would cause termination. In either of these cases, good employees
could have left. There may be employees who received the wrong compensation package
depending on the primary conditions and measurements. The effects of the metrics and
compensation were demoralizing and frustrating. Without effective measurements, the
supervisors had a very limited perspective of the team’s performance and development as well as
the organization’s.
The previous 360-degree appraisal method required eight hours to develop. The first four
hours were two meetings from both directors to convince the CEO to change the appraisal
method. It cost the company $236.85. The next two hours were two independent hours where
both directors developed their 360-degree appraisal form. The last two hours was a meeting
where they took portions of each other's form to develop a final form. The total development
cost was $473.71.
For implementation, each review peer and supervisor reviews took 15 minutes, and a
self-review took 30 minutes. It would take each employee approx. 90 minutes for evaluations
and another 30 minutes to go over the review. The review cost for the teams was $905.33. After
the review, each engineer had a meeting with the directors to cover the results. The meetings
took 30 which cost the company $235.77. The total cost of implementation was $1,141.10.
After the review, the CEO had met with the AI, back-end, both full stack, and operations
engineers. Each meeting took an hour to complete. The meetings cost $226.81. The CEO had a
15-minute meeting with both directors. It cost the organization $29.61. The total for the fallout
Performance Appraisal Method for Software Engineers
17
meetings cost was $256.42. If the negative morale reduced productivity by 6% over three
months, then it may have cost the organization approx. $1.8 million dollars. The full cost of last
year's appraisal was $1644.
Problem Resolution Summary
The first aspect to resolving the problem was separating the termination and promotion
process from performance evaluations. Employees are terminated under the conditions that they
had performed poorly* consistently for three months or four of the 12 months of the year.
Performing poorly at
was described as receiving less than an 80% for performance.
The description was a bit contradictory, but employees were allowed three months to correct the
performance with a month of leniency from the supervising director and fellow employees.
While promotions were similar in that employees must meet consistency and the support from
peers and a supervisor.
The second aspect to resolving the problem was redefining performance. An employee’s
performance was limited to his ability, motivation, and the environment. The engineer could
only perform what he knows, what he was motivated to perform, and whether the environment
supported his actions.
measured knowledge, skill, response, and values as the foundation of ability. It
followed a systematic path and each measure had its appraisal method.
•
Knowledge described what the engineer knew what to do and is measured with a skills
matrix.
•
Skill described how well he knew what to do and is measured with coding KPI metrics.
•
The response was the appropriate time to utilize the skill and was measured with business
relationship models.
Performance Appraisal Method for Software Engineers
•
18
Values were whom to use the skills for and measured with objectives.
Measuring engineers’ motivation was the most challenging aspect of the project. It was
measured by perspective, autonomy, and mastery. Measuring motivation was more important to
the CEO and directors. When the directors do not provide the engineers with the values, reasons,
and direction to work, then the engineers can suffer from drudgery and early onset burnout.
When employees have the reasons to work and exert significant effort, then the employee needs
to be able to manage themselves to reach the objective. Engineers dislike being micromanaged
(Garvin, 2013). Mastery discloses historical information that demonstrates competence,
progress, and completion. When a person works on a task for a significant period without
showing any improvement, it lowers their motivation.
The environment was easier to measure.
measured employees by the community,
job design, resources, policies & procedures, and management support. Environmental issues
were measured in checks & balances and percentages.
Problem Resolution Summary
The reason for the approach was that employees would understand what conduct and
performance will result in termination. When people have certainty in what will cause
termination, it helps them know the minimum standard and minimizes the stress of performance
reviews. For employees who want to improve their career, employees will have an explicit
understanding of their duties, expectations, and performance for the next level. To assess those
who want to improve requires accountability. The office reforms will hold employees and
supervisors accountable for office politics, hold supervisors accountable for employee
development, and hold subordinates accountable for their performance. Facebook utilizes a
similar code of conduct by offering a completely transparent work environment. If employee A
Performance Appraisal Method for Software Engineers
19
sends an email speaking poorly about a supervisor to employee B, then it is employee B’s
obligation to inform employee A and the supervisor. The supervisor cannot retaliate against the
employee, but it is required to resolve the conflict in a positive manner. Similarly, the supervisor
is responsible for the development of employees in their department. It is the employees’
responsibility to meet performance and conduct standards. The policy will make employees
consider how they interact with each other and how they are contributing to the team and the
product. The bright side of transparency is in a positive environment is employees can see how
they have contributed. At the end of the quarter, the teams celebrate employee contributions at
an annual party. After the first quarter’s data, supervisors will be able to quickly and regularly
identify bottlenecks in performance in the department and individuals.
For the first year, the organization had implemented the BARS approach. The BARS
approach had more objective data but failed to get subjective data. According to Šalková (2013),
“The BARS Method (Behaviorally Anchored Rating Scales) assesses the behavior required for a
successful work performance. It focuses on the approach to work, observation of the work
procedure and performance efficiency. The BARS Method is based on creating rating scales for
individual work behavior aspects and on the definition of the required work behavior at a
specific working place as a prerequisite for efficient work performance”.
According to Milkovich, Wigdor, & Broderick (1991), there is sufficient validity for the
BARS approach to providing convergent and discriminant evidence. “Since other measures of
the job performance construct has not been readily available in most settings, it has been
necessary for researchers in performance appraisal to rely on agreement among raters or to
develop special study designs that produce more than one measure of performance. Campbell
and Fiske (1959) proposed the multimethod-multirater method for the purpose of determining the
Performance Appraisal Method for Software Engineers
20
construct validity of trait ratings. When evaluators are utilizing the approach, two or more groups
of raters are asked to rate the performance of the same employees using two rating methods.
Examples of methods include BARS, graphic scales, trait scales, and global evaluation.
Convergent validity is demonstrated by the agreement among raters across rating methods;
discriminant validity is demonstrated by the degree to which the rates can distinguish among the
performance dimensions.”
When it came to industrial or technical performance analysis, it was impeccable. The
methodology included great strengths. The first advantage was the focus on the desired
behaviors. The individual had an explicit understanding of his duties, requirements, and
standards. The understanding derived from specific duties, responsibilities, and title. The scale
was specific to the job.
The BARS approach had many disadvantages as well. The most poignant are how timeconsuming it was to establish a quality survey that required a diverse parameter. The survey
often left very little subjectivity. For some people, they disliked the format since it required
people to quantify behavior or skill in numbers. The very process can be dehumanizing
(Brynjarsdóttir & Sigvaldason, 2016). The BARS approach can cause problems if the evaluator
discards the behavior descriptions. The approach also had issues with the complexity of
behavior. Evaluators could grossly simplify elaborate situations. Regardless of how the quality
of the survey, it required a high cost since it required significant participation from the supervisor
and the necessity for training (Murphy, Grynoviki, & Kysor, 2001).
For the second year, the organization implemented the 360-degree feedback. The
advantages of the 360 approach are the improved social relations. Many people can talk about
the reviews of others. There are more rounded profile of the employee which reduced
Performance Appraisal Method for Software Engineers
21
discrimination risk. The approach emphasizes team development and identifies negative
behavior. It emphasizes personal and organizational performance development which provides
the employee responsibility for career development. For many new employees, there was
training needs assessment (Heathfield, 2016).
There were many disadvantages the 360 approach that resonated with employees. The
approach had considerable subjective data but failed to quantify each employees’ performance.
The performance review failed to include accomplished objectives from employees. The process
was improved but still proved ineffective since it focused on negatives and weaknesses. With the
high amount of experience, that employees were providing there were exceptional expectations.
For less experienced employees, the approach was ineffective. It led to the downfall of the
design process and disconnecting the appraisal process. There was insufficient information to
align behavior and objectives properly. For the short term, it had a negative impact on product
development since many employees had disagreements about the performance review. The
appraisals provided no reasons for employees to change. It had increased frustration and
decreased morale (McQuerrey, N.D.). After the review, employees focused on contributing to the
minimum applicable work since management did not rate additional qualities of work. When
employees completed the reviews, it led to a data entry overload to personnel (Heathfield, 2016).
The new approach was more appropriate for the team because it organized objective and
subjective data. The organization needed a methodology that grasped the larger picture then
analyzes the individual elements of employees’ conclusions. It included the business and
technical objectives, impact of work, and relationships within the office.
Maintaining 360-degree reviews existing process were not worth the cost to benefits.
The process upset many people. It failed to scale for many enterprise organizations. Since
Performance Appraisal Method for Software Engineers
22
was still small, there was a limited margin of freedom to allow the organization to
experiment with appraisals. The company needed to transition from startup phase to growth
phase quickly because of massive workloads. The process needed to support additional
tools/services that provided radical growth of the app.
The lower performing employees were losing morale since the work was very difficult
while high performing employees were unsatisfied with the work environment. There were
rumors that three engineers may be leaving after receiving offers from large software firms.
The previous system involved an annual 360 review consisted of three peers and a
supervisor. Many people had issues with it. “Performance appraisals are very expensive,
complex systems for making people unhappy,” Kevin Murphy (Zillman, 2016). It will take
weeks or even months when the organization reaches 250,000 employees. For a vast majority of
people, it has led to the “dreaded annual review.”
Unlike traditional methods,
utilized very bleeding edge methodologies,
strategies, and techniques. The methodology the CEO was proposing involved quantifying
emotions and qualifying objectives through internal and external measurements. Changing the
review was not merely about peer and supervisor approval nor is it creating an evolutionary
algorithm. The appraisal was about creating value for the customer, the team, and the
organization. The 360 review failed in many aspects, particularly for software engineers.
Supervisors had adjusted the analysis to compare to the market performance (impact of the app),
programming performance over industry average (regarding the breadth of knowledge and
objective performance), and the impact to the team. Further details of the analysis are available
later in the document. Through a 5-minute interview and survey, we generated a critical
performance appraisal. However, chiefs and directors can provide direction and evaluate the
Performance Appraisal Method for Software Engineers
23
progress of the employee. The reason it was significant was that the approach evaluated
engineers how they wanted to be evaluated and managed.
Project Plan Summary
Since the organization utilized the latest technology, it was imperative to support
employee development since there were few opportunities for training. As stated before, the
only option for many high performers was self-evaluation. Peer and critic evaluations could also
be useful, but limited. For
it was critical to take conferences or training seminars.
Many within the organization had no interest in hiring consultants or coaches.
The implementation of the plan falls into seven milestones.
1. Implement policies for promotion and termination.
2. Reform office policies.
3. Revise the appraisal and change metrics for employees. Create a check-in report with
accomplishments, relations and career goals. During the milestone the directors will refine
measurements for competency, quantified skills that affect business objectives, qualified skills
that affect scientific objectives, and convergent skills that affect engineering objectives.
4. Adjust compensation package to reflect changes in policies and performance.
5. Create supervisors reports for the needs of various stakeholders.
6. Utilize the team performance reviews to create a performance index.
7. Hold impact conference and celebrate victories.
The reason it was carried out in this particular manner is that many tasks could not run in
parallel. The PERT chart later in the chart provides further details of tasks.
Performance Appraisal Method for Software Engineers
24
Quality Assurance Summary
The first method for evaluating the performance appraisal was the improvement in the
engineers' quality and quantity work over two quarters. When directors were evaluating
employees over quarters and years, it would demonstrate the scorecard analysis benefits. With
the temperature metrics, the report even identified the micro-fluctuations in performance. With
enough provided data, it could even predict downturns. Utilizing specific indicators on an
engineer when she gets in the “zone.”
The next tested method for the solution was the app's improvement. There were several
measurements to improve the app. The primary measurements for the app were explicitness,
release rate, reliability, resource management, scalability, and user acceptance. Explicitness
described how well a project could be managed and understood. When the team utilized the
proper code organization, communication, readability, and source code version control, then the
project was significantly easier to manage. It increased the app’s market availability when the
app had excellent resource management. It included compressing assets, minifying source code,
minimizing stress, providing strategies for the client and server side, and other optimization
methods. When the team lowered the system requirements for the app, it allowed lower end
devices to install the app. The release rate described how fast the team could provide a new
update or version of the app. It did not necessarily mean quality, but rather producing the least
viable product in the shortest period. Release rate only qualified the minimum standard.
Reliability described how stable an app is. Historically, engineering software required very strict
standards for a release. The standards often needed less than 1% errors or the app would not
compile or could even damage hardware. Within the last 20 years, there were great strides that
allowed developers, end users, and organizations to produce more fault-tolerant source code.
Performance Appraisal Method for Software Engineers
25
The fault-tolerant code does not mean the software will work correctly. The device will tolerate
the errors without producing fatal errors. Scalability described how well an app could scale from
Nano to an enterprise. There could be apps that work very well on a small level, but when it
scaled to 250 users, it would crash a server. The same could be true for reducing the scale of an
app. The most important measurement was user acceptance. The measurement had several
shades within it such as activity, acceptance, downloads, active users, and total users. Each
measurement had a different major stakeholder, but every stakeholder cared about each
measurement.
Post Implementation Summary
Sustaining the performance appraisal process was accomplished by providing check-ins
with supervisors through the dashboard for individual support and regular meetings covered team
performance. If an engineer felt he was struggling, then he could request coaching sessions on a
monthly or quarterly basis.
Finance, accounting, HR, and payroll & benefits were needed for additional support on a
regular basis. There were aspects of integrating the HRM and FA software with the training and
development software. There are manual aspects to integrating the data between systems. The
AWS Cloud server was an existing purchase for the project at $800 per year.
The appraisal metrics and methods started low incrementally improved over the two
periods. The directors and employees ensured their duties were assigned in line with the
requirements and specifications of the stakeholders and still within the employees’ job design.
The maintenance plan is to maintain the conditions for termination, retention, promotion,
and compensation packages processes. HR ensured the CEO and directors that the review
processes were utilized and enforced.
Performance Appraisal Method for Software Engineers
26
Needs Analysis
The problem is to produce an innovative method that addresses objective and subjective
perspectives that will please management and employees, identifies bottlenecks in performance,
and isolates and correlates performance to impact. The methodology will ultimately improve the
performance of the employee, supervisors, and the product.
Most appraisal methods are more than 60 years old. They were designed to assess the
organization's longevity and sustainability. Most operated under the premise of a simple
quantification to determine a ratio. Specifically, what is the ratio of the employee's contributions
to revenue and savings compared to the cost of the employee? Since many organizations do not
last longer than five years it was in the employer's best interest to calculate the bottom line. The
next aspect is how employees would perform over the year. There is an average of 2% inflation
per year. The organization would need all employees to improve by 2% to maintain the same
operations as last year.
The majority of existing performance appraisal processes were not appropriate for
We needed a solution that described the complex and dynamic environment of
software engineering. During the industrial revolution, performance appraisals were linear with
minimal variables. Software engineering is not similar to laying railroads. Software engineering
requires several qualified skills that translate into quantifiable value. Even when programmers
create an app that is correct according to engineering and science, the market and public may not
understand the value; thus, their compensation package may not reflect required task complexity,
difficulty, and effort. It was in
interest to fulfill the needs of its customers.
Providing features that customers did not need or understand would not improve commercial
performance.
Performance Appraisal Method for Software Engineers
27
The first year BARS was used. The organization used BARS because the team needed to
focus on the development of the product. There were limited competitors for alternative
currencies, but the government requires very strict rigorous testing.
utilized know
your customer (KYC) rules for auditing and a variety of security compliances. The BARS
approach met with moderate success regarding business and technical objectives but failed to
gather subjective data and information adequately. The BARS assessment provided a more
objective assessment of the employees’ skills needed for establishing the app’s performance
requirements.
The second year 360 degree reviews were used. The team utilized the 360-degree
reviews because the product was stable and performed very well. The directors felt that team
members were not acting as a team within the organization. It was unable to determine whether
there were conflicts and power struggles between employees and directors. Overall, the reviews
interpreted as a moderate success for (in terms of) subjectivity but failed poor at gathering
specific performance objectives. The 360-degree review provided more soft information but
neglected much of the objective data.
For the organization to emphasize long-term success, an assessment method that captured
critical data associated with various stakeholders was required. There were three major types of
evaluation values within computer science. The first value was scientific measurements. These
aimed to create high impact models or simulations to (dis)prove an argument e.g. case studies,
research, or simulations. Scientific values are implicit or theoretical. It was comparable to
abstract potential currency. The next was engineering measurements. The developer created
high-performance software with the least amount of resources. It has efficiency or productivity
value. The last was economic measurements. These measurements created high (fiscal) value
Performance Appraisal Method for Software Engineers
28
software with the minimal amount of resources. For many engineers with traditional computer
science backgrounds, the objectives are to create programs that are scientifically or technically
accurate. They analyze the requirements, specifications, and documentation to determine the
exact product they are going to create. When a client, customer, or manager alters the
specifications or moves the scope just a bit during development, it can cause the product
impossible to complete within the provided time. For
we utilized all three value
systems for internal and external customers to ensure success on all boundaries.
had utilized an alternative currency model that had never existed. There was no
scientific research for the duration the organization has achieved. There had been a few cases
where scientists and a community tried it for a month with success then ended the project. The
validity of the data needed to ensure the safety of all the customers.
The engineering value must ensure that the app will run on all major operating systems
and devices since customers come from different backgrounds. Some customers have used
phones that are six years old. Optimizing the application for backward compatibility and
performance is an issue. Lastly, without considering the fiscal implications, the organization
would have to close its doors.
Need for the Solution Analysis
The need for the solution occurred from the massive amount of negativity and stress
associated with appraisals. Most people hated appraisals (Wakefield, 2015). For the
organization to maintain the bottom-line, we need to rely on data and less on opinions. To
successful implement the strategy required having less biased information and leveraging
objective data. According to Rummler and Branch (1990) “linking performance to strategy
separates successful from unsuccessful firms.” If an employee destroyed the morale for a large
Performance Appraisal Method for Software Engineers
29
portion of employees and created a hostile work environment, then the person needs to be
reprimanded, displaced, or removed. The team created their best method to generate the data and
information to support the process was with a multimethod-multirater appraisal. The objective
data was inadequate for management, and subjective data was insufficient to employees. The
new appraisal method included the employees’ impacts to various stakeholders and more
concrete work performance metrics. The most successful organizations such as Adobe,
Facebook, Google, and SAP utilized similar methodologies. The organization needed to create
an office environment that supported it. If the organization developed superior employees, then
a superior product could be created. If supervisors had concrete and subjective data to identify
micro and macro issues within their departments, then a strategy and solution can be created to
address the issues.
The primary stakeholders in the project were the various software engineers and the
directors of development and quality assurance. The project was critical to employees and
supervisors positions. The project also had secondary stakeholders such as customers, investors,
and public opinion. A portion of customers and public opinion cared about the employees’
quality of life. The three investors were
and
The
investors’ primary concerns were the health of the app, the public opinion regarding attracting
high caliber talent, and the employees’ quality of life.
The 360 review process had produced more problems than solutions. There were other
methods for quantifying contributions from employees to market performance. The performance
process had decreased the value of the employees even though many employees were long
standing employees at large firms. The process had failed to show the objective performance
between our team and the industry standard.
Performance Appraisal Method for Software Engineers
30
There were high costs involved with hiring a new employee. The figure below were the
costs associated with hiring a new employee. Fiscally, it was more advantageous to maintain
employees for as long as possible.
Figure 3
Problem Statement
The problem was for the team to produce an innovative method that addressed objective
and subjective perspectives that pleased major stakeholder perspectives, identified bottlenecks in
performance, and isolated and correlated performance to impact. The methodology was required
to improve the performance of the employee, supervisors, and the product.
Performance Appraisal Method for Software Engineers
31
Different engineers have different requirements and specializations. Measuring a two
different types of engineers with the same weight of work with the same metrics would be unfair
for one of the engineers. Comparing a front end and security engineer is drastically different.
Within
there were several specializations.

Artificial Intelligence

Configuration Management

Contemporary Business and Industry Systems

Data modeling for Accounting, Economic, and Financial systems

Geographic Information Systems

Information Designs and Systems

Interactive Designs and Systems

Operation Systems (DevOps, SysOps, and SecOps)

Quality Assurance

Security Engineering*
Security engineering did not include areas of expertise that were covered in a security analyst or
a security technician position. Security engineering described the ability to construct the tools
that a security analyst or technician would utilize. A full description of the skills are found in the
skills matrix attachment.
Problem Causes
There were four causes that made the issue a serious problem. The first issue was the
process of retention, promotion, and termination did not accommodate the 21st-century
application development environment and development. The process implied the organization
Performance Appraisal Method for Software Engineers
32
consistently rewards employees who exhibit good behavior and performance. The combination
(of behavior and performance) was proportionate to the compensation. An impressive objective
performance might not even qualify a pay raise. At some organizations, an employee could earn
an organization a significant profit without evening receiving any appreciation.
The second cause was the current methods for determining employee performance and
development were not effective in the situation. Each year stood alone with no correlation from
one year to the next (Sullivan, 2011). Employees’ performance seemed to exist in a state of
stasis.
The third cause related to the second cause which was that no process was in place for
quantifying and qualifying the employees’ contributions. From the previous
the team
inferred the organization's need for a method to gather objective and subjective information,
analyze the information into the desired results, create actionable tasks based on the desired
results, and provide high impact value to the employees, team, and customers.
Another cause was the supervisors need more awareness of the app (objective, user
acceptance, and market performance), the supervisors' themselves performance, and the team’s
performance. Quantifying and qualifying employees’ performance was not the only purpose of
the performance appraisal. It was to utilize the information into an actionable task and generate
massive value. If the director of development and the director of QA possessed the additional
tools for the app performance, it would have provided greater insights into the team’s
performance. While technical app performance was important, it was more important for them to
create features and build (a version of the app) based user acceptance reports. User acceptance
was the interpreted anticipated market performance and estimated employee compensation.
These reports were double edged swords to the supervisors as it was the leverage for the team,
Performance Appraisal Method for Software Engineers
33
but also part of the evaluation of the supervisors. It would provide the most visibility to impact
value. From the previous reports, the team generated a derivative report of the team’s
performance.
Consequences
There were dire consequences if the appraisal method did not change. The first was that
employees would not know the conditions for promotion. Even good employees risked
termination or leaving. There is the possibility that employees received the wrong compensation
package because directors or peers poorly evaluated the employees’ abilities or relations.
Without any concrete measurements, the supervisors limited knowledge how the team was
performing, company development, or sustainable operations.
The second issue was without utilizing a multimethod-multirater appraisal it would not
identify the advantages, disadvantages, and needs of all of the stakeholders. Each method had
only identified the issues for a specific portion of each group. The BARS appraisal worked well
for the engineers. The 360 appraisal worked well for the directors. Neither method thoroughly
addressed the customer. Without measuring the organization’s performance against the customer
implied the organization might not maintain longevity.
The third issue was that neither method emphasized employees’ accomplishments. Even
an average performance appraisal came across as negative for the teams. Engineers had a
tendency to avoid conflict and confrontation. The new solution addressed the issue by
identifying individual’s accomplishments, providing others with understanding, recognizing the
accomplishment, and validating it to all stakeholders.
Performance Appraisal Method for Software Engineers
34
The work environment would have stayed the same without implementing office reform
policies to support the procedural changes. Negativity was an infection that needed to be
identified, addressed, treated, and removed.
The last issue was the directors need performance data that is accurate and relevant.
Neither method previously addressed it effectively. 360 reviews took time, and the BARS
review was good for quick assessments but poor at in-depth data.
Stakeholder Impacts
The performance appraisals affected all the software engineers in the app development
and QA departments. It directly measured the policies, procedures, and consequences of
performance appraisals. It determined how the organization retains, promotes, and terminates
employees. It also determined employee morale and the elements of a good work environment.
Performance appraisals were such a far-reaching issue because it strongly affected the
performance and longevity of the organization. Annual reviews were better than nothing, but it
failed to create innovators or market leaders. A continual feedback loop between performance
and market performance were required, otherwise; the company could have suffered from not
identifying the market environment. Ultimately causing the business to let go of all employees
and close the doors.
The annual reviews from the 360-degree reviews included major goals for the year and
audit checklist, but there was no (measured) improvement for technical abilities over the year. It
was difficult to assess whether employees improved, worsened, or maintained performance.
Their motivation drastically decreased. Employees were reconsidering utilizing their skills for
social work and working at the organization in general. The employees were very frustrated with
the directors. Employees requested to telecommute to avoid working physically with directors.
Performance Appraisal Method for Software Engineers
35
Some started questioning the CEO's leadership by allowing them to utilize the 360 reviews. The
employees who received lower performance scores had self-doubt, and many of the high
performing employees were resentful toward the directors since it ruined the work environment.
Trust among some employees even thinned, and teamwork plummeted. It took several months to
repair the damage between employees and the directors.
For the directors, they felt the BARS appraisal neglected many soft skills. The appraisal
did not include attitudes, communication, ethics, paradigm, perspective, or teamwork. It
included hard skills like data structures, documentation, and network security. Without soft data,
it made it difficult for the directors to assess the impact on internal customers. The marketing
department frequently complained about the communication style of some of the developers.
The development team’s best engineer often exhibited a pessimistic attitude. The operations
engineer held a drastically different paradigm compared to the directors and the CEO. The
organization had strict JavaScript and TypeScript policy. The operations engineer had utilized
Java to complete a few DevOps tasks. The problem was it created a huge security gap. The
security analyst did not anticipate the server using Java and had to temporarily pause the sessions
for all users without crashing the servers.
For customers, it was about producing a better app. Most didn’t care who made it better.
At the same time, it was difficult for them to describe what feature or experience is better until
they see it.
Other employees are not directly affected, but the improvement of the engineers does
affect them. The data scientist, full stack developers, and operations engineer take requests from
accounting, finance, and marketing when time permits. The soft skills affect internal customers
much more than the hard skills.
Performance Appraisal Method for Software Engineers
The three investors are
36
and
The investors’
primary concerns are the health of the app, the public opinion regarding attracting high caliber
talent, and lastly the employees’ quality of life. If the directors cannot manage the employees
effectively, then they need to realign with the organization’s mission or be let go. From the risk
perspective of the app, it is more important to keep the ten engineers than the two directors. It
agitates them to hear how poorly the appraisal went particularly when two of the engineers have
previous ties with
is more fortunate than the
regarding how fast rumors travel. If
(in CA) or
gains a reputation for having a poor
leadership and works environment, it will take paying new engineers at least another 10% higher
salary to get them on board.
For potential recruits to see any bad reviews on Glassdoor, Indeed, or Payscale could
have adverse effects considering the small size of the organization. No one wants to work in a
negative work environment. If an employee released a review (of the company) that described
appraisals as “Dilbertesque” and the environment paralleled IniTech, then the appeal of the
organization would substantially decrease.
Problem Analysis
Cost
The figure below are the various employees’ salaries. The salaries were the average of
the reported salaries from Glassdoor, Indeed, and Payscale.
has ten different types of
developers including the director of development and the director of QA.
Performance Appraisal Method for Software Engineers
QTY
1
2
1
1
2
1
2
1
1
1
Position
Front-End Developer
Back-End Developer
Data Scientist
Cloud Systems Architect
Full Stack Developer
Director of Software
Development
Information Security
Engineer
Artificial Intelligence
Developer
Ops Engineer
Director of QA
Total
Salary
37
Per Paycheck
Per hour
Per 30
min
Per 15
min
$90,024
$86,977
$71,172
$124,297
$105,658
$3,215.14
$3,106.32
$2,541.86
$4,439.17
$3,773.49
$40.19
$38.83
$31.77
$55.49
$47.17
$20.09
$19.41
$15.89
$27.74
$23.58
$10.05
$9.71
$7.94
$13.87
$11.79
$132,114
$4,718.36
$58.98
$29.49
$14.74
$103,078
$3,681.37
$46.02
$23.01
$11.50
$97,323
$112,441
$133,162
$1,056,246.00
$3,475.83
$4,015.76
$4,755.77
$37,723.07
$43.45
$50.20
$59.45
$471.54
$21.72
$25.10
$29.72
$235.77
$10.86
$12.55
$14.86
$117.88
The previous 360-degree appraisal method required eight hours to develop. The first four
hours were two meetings from both directors to convince the CEO to change the appraisal
method. It cost the company $236.85. The next two hours were two independent hours where
both directors developed their 360-degree appraisal form. The last two hours was a meeting
where they took portions of eachothers’ form to develop a final form. The total development
cost was $473.71.
For implementation, each review peer and supervisor reviews took 15 minutes, and a
self-review that took 30 minutes. Each employee took approx. 90 minutes for evaluations and
another 30 minutes to go over the review. The review cost for the team was $905.33. After the
review, each engineer had a meeting with the directors to cover the results. The meetings took
30 which cost the company $235.77. The total cost of implementation was $1,141.10.
After the review, the CEO had met with the AI, back-end, both full stack, and operations
engineers. Each meeting took an hour to complete. The meetings cost $226.81. The CEO had a
15-minute meeting with both directors. It cost the organization $29.61. The total for the fallout
Performance Appraisal Method for Software Engineers
38
meetings cost was $256.42. If the negative morale reduced productivity by 6% over three
months, then it may have cost the organization approx. $1.8 million dollars.
Risk
The risk registrar table is below. It contains the risks related to ability, motivation, and
work environment. If the engineers had been developing skills over the last year that were not
less relevant to their position, then their ability and effectiveness would naturally decrease.
Motivation was one of the primary metrics for measuring performance. The motivation
for many
employees was low for almost a quarter after the second year appraisal. If
employees were not passionate about their work, then they would only work minimum amount to
keep from being terminated. With decreased productivity, it had the potential to cause delays in
feature releases or the quality of the app.
A decrease in the work environment quality could come in two ways in the office. The
first was a consistently low motivated group of employees. The other was the management
support for the employees.
The most valuable aspect of the business was the vast amount of talent. “Human capital
includes the skill sets of an organization’s workforce, the depth of expertise and breadth of
experience.” The loss of top-tier talent is very difficult to qualify and quantify. A single talented
engineer could produce a simple widget or app that could give the organization a million dollars
(Spolsky, 2009). The engineers had created frameworks, libraries, and tools that produced
massive morale or productivity.
Performance Appraisal Method for Software Engineers
Risk
Decrease in
ability
Decrease in
motivation
Decrease in
productivity
Minor product
quality
decreases
Major product
quality
decreases
Severe product
quality
decreases
Work
environment
quality
decreases
Source
Likelihood
of
Occurrence
Severity
of Impact
Risk Controllabilit
Factor
y
0.4
0.12
High
0.5
0.125
High
0.3
0.06
High
0.2
0.036
Medium
0.09
0.4
0.036
High
0.0009
0.9
0.0008
High
0.1
0.2
0.02
Medium
Performance evaluation is measuring less
0.3
relevant skills
Employees receive negative or
0.25
contradictory reviews
If ability or motivation decrease then the
0.2
productivity will decrease.
If productivity decreases then the product
0.18
quality may decrease.
If productivity decreases and seven or
more employees quit then the product
quality may decrease.
If productivity decreases and seven or
more employees quit then the product
quality may decrease.
If directors are incorrectly supporting
employees and team work qualify
decreases then the work environment
quality would decrease
39
The second area of risk was the longevity of human resources. Under the assumption that
all employees disliked the new evaluation system that they walk out and it required 30 days to
find new employees. The company would stand to lose approximately $1.3 million at our
current rate of growth. While the tangible loss in value is terrifying, the loss of the intangible
loss is possibly more palpable. The three major aspects of the loss are the loss of human capital,
relational capital, and structural capital (AICPA & CIMA, 2012). The other possibilities are
whether the directors quit the organization or investors pressured the board of directors to
terminate the department directors. Additional details about human resource loss are available in
the table below. The asterisks indicate that the standard formula for calculating turnover was
unavailable since there has yet to be an employee who has quit or terminated.
Performance Appraisal Method for Software Engineers
Risk
Likelihood
of
Occurrence
Severity of
Impact
Risk
Facto
r
Controllabilit
y
0.1125
0.3
0.0338
Low
0.03375
0.5
0.0169
Medium
0.016875
0.7
0.0118
High
0.0084375
0.9
0.0076
High
0.0263
0.3
0.0079
Medium
0.01315
0.5
0.0066
Medium
If the issue isn't resolved in an
appropriate manner then investors could
pressure the director(s) to be terminated
0.1
0.3
0.03
Low
If the issue isn't resolved in an
appropriate manner then investors could
pressure the director(s) to be terminated
0.02
0.5
0.01
Low
Source
One engineer
quits within a
month*
Three engineers
quit within a
month*
Seven engineers
quit within a
month*
Ten engineers
quit within a
month*
One director
quits within a
month*
Both directors
quit within a
month*
Investors
pressure the
board to fire a
director
Investors
pressure the
board to fire
both directors
40
If morale exceeds threshold and/or
employee(s) receive offers from other
organizations then they may quit.
If morale exceeds threshold and/or
employee(s) receive offers from other
organizations then they may quit.
If morale exceeds threshold and/or
employee(s) receive offers from other
organizations then they may quit.
If morale exceeds threshold and/or
employee(s) receive offers from other
organizations then they may quit.
If morale exceeds threshold and/or
employee(s) receive offers from other
organizations then they may quit.
If morale exceeds threshold and/or
employee(s) receive offers from other
organizations then they may quit.
The third area was legal and reputational capital risks. The legal possibilities were
whether an employee retaliated against their director or the organization. It was very unlikely
with the attitudes and culture, but not impossible. The next aspect to consider was when the
CEO met with directors and managers, if there were a code of conduct incidentally violates a law
such as when employees are requested to work extended periods of time. An event happened to
a large software firm that unknowingly violated a local ordinance. The HR manager and legal
counsel must edit the code of conduct. If an employee finds the work environment hostile, then
the employee could place a lawsuit against the organization. The last aspect was the relationship
Performance Appraisal Method for Software Engineers
41
capital from employees. “Relational capital includes all the relationships that exist between an
organization and any outside person or organization. These included customers, intermediaries,
employees, suppliers, alliance partners, regulators, pressure groups, communities, creditors, and
investors.” Similarly, many of the employees had strong ties with the West Coast software scene.
If an employee left a negative review of the organization on social media or networking site, then
it had the potential to damage the public reputation of the company and take longer to obtain a
quality employee.
Performance Appraisal Method for Software Engineers
Risk
Source
Likelihood
of
Occurrence
42
Severity
of Impact
Risk
Controllability
Factor
If the employee is disgruntled then the
HR violation
individual could retaliate against the
0.005
0.4
0.002
Medium
director(s).
If the HR manager and legal counsel
Code of
don't properly analyze the code of
conduct violates
0.015
0.7
0.0105
High
conduct standards then there could be a
a law
legal violation.
Lawsuit from If an employee's morale reaches too low
an employee for and he can prove that the directors
0.005
0.7
0.0035
Medium
a hostile work created a hostile work environment then
environment
the organization could have a lawsuit.
Negative
An employee with low motivation and
reputation for work environment could leave a bad
0.6
0.2
0.12
Low
poor leadership review for poor leadership.
Negative
An employee with low motivation and
reputation for work environment could leave a bad
0.3
0.3
0.09
Low
poor work
review for bad work environment.
environment
A lawsuit for a hostile work environment had the potential for massive damages. Below is
weighted decision tree analysis of the outcome of a trial.
would be looking at a
minimum of $50,000 in legal fees. At the bottom of each column is an estimation for each
outcome.
Performance Appraisal Method for Software Engineers
43
Problem Resolution
The proposed solution had three major objectives.
1. A mixed qualitative and quantitative method to evaluate software engineers’ work
2. Utilizing and leveraging the evaluation data to improve the employees, supervisors,
and customer experience
3. A process for termination, retention, promotion, and compensation packages
It was designed to minimize and eliminate the stress of performance appraisals, evaluate
employees in holistic and relevant manner, guide employees within a supportive and transparent
work environment.
Best Approach
The first step was to separate the processes of termination, retention, promotion, and
compensation packages from performance evaluation reviews. Employees needed to know the
conditions for a termination. It was poor performance for three consecutive months or four
months out of the year. The specific measurement was the average of the employee’s
performance index is <= 80%. There may be extenuating circumstances for delaying a
termination or even a gross violation of the code of conduct such as if an employee copied the
organization’s files onto external hard drives then sold them to competitors. The organization
would have no choice but to terminate the employee and pursue legal action.
Employees also needed to understand what behavior was promoted within the
organization. Establishing behavior helped develop the organization’s culture. It also provided
employees warnings.
The promotion process would be similar to the termination process.
Employees are provided three months before the position being available to the public. The
individual would need at least an 80% performance index, the knowledge and skill of the new
Performance Appraisal Method for Software Engineers
44
position near industry level standards, and the approval of his subordinates, peers, and a
supervisor. The employee would submit an assessment and recommendation for the position.
The assessment would ensure the employee’s abilities compared to the industry standard. The
recommendations would be based on a voting system from the team. It would rank employees
position from one to five. One being an entry level employee and five an executive level. If the
employee earned 80% support, then the position would be theirs. If two employees both applied
for the same position with the same score, then a three round competition is implemented. The
employee would provide the most significant contributions to each stakeholder. The first round is
for the organization, the second is the team, and the last is the public. The winner of two of the
three rounds gets the position.
The last process was to adjust compensation package to reflect changes in policies and
performance. The organization’s compensation philosophy is committed to compensating
employees that are fair, consistent, reflective of the market leaders, and provides recognition and
validation for each’s achievement of personal goals, corporate objectives, and professional
competency. Specifically, our objectives were internal equity, external equity, compliance with
laws and regulations, and increased performance and productivity. The compensation package
consisted of four types: new employee, quarterly, annually, and executive. The new employee
compensation package includes base pay, benefits, and perquisites. The organization will
provide dental, health, accidental and life, optical, and legal insurance. Employees are
immediately provided two weeks of vacation with one week of sick time. The office offers
several indirect incentives that would appeal to recruits.

Carpooling and Transit programs

Contests
Performance Appraisal Method for Software Engineers

Dry cleaning and laundry services

On-site gym

On-site paid meals

Optional 30 minute or 60 minute lunches

Relocation & Office package

Stress management package

Telecommuting when needed
45
The quarterly incentive plan included access to conferences, performance bonus, profit
sharing, and paid certification exams and materials. These would be fully-paid, relevant, and
nearby conferences. Pay increases and bonuses would occur on a quarterly basis. The option for
profit sharing would be after the first performance appraisal. Employees would also have paid
certification, training, and an open library.
For the annual incentive plan, the company hosted a party during the one week fully paid
vacation with family members. During the party, the company released a video of employees’
greatest achievements over the year. After the vacation was over, a publication was released for
the achievements (without disclosing confidential information). For the employee that had the
best overall performance in either department over the year received an award. The best of the
best (Bot B) award provided an employee an entire month to create their new feature or project
with the boards of directors’ approval.
The executive incentive plan was a four-year plan of the company’s objectives. For the
executive incentive plan, the organization offered degree programs, equity, return on assets, and
stock earnings.
Performance Appraisal Method for Software Engineers
46
1. Reform office policies
a. Remove anonymous reviews and provide a 100% transparent environment in the
office between employees. If employee A negatively complained about their
supervisor to employee B, then it was employee B’s responsibility to bring it to
the supervisor’s attention. At the same time, if a supervisor complained about
employee A then they must inform them.
had a zero tolerance
retaliation policy for either side.
b. It was the supervisor’s responsibility to ensure that the employee was continuing
to meet expectations. A portion of the supervisor’s performance review was to
demonstrate continual improvement to his employees. If the performance was
20% less than industry standard, then the supervisor is punished, and if it is higher
than industry standard, then the supervisor is rewarded. The supervisor will
ensure the employee has good morale as happy employees perform better
(Revesencio, 2015). These meetings were designed to provide direction,
feedback, and guidance to the employee. It had helped establish boundaries,
objectives, and explicit goals. The system established standards and performance
to compensation rewards. The performance modifications were altered to match
the scrum project management style. The teams had two-week sprints to
accomplish goals. Every six sprints, the directors performed in-depth reviews of
employees’ performances. It functioned as a refinement process. The directors
and employees determine the standards, establish the objectives for the sprint, and
the supervisor coached the employee to match performance to the standards.
Performance Appraisal Method for Software Engineers
47
After the submission, the supervisor measured the employee’s performance and
compensated him appropriately.
c. Employees were provided a three-month opportunity to apply for a new position
before the position was released to the public.
d. The company provided the employees with their personal choice of technology
for computer, monitor, peripherals, and software.
e. The organization provided any relevant and necessary training. When
organization provided ongoing education, it increased employee retention
(Florentine, 2015). The organization promoted extended certifications, training,
and scholarships for school.
f. New employees were provided an on-boarding program to get them up to speed
as soon as possible.
Performance Appraisal Method for Software Engineers
48
g. It shortened the required amount of time for the evaluation and increase the
number of evaluations to four times per year.
h. Evaluators were trained to identify, distinguish, and analyze contributions.
i. Utilize proactive talent retention methods. The organization already provided
benefits such as 401k, dental, life, medical, and optical insurance. It needed to
provide superior recruiting and retention packages that included carpooling, office
meals, laundry services, telecommuting, and relocation. Finally, HR regularly
scheduled stay/satisfaction reviews to determine employee satisfaction.
2. It needed to create an employee performance review that analyzed the activities and
behaviors. The guiding metric for the appraisal was:
Performance = Ability * Motivation * Environment.
The core of the formula was based on expectancy theory, Theory X, Theory Y, Theory Z,
and Vroom’s equation. There are still other aspects within it that utilized additional
theories such as Management by Objective, SMART theory, and Herzberg Two-Factor
Theory. The CEO abhors micromanaging so he has left the discretion of the directors to
manage their departments. Depending on the employee position, it could use different
weights to each metric. Engineering objectives are less important to a front-end engineer
than an operations engineer. The security engineer provides revenue in the form of
customer trust and legal compliances that are mostly qualified objectives. It was the
responsibility of the directors to decide the specific weights for each area.
a. The first primary measurement was an engineer’s ability. It was measured as
A=W 1∗K +W 2∗S+W 3∗R+W 4∗V
Performance Appraisal Method for Software Engineers
49
Where:
A = Ability
W = Weight
K = Knowledge
S = Skill
R = Response
V = Value
When an evaluator fills each variable, the metric will measure a percentile.
In most cases, an employee’s range is between 80-100%. For the
organization and employee to produce the most value, the measurement
can exceed 100% where the employee has performed beyond the required
objectives.
i. Knowledge of the engineer was represented with the skills matrix. It was a
measure of several skills that a software engineer utilized. These included
the knowledge, application, breadth, and depth of several skills. The skills
matrix does not include the contributions of the employee. The evaluator
measures an engineer with critical and relevant skills within the skills
matrix.
ii. Skill was based on coding metrics. Alexander (2011) provides many of
the specific metrics designed for specific metrics. For the review, we
utilized offensive skill metrics.
● Points-Measured the overall productivity of each coder on assigned
tasks.
○ Points = Sum (Complexity for all completed tasks)
● Power-Measured the average complexity of the tasks that a coder
completes.
Performance Appraisal Method for Software Engineers
50
○ Power = Points / Utility
● Utility-Measured how many assigned tasks each coder completes.
○ Utility = Number of tasks completed
● Temperature-Measured how “hot” or “cold” a coder is at any given
time.
○ Starting Temperature = 72 at start of period measured (see
Notes)
○ Current Points = Points in the most recently completed
measured interval
○ Previous Points = Points in the prior measured interval
○ Heat Index = (Current Points / Previous Points)
○ Temperature = (Previous Temperature × Heat Index)
● O-Impact-Provided a single “Offensive Impact” number that
summarized the contributions of a coder in moving projects along.
○ O-Impact = Points + Utility + Assists
iii. Response described the various interactions between stakeholders. It
included the team memebers, supervisors, subordinates, other employees,
and customers. Many of the QA engineers had more interactions with
internal customers. It utilized a mix between customer relations
management and business relations management. In the appendix is a
template
Performance Appraisal Method for Software Engineers
51
iv. Value had three major metrics. These were objectives based on business,
engineering, and scientific contributions. Quantified skill contributions
that affect business objectives. These are the skills that were measured
according to the app’s revenue measurement and user acceptance metrics.
Qualified skill contributions that affect scientific objectives. These are the
skills that may not necessarily increase the revenue, but provide implicit
value. Convergent skill contributions that affect engineering objectives.
These are the skills that affect efficiency and productivity.
b. Motivation is the second major factor for evaluating employees. There are some
employees who are very intelligent but have no motivation to work. It is critical
to management to understanding what motivates employees. The employee’s
perspective, competency & mastery, and autonomy are foundations of motivation.
c. The environment is a major factor when evaluating employees. People who are
the best at their discipline are high maintenance. They have to be. When a person
is running on all cylinders, every aspect needs to be in perfect order. The smallest
miscalculation can have dire consequences. Elite software engineers are no
different. Mark Zuckerberg only ate meat he killed and Aaron Swartz would only
eat white food. For the organization to hire and retain the best and brightest, the
organization must offer the minimum environment for elite employees to
function; otherwise, they will leave. Job design, equipment & materials, policies
& procedures, economic conditions, and management support are the five areas
for measuring a work environment.
Performance Appraisal Method for Software Engineers
52
d. Create specific measurements for relations. These are the relationships among
subordinates, peers, supervisors, and the team.
i. Organization
● Senior
● Peer
● Junior
● Client/Customer
● Self
ii. Community
● Customers
● Open-source
● Local
e. Produce a visible impact of the employee contributions. Employees will submit
their contributions on a regular basis and marketing will pull customer content
and reviews.
3. The employee will have a report where they started, their progress since the last check-in,
and a course plan for accomplishments, relations, and career goals.
4. Create environmental changes to support employee growth.
5. The supervisors have a deeper understanding of the department and relevant metrics.
a. Market performance and user acceptance. The supervisors will have information
based on the app’s revenue and the user acceptance.
b. Team’s performance vs. industry standard. The supervisors will have information
based on the software engineers’ industry standard.
Performance Appraisal Method for Software Engineers
53
c. Team’s performance vs. satisfaction. The supervisors will have information based
on the employees’ satisfaction.
d. The performance index to identify bottlenecks in performance within the
department. Based on the team’s performance and the previous three reports, the
supervisor will be able to identify any performance obstructions.
Approach Justification
The reason the organization should separate the processes of termination, retention, and
promotion from performance evaluation reviews is that the event creates a lot of anxiety,
depression, and distress around the time. It is not an accurate account of how the employee
performed throughout the year. In most cases, a manager will only remember a few events
throughout the year to make a decision. Human memory is prone to significant flaws (Arkowitz
& Lilienfeld, 2010).
Office reforms were required to gather necessary information for compensation packages.
The first policy was the organization needed 100% transparency between employees to gather
appropriate information. The supervisors were required to be responsible for the employees’
success otherwise; they were policing the employees and not providing leadership. Employees
also needed to hold themselves accountable for personal management by meeting ability and
relational goals. Supervisors needed to prepare employees for positions that were coming down
the pipeline. It was in the company’s best interest for employees to provide opportunities to
apply for new positions before they release were released to the public. New and existing
employees required training to identify, distinguish, and analyze work contributions.
The reason the employee performance review needed to include activities and behaviors
is that engineering is a very technical position, but it hinges on many relationships. Stakeholder
Performance Appraisal Method for Software Engineers
54
requirements can be ambiguous or nuanced. All employees’ accomplishments and impact were
measured as the performance. It was important for employees to see what impact they have had
on the team and customers.
The supervisors need a deeper understanding of how the app is performing in the
marketplace and user acceptance because otherwise there is no correlation between team
performance and market performance. The supervisor needs an understanding of how the team
is performing compared to the industry standard. It helps keep our team in perspective within
the industry and will answer many questions regarding recruiting. It is also important for a
supervisor to know how satisfied they are in the workplace as employee retention is critical to
development and operations.
The combined performance appraisals generated an overal performance index (PI). The
PI identified bottlenecks in performance within the department through abilities, motivation, and
environmental causes. For the team to create an incredible work environment was more than just
a sleek industrial design, state-of-the-art equipment, snacks, and an open door policy. It was
about all the aspects that affected employees within the environment. For PI, it was about
providing a job that the person was specifically designed for, creating a work environment that
supported sustainability and growth, implementing policies and procedures to prevent negative
elements, and utilizing management to promote positive elements.
Alternative Solutions
While there were many other approaches, the next most appropriate was the Management
By Objectives (MBO) approach. According to Investopedia (2010), Management by Objectives
(MBO) was a management model that aimed to improve the performance of an organization by
clearly defining objectives that were agreed to by both management and employees. According
Performance Appraisal Method for Software Engineers
55
to the theory, having a say in goal setting and action plans would ensure better participation and
commitment among employees, as well as the alignment of objectives across the organization.
The term was first outlined by management guru Peter Drucker in 1954 in his book "The
Practice of Management."
Armstrong (2003) noted that performance management has risen from the old-established
but somewhat discredited systems of merit rating and MBO. Many of the most recent
developments in performance appraisal had been absorbed into the concept of performance
management which aimed to be a wider, more comprehensive, and more natural process of
management. Performance appraisal had too often operated as a top down and largely discredited
bureaucratic system owned by the personnel department, and it limited its intended value
(Akampurira, 2014).
Investopedia continued with an implementation process, MBO called for five steps that
organizations implemented management technique into practice. The first step was to either
determine (or revise) organizational objectives for the entire company.
mission and
vision created the broad overview. The next step was translating the organizational objectives to
employees. Drucker used the acronym SMART (Specific, Measurable, Acceptable, Realistic,
Time-bound) to express the concept. It is important during this time to record and measure
standard operations for engineers. Step three was stimulating the participation of employees in
setting objectives. After the CEO and directors shared the organization's objectives with
employees, from top to bottom, employees would be encouraged to help set their objectives to
achieve these goals. The fourth step was to monitor the progress. In step 2, a key component of
the objectives was that they are measurable for employees and managers to determine how well
Performance Appraisal Method for Software Engineers
56
these were met. Lastly, progress is evaluated and rewarded. The step included honest feedback
on what went well and what did not.
Consequences
There were very critical issues that would occur if metrics and policies failed to change.
The first was that employees would not know the conditions for promotion. There was the
potential for good employees leaving or terminated from supervisors. The employees may have
received the wrong compensation package because the directors may have incorrectly evaluated
employees’ priorities. Without any measurements of the employees accomplishments, the
supervisors will not know how they are performing or if the company development or operations
are sustainable.
Risk Assessment
The risk assessment for the solution was very similar to the risk assessment of the
problem. The risk impacts do not change, but the likelihood will decrease for the solution. The
issues about the decrease in abilities will decrease since it includes a more rounded but more
specific approach. The methodology systematically addressed the major areas of motivation by
utilizing the most accepted theories in business management and psychology. An increased
ability and motivation would lead to improved productivity and product quality. The
compensation and indirect perquisites addressed many of the work environment quality issues.
Performance Appraisal Method for Software Engineers
Risk
Source
57
Likelihood
of
Occurrence
Severity
of Impact
0.1
0.4
Risk
Factor
Controllability
Performance evaluation is
Decrease in ability
Decrease in
motivation
measuring less relevant skills
Employees receive negative or
contradictory reviews
If ability or motivation
High
0.04
0.042
0.5
0.021
High
0.03
0.3
0.009
High
0.006
0.2
0.0012
Medium
0.003
0.4
0.0012
High
0.0015
0.9
0.0014
High
0.005
0.2
0.001
Medium
Decrease in
decrease then the productivity
productivity
will decrease.
If productivity decreases then
Minor product quality
the product quality may
decreases
decrease.
If productivity decreases and
Major product quality seven or more employees quit
decreases
then the product quality may
decrease.
If productivity decreases and
Severe product
seven or more employees quit
quality decreases
then the product quality may
decrease.
If directors are incorrectly
supporting employees and
Work environment
team work qualify decreases
quality decreases
then the work environment
quality would decrease
Performance Appraisal Method for Software Engineers
58
The organization needed to minimize the possibility of employees taking other
opportunities, the organization ensured that they match industry salaries compared to the five
market leaders Amazon, Apple, Facebook, Google, and Microsoft. The organization also offered
additional benefits that most of the other companies except for immediate stock options. Since
is an LLC, stock was more limited compared to an IPO. After eliminating the risk of
employees quitting, the next step was to address the needs of the stakeholder, but highlighting
the impacts of employees. It also provided the organization with a platform for positive PR. The
directors would perform better since the new methodology held both the employees and directors
responsible for personally developing.
Risk
One engineer quits
within a month*
Three engineers quit
within a month*
Seven engineers quit
within a month*
Ten engineers quit
within a month*
One director quits
within a month*
Both directors quit
within a month*
Source
If morale exceeds threshold
and/or employee(s) receive
offers from other organizations
then they may quit.
If morale exceeds threshold
and/or employee(s) receive
offers from other organizations
then they may quit.
If morale exceeds threshold
and/or employee(s) receive
offers from other organizations
then they may quit.
If morale exceeds threshold
and/or employee(s) receive
offers from other organizations
then they may quit.
If morale exceeds threshold
and/or employee(s) receive
offers from other organizations
then they may quit.
If morale exceeds threshold
and/or employee(s) receive
offers from other organizations
then they may quit.
Likelihood
of
Occurrence
Severity
of
Impact
Risk
Factor
Controllability
0.0178
0.3
0.0053
Low
0.003
0.5
0.0015
Medium
0.0007
0.7
0.0005
High
0.0002
0.9
0.0002
High
0.0088
0.3
0.0026
Medium
0.0044
0.5
0.0022
Medium
Performance Appraisal Method for Software Engineers
If the issue isn't resolved in an
Investors pressure the
appropriate manner then
board to fire a
investors could pressure the
director
director(s) to be terminated
If the issue isn't resolved in an
Investors pressure the
appropriate manner then
board to fire both
investors could pressure the
directors
director(s) to be terminated
59
0.008
0.3
0.0024
Low
0.0027
0.5
0.0013
Low
There would not an issue for employee retaliation since it is in the directors’ best interest
to effectively manage and develop employees. Since HR and legal were reviewing the changes,
it was very unlikely that a code of conduct would violate a law. Since the work environment had
changed, there would be very little chance of a lawsuit. The situation would be similar to
receiving a negative review for leadership or work environment.
Performance Appraisal Method for Software Engineers
Risk
Source
HR violation for
employee conduct
or employee(s)
retaliate against
the directors
If the employee is disgruntled
then the individual could
retaliate against the director(s).
Code of conduct
violates a law
Lawsuit from an
employee for a
hostile work
environment
Negative
reputation for
poor leadership
decreases
recruiting appeal
Negative
reputation for
poor work
environment
decreases
recruiting appeal
The project is a
success, but other
employees are
jealous
If the HR manager and legal
counsel don't properly analyze
the code of conduct standards
then there could be a legal
violation.
If an employee's morale
reaches too low and he can
prove that the directors created
a hostile work environment
then the organization could
have a lawsuit.
60
Likelihood
of
Occurrence
Severity
of
Impact
Risk
Factor
Controllability
0.0005
0.4
0.0002
Medium
0.0015
0.7
0.0011
High
0.001
0.7
0.0007
Medium
0.03
0.2
0.006
Low
0.03
0.3
0.009
Low
0.2
0.2
0.04
High
Employee motivation and
work environment
Employee motivation and
work environment
If employees from other
departments find the appraisal
method and compensation
unfairly skewed toward the
engineers then it could cause
unrest.
Cost/Benefit for each risk
Because of limited spacing, the tables are in the solutions.xlsx file. The attached spreadsheet
contains the cost analysis in the “Cost by Milestone” tab. The cost is the separated by milestone
with stakeholders indicated.
Performance Appraisal Method for Software Engineers
61
Within the “Benefit” tab describes the success of the previous analyses. The CEO provided
assumptions for the benefits of the project.
Departments that generate revenue
Development & Quality Assurance
52%
Finance & Accounting
15%
Marketing
33%
Departments that save spending
Customer service
19%
Development & Quality Assurance
17%
Finance & Accounting
21%
Legal
43%
Departments that reduce liability
Finance & Accounting
11%
Legal
89%
Based upon these assumptions are the conclusion of the benefits. The potential benefits of
adopting the project could double the organization’s value.
A quick summary is provided on the described tab. It shows the relationship between the
costs and benefits of the project. For the provided document, a Direct Intellectual Capital
models (DIC) method was used with fair use as described by the IRS. However, when reporting
to investors a scorecard method must be utilized for the next round of funding. Finally,
Performance Appraisal Method for Software Engineers
62
calculating the fiscal value of specific improvements of the performance appraisal is highly
speculative. The CEO had included the gross sales, estimated savings, and risk mitigation.
Risk Mitigation
Below describes the risks involved in the project. Each risk has a strategy, tactic, or method
to manage the issue.
Risk
Decrease in ability
Decrease in motivation
Decrease in productivity
Product quality decreases
Work environment quality decreases
Employees quitting within a month
Investors pressure the board to fire a director
HR violation
Code of conduct violates a law
Lawsuit from an employee for a hostile work
environment
Negative reputation for poor leadership
Negative reputation for poor work environment
Solution
Provide additional training to employees and
revise specific instructions to tasks. Implement a
policy for engineers to not start on any work
without requirements and specifications.
Demonstrate commonality between work and
values, provide meaning to tasks, demonstrate
impact of work, show the value of opinions,
demonstrate purpose to work, offer direction
with autonomy
Differentiate activity to productivity by refining
metrics.
Utilize objective measurements; provide
standards for user-acceptance.
Address each aspect of the work environment
individually. Creating stronger relationships
within the team, resources of choice, limited
policies & procedures, leading economic benefits
& pay, and management support.
Address primary aspects of employee concerns.
Reflect metric changes to revenue
improvements.
Provide code of conduct.
Legal counsel reviews any modifications.
Directors are penalized for creating hostile work
environments.
Directors are held responsible for not developing
employees.
The work environment is substantially improved.
Performance Appraisal Method for Software Engineers
63
Project Design and Requirements
The primary design revolved around the industry’s highest best practices. For
to
become one of the highest performing software studios, the teams needed to determine where
their performance was and the goal. The highest performing software organizations are Apple,
Amazon, Facebook, Google, and Microsoft. Each organization has had drastic improvements.
They had made incredible developments in their management policies, procedures, and
strategies. Amazon had utilized a strategy to drop the bottom 1% of performers from the
organization every year. Google had hired consultants over a five-year period to answer whether
management was needed at all. The answer was, only if the management performed well
(Garvin, 2013). Facebook had adopted a 100% transparency policy and impact only
performance appraisals.
Execution Requirements
The first aspect of the execution requirements are the softer aspects. These include aligning
the engineers’ metrics to
vision. Aligning the metrics derives from the organization’s
vision, mission, culture, strategy, standards, objectives, methodologies, tactics, and then metrics.
The next was determining HR and legal requirements. These requirements were various federal,
state, and local compliances. If the alignment and policies restricted the employees too much,
then it could have hindered employee satisfaction. It was important to ensure employee
satisfaction to a minimum of 90%. The HR department were required to determine employee
satisfaction by providing regular stay interviews. The satisfaction requirement tied into the
subjective metrics of the performance appraisal. Business relationship management was a major
influence on the development and QA directors. With a more formalized business relationship
management map, employees could guide the development of their relationship. Employees
Performance Appraisal Method for Software Engineers
64
were required to have a 5% improvement over the next three months. Each employee needed to
meet a minimum of 70% between all regular stakeholders. The team had implemented a
development program depending on the performance of the employees.
The harder aspects of the execution requirements revolved around the physical deliverables.
These were the metrics, dashboards, reports, and objective implementation of the appraisal. The
metrics provided explicit measurements to performance. The requirement was to provide
concrete measurements to performance. The performance metrics were transferred to the
dashboards and reports that engineers and the directors will utilize. The objective
implementation required a 5% improvement of the engineers and the directors within three
months. Each employee was required to meet an 80% for overall performance. The directors
and the CEO would determine a training program for incrementally improving employee
performance.
Objectives
Each requirement had specification requirements.
1. A process for termination, retention, promotion, and compensation packages

Termination-All engineers have 100% explicit understanding what behavior and
performance will result in termination.

Retention-All engineers have a minimum of 80% satisfaction, want to stay, and cannot
see themselves work at anywhere else but

in the foreseeable future.
Promotion-Engineers are aware of the upcoming positions, provided training for current
and future positions, and explicit understanding of the positions.

Compensation packages- There are four compensation packages: new employee, quarter,
annual, and executive.
Performance Appraisal Method for Software Engineers

65
Compensation fairness-95% employee certainty that their compensation package
approximately matched their contributions.
2. A mixed qualitative and quantitative method to evaluate software engineers’ work

Employees and supervisors can accurately gauge standards, objectives based on
requirements and specifications, and performance within a short period.

Every engineer knows the methodology and techniques to produce the wow factor
whether in business, engineering, and scientific environments.

90% of engineers want to keep working long after the “end of the day.”

Employees are provided the standards and support to make independent choices.

Employees have the ability and direction to accomplish the requirements and
specifications.

The organization instills passion, purpose, values, and vision into employees.

Employees can see a measurable improvement with their contributions to customers and
the organization.

Each engineer has a minimum 78% job design match.

Employees have the required and desired equipment to accomplish their job duties.

The organization is committed to compensating employees that are fair, consistent,
reflective of the market leaders, and provides recognition and validation for each person's
achievement of personal goals, corporate objectives, and professional competency.

Employees feel backed by management by providing a minimum of 80% of the
management support measurements.
3. Utilizing and leveraging the evaluation data to improve the employees, supervisors, and
customer experience
Performance Appraisal Method for Software Engineers
66

The supervisor can see the team’s performance against the market.

The supervisor can see the team’s performance against the software engineering industry.

The supervisor can see the team’s performance against the team’s satisfaction.

The supervisor can identify bottlenecks in performance within individuals and the
department.
Existing Gaps
The first gap that the performance appraisal fulfilled was gathering data from direct
stakeholders and comparing it to market performance. Gathering information from the public,
the organization, and team was part of the highest priority. Each stakeholder had specific needs
that must be fulfilled for a fair performance appraisal.
The next gap was the direction and focus of the performance appraisal. In the prior years,
directors conducted performance appraisals on an annual basis. The directors did not take
specific notes over the year. The performance evaluation was based on memory and near
occurring events. The new standard compared objective and subjective standards. The
performance of each created an impact on compensation and promotions.
Since the CEO changed the direction and focus, the productivity for employees' metrics is
adjusted. Since the metrics are changed, it affected the setting and measurement of goals. The
upper management designed the goals for superior performance. Since each engineer had
regular and specific goals that are measured against the stakeholders, it created a more fulfilling
experience and product.
The new workflow created more concrete measurements for promotion and termination. No
supervisor had terminated an employee; there were employees who felt threatened after the last
performance appraisal. Supervisors are going to measure promotions against specific skills
Performance Appraisal Method for Software Engineers
67
within the skills matrix. As an example, if a junior backend engineer wanted to be promoted
then he would need to measure his skills against the future position requirements.
The new methodology heavily impacted the compensation packages. It determined the
compensation changes. It even provided employees the ability to invest in the company. The
methodology validated the hiring decision for employees. If the program was successful, then
different supervisors could extend the system to on-boarding and training process. Compared to
the other appraisal, it provided legal defensibility for personnel decisions. The method created
historical data for performance and decision making. It also allowed the organization to see the
overall performance of the engineers’ performance.
Performance Appraisal Method for Software Engineers
68
Project Plan
The organization utilized Scrum, Kanban, and Pomodoro. Schwaber and Sutherland
(2013) defined Scrum as “A framework within which people can address complex adaptive
problems, while productively and creatively delivering products of the highest possible value.”
Since the changes to the app had the potential to cause harsh changes to the currency, the
organization upgraded the app in four stages.
The app started in the branch test, committed to the stage test environment, confirmed
changes in the development environment, and finally pushed the changes to the live
environment. The organization had pushed updates every two weeks since release. Therefore,
all customers were required to update to the latest version to continue using the app.
Kanban was based on the Lean principle of demand-based creation and replenishment
(known as Just in Time or JIT). The philosophy to quickly develop and release new releases.
Customer usage and demand dictated the provision forecasts for feature development and
influences quality assurance. To effectively manage sprints, the organization used the Pomodoro
time management technique. The technique broke-down work into 25-minute intervals with
five-minute breaks. These intervals are called Pomodoros. After the first four consecutive
Pomodoros, the employee took a 10-minute break. During the 10 minute break, the person could
send their branch test source code commits to the stage test environment or start a build of their
branch. After the first eight consecutive Pomodoros, the person took a 30-minute or 60-minute
lunch break. The technique made it easier to correlate a work breakdown structure to
Pomodoros, thus creating a more predictable schedule.
The process of implementation relied on two phases with seven major milestones. The
first was to create end implement the criteria for promotions, terminations, and compensation
Performance Appraisal Method for Software Engineers
69
packages. The CEO communicated to employees that the changes that would occur over next
quarter. The next milestone was management enforced the new office policies. These created
the foundation for the new appraisal method; otherwise, employees and supervisors could feel
ambushed and measured unfairly. Afterward, the data scientist needed to create the reports for
the supervisors to record the data. The next milestone was enforcing a multimethod-multirater
appraisal process. During the first quarter of usage provided the finance and accounting
department with sufficient time to adjust compensation packages. During the first quarter,
employees could capture moments to show the impact of each other. After the first quarter,
supervisors utilized the reports to create performance indexes on the departments.
Project Development Plan
The development plan utilized the organization’s existing project management style. The
performance appraisal method was introduced to the directors for self-evaluation until upper
management properly introduced the project to the team. During the test period, the directors
could test many aspects of the appraisal method. After the specific aspects of the appraisal
method passed then the directors developed it toward the teams. Directors and engineers came to
the agreement for the weights on their position and different objectives.
Scope
The scope of the project included three major objectives.
1. A mixed qualitative and quantitative method to evaluate software engineers’ work
2. Utilizing and leveraging the evaluation data to improve the employees, supervisors,
and customer experience
3. A process for termination, retention, promotion, and compensation packages
Performance Appraisal Method for Software Engineers
70
There were many aspects of the scope that fell outside of the project. There was not a Human
Resources process or protocol around hiring employees based on the performance index. It was
assumed that new hires submit a resume, file a skills matrix assessment, and undergo a software
engineering interview test.
There was not specific methods, tools, techniques, or utilities to improve software
engineers’ performance index. Training and development advanced was designed by a different
department and the directors. It was the employees’ and supervisors’ discretion for improving
specific aspects on a case by case basis.
Each director knew how to perform an appraisal. The project excluded the basic steps of
an appraisal. Part of the appraisal included an alignment strategy for performance index. The
broad concept was covered, but not details.
There was currently no third party research or trial group for the solution. The CEO
provided explicit business management, computer science, and psychology theories that
comprised the solution. These were the readily accepted major theories within each field.
It was the responsibility of the data scientist to create the report based on the provided
metrics. The project details would not include how to create reports with tools such as Excel,
Hadoop, Heidi, or NodeJS ETL. Additional details about how the data scientist and the
operations engineer will automate report generation and the policies on confidentiality or
distribution of the report would be disregarded. Disclosure of the reports are assumed to be
internal to the specific individuals, departments, and directors.
The project will not include a compensation package algorithm. Reallocating the
compensation package was another project for the accounting, finance, and HR department. The
Performance Appraisal Method for Software Engineers
71
project would address compensation packages compared to the industry standard and skewed
toward the organization’s success.
Assumptions
1.
The evaluation standards are critical and related factors to the organization and its
customers (Wilson, 1998).
2.
Evaluations will improve employee's performance.
3.
The performance standards are specific, measurable, achievable, reasonable, and time
bound.
4.
Compensation will be increased in proportion to increase performance.
5.
The employee was provided the necessary elements to complete the requirement or task.
a. The employee has the necessary knowledge and skill or provided guidance and
training.
b. The employee has the required resources to complete the task(s) in time.
c. The employee has a constructive work environment to complete the task(s).
d. The employee has control over the results.
e. The employee has connections with others within the team.
6.
The employee’s individual contribution can be discerned from workers or supervisors.
a. The employee's work has an evaluation method for internal stakeholders.
b. The employee's work has an evaluation method for external stakeholders.
7.
The evaluation covers performance the entire cycle of evaluation, not just the period
recallable by accurate memory.
8.
All evaluators are consistent with each other, and each evaluator is consistent from one
employee to the next.
Performance Appraisal Method for Software Engineers
72
a. Evaluators can determine environmental or time-based conditions
9.
When all of the evaluations are complete, supervisors will have a better understanding of
the employees.
Important Milestones
1. Implement policies for promotion and termination.
2. Reform office policies.
3. Revise the appraisal and change metrics for employees. Create a check-in report with
accomplishments, relations and career goals.
o Create measurements for competency, quantified skills that affect business
objectives, qualified skills that affect scientific objectives, and convergent skills
that affect engineering objectives.
o Create specific measurements for relations.
4. Adjust compensation package to reflect changes in policies and performance.
5. Create supervisors reports in the following areas:
o Alter the current app report to include market performance and user acceptance.
o Create a report of the team’s performance vs. industry standard.
o Create a report of the team’s performance vs. satisfaction.
6. Utilize the team performance reviews to create a performance index. Reviews produce a
visible impact of the employee contributions for the annual review.
7. Hold impact conference and to demonstrate employee impact.
Performance Appraisal Method for Software Engineers
Timelines
Figure 4. Timeline. Adapted from “Human Resource Management Archives,” Courtesy of
AtmanCo, 2015, retrieved from https://atmanco.com/blog/category/hcm/.
In Figure 4 is each step the organization took for the plan. In the circled areas were the
number of hours each step would take. Standard business days were Monday through Friday;
however, the CEO would work over weekends to ensure that the necessary steps were taken to
migrate the performance review over for the 2016 review. The project was completed before
2016, but the success of the project would not see the full breadth of success until the 2017
review.
The first milestone on the timeline was creating criteria policies for promotion and
termination. The reason it was the first milestone was that employees needed to know what
73
Performance Appraisal Method for Software Engineers
74
conduct and performance would cause promotion and termination. Working with the human
resources and legal department took approximately eight hours.
Reform office policies took approximately five hours to complete. These were the
policies that provide employees with accountability to employees’ and supervisors’ development.

Evaluators were trained to identify, distinguish, and analyze contributions.

Employees were informed when a position is available.

Supervisors would provide necessary and relevant training.

Employees would have the opportunity for the technology of choice.

The departments were provided a transparent office environment.

The HR department would utilize proactive talent retention methods.

HR will regularly use stay/satisfaction reviews to determine employee satisfaction.
Revising the appraisal process took the longest amount of time. It took approximately 32 hours
between the director of development and the director of QA. The implementation took over 10
business days to maintain normal operations of the app. The milestone implemented an
employee performance review that analyzes the activities and behaviors. The process included:

Create a check-in report with accomplishments, relations and career goals.

Create ability measurements for competency, qualified skills that affect scientific
contributions, quantified skills that affect business contributions, and convergent skills
that affect engineering contributions. When implementing the milestone, the longest
amount of time is deciding the weights associated with each employees’ objectives and
skills.

Create motivation measurements that analyze effort and support to the employee to
determine which forces are pushing and pulling the employee.
Performance Appraisal Method for Software Engineers

75
Implement work environment measurements to evaluate job design, equipment &
materials, policies & procedures, economic conditions, and management support.
o An employee may be hired for a specific position, but the position may gradually
shift based on the business needs and the employee’s abilities. When a job design
shifts too far out for the employee, then it decreases the work environment. The
reason is that it places heavy stress on the employee by requiring them either to
learn new responsibilities and/or skills.
o For the employee to do their job well, they need the required (or desired)
equipment and materials to do the job. For many software engineers selecting
their computer, operating system, peripherals, and programs is more important
than choosing a car. Ask any programmer whether he prefers Apple, Linux, or
Windows? Emacs or Vi? Eclipse, Visual Studio, or Webstorm? Spaces or tabs for
indentation? These kinds of disagreements can cause long arguments. Within our
office, a developer can choose any tools they want since our app is device and
operating system agnostic.
o Policies and procedures are a work environmental measurement; however, these
traits affect the employees' abilities and motivation. These are designed as safety
precautions.
o Economic conditions are the compensation packages we utilize to keep employees
comfortable and retaining them. In most cases, an employee will not stay if it is
not fiscally advantageous to work there. Everyone has bills he or she need to pay.
We provide base pay, benefits, bonuses, equity, and perquisites.
Performance Appraisal Method for Software Engineers
76
o Management supports employees with career and technical direction, objectives,
and strategies. It is part of the employee development check-in program to keep
employees with the best training. It includes ten major areas (Garvin, 2013):


Consistently approach to performance and development

Clear vision and strategy for the team

Be a good coach

Empower your team

Refrain from micromanaging

Utilize active listening with your employee and team

Express interest in team members' success and well-being

Be productive and results-oriented

Provide career development

Possess key technical skills to advise the team
Create specific measurements for relations. These include peers, subordinate,
supervisors, and team members and the traits associated with each relationship type.

Adjust compensation package to reflect changes in policies and performance. Adjusting
the compensation package will not be difficult, but maintaining it with HR, payroll, and
benefits will require additional time. It will take seven hours to complete adjusting and
implement into the system.

Create supervisors report in the following areas: a current app report that includes market
performance and user acceptance, create a report of the team’s performance vs. industry
standard and create a report of the team’s performance vs. satisfaction. The directors
Performance Appraisal Method for Software Engineers
77
create the reports in the revised appraisal milestone, but the data scientist needs to create
a front end form. It is utilized with business intelligence rather than an Excel spreadsheet.
It will take approximately four hours to complete. One hour for each report and an hour
to integrate each together. The data scientist starts working on the reports on the same day
as the HR manager starts on the compensation package.

Utilize the team performance reviews to create a performance index. It will take
approximately 2 hours to create a performance index since each engineer and the
directors take approximately 10 minutes. The directors can gather two months’ worth of
data for the first review.

Reviews produce a visible impact of the employee contributions for the annual review. To
compile a piece from each employee into a presentation will require an hour. It will be a
simple presentation describing the background, accomplishment, and impact. The last
week of February will be vacation and compensation week for employees
Performance Appraisal Method for Software Engineers
78
PERT Analysis
The PERT analysis included many of the steps for developing and implementing the project into practice. The CEO identified each
of the milestones within the table below under the description. Many of the tasks were dependent on each other, so there was little room for
parallel processes.
Description
Implement policies for promotion and
termination.
Reform office policies.
Adjust compensation package to
reflect changes in policies and
performance.
Create an employee performance
review that analyzes the activities and
behaviors.
Create a check-in report with
accomplishments, relations and career
goals.
Create supervisors reports.
Utilize the team performance reviews
to create a performance index.
Node Pessimistic Optimistic
Variance
Expected
Slack
Time
A
B
10
7
6
3
0.444444444
0.444444444
8
5
0
0
C
40
24
7.111111111
32
0
D
10
4
1
7
0
E
F
6
3
3
1
0.25
0.111111111
4.5
2
2.5
0
G
1.5
0.5
0.027777778
1
0
Z-Score
σ(P)
4.176232687 3.023059525
Performance Appraisal Method for Software Engineers
The project was very straight forward except for the compensation packages and supervisors’ reports. For each of the tasks, there
were not any overlap for the assigned team members. The compensation packages took longer, so the supervisors’ reports had slack for
development within the analysis. The project's development needed to be deducted from the development and QA teams. There will be
additional time available after the revised appraisal for the directors to gather information. There are additional details regarding specific
dates in the timelines figure.
79
Performance Appraisal Method for Software Engineers
80
Resource Requirements
The most important aspect was utilizing a document editor, project manager,
spreadsheets, and timer. The company currently used Google Docs for document editor and
spreadsheets. Stakeholders were required to document most milestones. The team used Trello
for task and phase planning. Stakeholders saw when a task was ready in Trello.
had
developed their proprietary Pomodoro timer for the organization.
The organization used an existing Amazon Web Services (AWS) cloud server and open
source software. It had cost $800 per year to run the Ubuntu (Linux) server. The server was
needed to view, store, and distribute reports through the dashboard. The team used Meteor
framework for app development. The two major aspects of Meteor the team used were Node.js
and MongoDB. For the team to access the server, they used Node.js for server software.
Node.js runs on top of Ubuntu. It provided additional protocols and services. MongoDB was
used for databases (DB) services. The engineers and directors used (their choice software) Heidi
or RoboMongo to manipulate the database. Manipulating the databases was important in
creating the supervisors’ reports and the performance index dashboard. The directors used Visual
Studio Code and Emacs for editing source code of the PI dashboard.
The HR manager used EasyERP for customer relations management (CRM), enterprise
resource planning (ERP), and Human Resource Management (HRM) on the server. The HR
manager used the HRM for associating the compensation packages to employees, promotion and
termination policies and procedures, and revising the appraisal process.
For the impact conference, the company used Google Docs for text copying, Snipping
Tool for image capture, Blender for video editing, and Prezi for the presentations. Employees’
content required different mediums to capture the accomplishment.
Performance Appraisal Method for Software Engineers
81
Budget
It had cost the organization approximately $1,780.93 for development. The estimated
amount is based on the assumption that rating a peer took 15 minutes and 30 minutes for a selfreview. There were three peers and a supervisor. The implementation for changing the process
had a lower cost but it required more time. Salaries for other employees were based on the same
as engineers. It was the average reported cost between Glassdoor, Indeed, and Pay Scale. The
average salary for a marketing manager was $76,544, the average salary for the HR manager was
$74,515, and $126,398 for legal counsel. It required additional work from the director of
development and the director of QA as described in the personnel section. The implementation
plan would cost $1688.33. The figure was including the assistance hours from other employees.
The total project cost was predicted at $3,469.26.
Personnel
It was critical for the director of development and the director of QA to know the required
and desired information they would like and also the information that needs to be focused on to
reach specific objectives. When the directors gathered the necessary information, it was assigned
to the data scientist from the development department to create the necessary forms. For the data
scientist to generate the report, the marketing manager needed to gather information for the
individual.
We needed the HR manager and legal department to ensure the legality of the change in
operations. It was the responsibility of finance and accounting to ensure that the changes in the
development department reflected the improvements. If there were significant improvements in
the development department six months after the implementation, then we would consider an
adoption of similar programs for each department.
Performance Appraisal Method for Software Engineers
82
Deliverables
The non-financial assets were the metrics, policies, and procedures surrounding the
performance appraisal. Metrics and the weights related to the employees and their position is a
cornerstone of the project. The core metrics were within the document, but details for the
application needed to be further developed toward the development and QA departments. The
policies and procedures around termination and promotion were radically simplified.
provided employees with clear visibility of the goals and how to attain them.
Project Implementation Plan
The implementation plan followed the organization’s standard development path. The
plan was proposed to address specific stakeholders because of a demand, align the design for a
specific reaction and objective, test the option with the design, utilize specific measurements and
values in development, and when 95% of the specifications were complete then the deliverable
was released.
Strategy for the Implementation
The strategy was determining the needs of each stakeholder, measure each stakeholders’
contribution, create a solution based on priorities, and mitigate risk from social risk to financial
risk since the organization utilized less than 20% of resources in the last
The needs of
the customer were determined through business analysis and market analysis. The employee
needs were determined by the directors on a case by case basis. For the directors to measure the
engineers’ performance, they were using the same (or similar) scorecard. The scorecard served
as a template for engineers. Different positions had different focuses and value. The team
considered it unfair to use the same metrics for a front-end engineer as a security analyst. The
directors determine the weights for each employee during the revised appraisal.
Performance Appraisal Method for Software Engineers
83
Phases of the Rollout
Many of the aspects of the design had been completed in the capstone document. It was
designed as a template for the directors. It was a foundation for specific directions toward each
engineer. Operationally, the organization functioned on demand and supply through Scrum
sprints. The sprints were aligned to match pay periods. Each team had enough time to adjust
accordingly. The objectives, requirements, and specifications was determined in the project
design and requirements. The stakeholders were already aware of the project. Disseminating the
tasks was provided in a private Trello board available to the employees.
The directors ensured the alignment between the organization’s vision to employee tasks.
The specific alignment matched the vision, mission, culture, strategy, standards, objectives,
methodology, tactics, metrics, and tasks.
was lean organization. The 360-degree
review was not necessarily incorrect, but the focus and significance may not have been measured
correctly. Measuring skills that had little relevance or impact was a waste of time and money.
Conflict management, communication, and teamwork are essential to all organizations. These
skills were important for engineers to advance their careers and work with different types of
people. Integrating the importance of each skill for employees was determined by the directors
and the employee’s direction.
The directors tested the scorecard metrics in their existing work to reduce time. Testing
from the directors’ perspective allowed them to get ahead of employees for any questions they
had. The testing period started before communicating the promotion and termination milestone.
The CEO had utilized the scorecard metrics and improved his metrics by 9% and overall
productivity by 17% in three months. The CEO was sponsoring the change for the appraisal.
Performance Appraisal Method for Software Engineers
84
The directors' group was the focus group for the implementation. He conducted a trial for the
directors’ trial and review the findings.
The development process started with the reform office policies milestone. It provided
employees with the necessary environment to generate greater change. It was necessary to create
high employee loyalty to continue the team's success. Altering the office policies established a
forum of disclosure for the employees.
The development process ended at making the performance index dashboard (milestone
6). After the directors created the reports, the metrics were included in a weekly team meeting.
The project launch was fully initiated after the supervisors’ reports. It was the time when
employees had direction, focus, and metrics for their performance.
The release process was quicker than normal since the density of the timeline was tighter
on some milestones than others. The physical deliverables were the reports, dashboards, and
employees’ impacts in the presentation. The non-financial deliverables were the metrics,
policies, procedures, and scoreboards surrounding the performance appraisal.
Dependencies
The directors and marketing created additional content for the impact conference from
customer feedback, stakeholders' comments, and the supervisors’ reports. Directors needed to
see what employees and customers submited for engineers’ accomplishments. Engineers also
submitted their accomplishments that they were proud of.
Deliverables
The project provided only a few assets to the organization but they were very valuable.
The deliverable assets were the dashboards, documentation, reports, and employees’ impacts in
the presentation. The documentation was finalized during implementation. It included
Performance Appraisal Method for Software Engineers
85
information from specific stakeholders. The reports were the developed versions of the metrics.
A tangible report for the metrics to interact with. The dashboards made interacting with the
reports more user-friendly. The (employees’) impact conference were several small deliverables
for each employee.
Training plan for users
The directors provided framework training (for the appraisal) to the employees during the
reform office policies in milestone 2. The training did not take long since it was in a group
setting. Directors created specific weights and metrics for each position in the revised appraisal.
Directors provided an email to each employee after the supervisors’ reports milestone was
complete. Specific details for each area of the metrics were included and the expectations.
Since a majority of the content evaluated technical aspects of their job, it was easy for
engineers to understand. The challenging part was explaining the business relationship model.
Engineers had both broad traits and specific traits for interacting with each major stakeholder.
Since the engineers understood they could accomplish more by cooperating and both parties
were evaluated against each others’ relationship. Further development in relationship modeling
could provide the organization with a higher standard of life (Gladwell, 2013).
Performance Appraisal Method for Software Engineers
86
Quality Assurance
Determining the quality of an appraisal method was a meta-analysis. The skill required
many of the skills
promoted. It addressed who were the stakeholders, what were their
values, and what were their expectations? The four major stakeholder groups were the directors,
engineers, customers, and employees. The primary stakeholders were the directors and the
engineers. The meetings after the 360-degree performance reviews revealed a great deal about
the different stakeholders’ values. The information below indicated the stakeholders’ primary
values and priorities for the engineers.
A. Directors
1. Fiscal success of the app
2. Balance between the departments and the organization
3. Science
4. Engineering
B. Engineers
1. App popularity
2. Science
3. Engineering
C. Customers
1. User experience
2. Additional rich feature options
3. Engineering
4. Treat employees fairly
D. Employees
Performance Appraisal Method for Software Engineers
87
1. Smart
2. Cool
There were communication and values disconnections between directors, engineers, and
employees. Aligning the values of each major stakeholder created more focused and rounded
engineers. The alignment addressed the communication skills of the engineers.
It was challenging to determine the objective abilities since the organization had
inconsistently utilized a different performance method each year. The team used code metrics
analysis over the source code that has been submitted over the year. A broad idea of performance
improvements were made.
Quality Assurance Approach
The quality assurance approach utilized specifications for compliance. It addressed the needs
of each major stakeholder. It utilized specific measurement for each priority. The specifications
were critical since it ensured the quality of the appraisal. The specifications comprised the
project’s objective requirements. For the final check off, a consensus was required between each
stakeholders’ needs. The directors checked each specification against the objective with
incremental improvements.
Solution Testing
The simplest way of evaluating the performance appraisal was the difference in the
quality and quantity of the work over time. It would be simple for directors to see evaluating
employees over quarters and years demonstrated in the scorecard analysis. The scorecard even
identified micro-fluctuations in performance. There were key characteristics when an engineer
got in the “zone” and produced massive amounts of high-quality work. Each person had their
Performance Appraisal Method for Software Engineers
88
opinion of higher quality work, but a direct assessment by each stakeholder and group created a
consensus.
The next way of testing the solution was in the app performance. There were several
ways the team will measure the improvements to the app. It was easier for the developers to
identify performance enhancements during branch testing. A branch test is when a developer
tests his individual contributions in an isolated build of the app. The primary measurements for
the app were explicitness, release rate, reliability, resource management, scalability, and user
acceptance. Each measurement had a different major stakeholder, but every stakeholder cared a
certain amount about each measurement.
Explicitness was how easily the source code was understandable. Customers overlooked
explicitness, but it was critical to developers and other major stakeholders. An engineer could
identify poor explicitness by lack documentation, poorly named variables, no comments in the
source code, improper file structures (such as poor spacing or indents), disorganized filing, and
branch disorderliness. When the code had poor explicitness, it could have made it difficult to
understand for team members. Getting started, continuing work, and collaboration would have
been challenging. These hurdles would slow releases. When the organization would slow
releases it could result in decreased sales.
The release rate was critical to a company as a release often implied product revenue.
The release rate was determined by how fast an engineer could understand the objective,
determine implied or communicated requirements, deconstruct the requirements into
specifications and actionable tasks, turn tasks into code, leave code openings for the team to
hook, and document the assignments. Programming can sound straight forward, but with every
environment, there were required roundabout methods to achieving conventional functions.
Performance Appraisal Method for Software Engineers
89
When analyzing a task, a programmer could estimate the task will take three days. It was an
industry standard to multiply the time required of a task estimate by two, six days. The PERT
technique is (optimistic time + pessimistic time + (4 * average))/6. If there were unpredictable
events, then it would be exponentially longer for the engineer. There were several examples of
similar situations that have occurred. Microsoft had abandoned frameworks such as Silverlight or
XNA (Microsoft, n.d.). Google Chrome no longer supports SLA-1 encryption (Google, 2014).
Apple had drastically altered their terms of service agreement for ad identifiers (Perez, 2014).
Reliability was a critical performance metric. It directly measured how stable the app
was. It included superficial bugs to system collapsing bugs. Within the last five years, there had
been interesting strategies in managing reliability. Google had adopted a very traditional
approach to reliability with very creative tactics. "The company uses arrays of small, cheap
switches to provide some of the same capabilities as much more expensive hardware, then
manages workload distribution in software"(Hruska, 2015). Netflix took the opposite approach.
They expected every function, packet, or session to become lost or destroyed. The system was
fault tolerance in a high volume, distributed system (Christensen, 2012). Part of reliability is test
user acceptance rate for crashes. Almost 10 years ago Myspace was the leading social media site.
At one point, the site would have outages for hours at a time. Users were outraged. These series
of crashes led to the demise of Myspace and the rise of Facebook. Since 2004, Facebook's
longest outage was 2.5 hours in 2010 (Facbook Engineering, 2010). Facebook had tested the
limits of user acceptance by artificially crashing their new app to determine user loyalty.
Resource management was defined as the allocation, optimization, and distribution of
system resources. Poorly managed resources resulted in negative user experiences such as
clipping, chopping, excessive caching, lag, overheating, massive RAM usage, and large data
Performance Appraisal Method for Software Engineers
90
storage. Resource management occurred on the client app, client browser, and server sides.
Resource management affected the minimum system requirements for an app. Determining the
minimum and recommended system requirements affected the competitive advantage. More
people could purchase the software with lower minimum requirements. It allowed an
organization to gain a significant competitive advantage, increase market segmentation, increase
market share growth.
Scalability affected the ability to manage larger resources and also affected the
marketability to enterprise organizations. Scalability could affect the growth potential of the app.
In the case of many people utilizing the app, they may use it in unexpected ways that would
violate system resources. Microsoft Excel is a prime example. For a long period, an Excel table
could only contain 65,536 rows. A large organization could work on a portfolio report and
consume the limit in less than an hour. There were roundabout ways to generating the report.
Both methods required separating the document into several files. The first method was
separating the report into several documents, and the viewer would need to open each one. The
second was to reference the desired cell in another report but required all the files to be in the
same folder. Microsoft worked to patch the limitation and increased the limit to “more than a
million” (Microsoft, n.d.). It took the company more than five years to fix the bug, but during
that time it allowed the company to migrate firms over to SQL server.
The most important and easiest to understand was user acceptance. Various metrics
comprised the measurement within it such as activity, acceptance, downloads, active users, and
total users. There were a required minimum of two user acceptance tests for every feature
implemented into the app before release. The first was the specific app demographic and the
second for the general market.
Performance Appraisal Method for Software Engineers
91
Revisions
The first aspect was ensuring employees were regularly reporting their progress. Reports
were completed quarterly and audited. While all data was kept secret, the derived metrics from
all employees was readily available. It was made possible by utilizing development operations
(DevOps) and systems operations (sysOps) automation. If there had been any changes, they
were updated daily. The reports demonstrated the team’s productivity and impact.
Office administration, the director of QA, the director of development, and HR ensured
that changes in office policy were enforced. When an employee left the company, HR had an
exit interview and assessed the situation to ensure that it complied with office policy. Similarly,
HR needed to audit when an employee title changed within the department. Payroll and benefits
needed to perform regular auditing to make sure compensation packages matched the employees.
Regardless of employee changes, the HR team implemented regular stay interviews to ensure
employee retention was improved.
Summative Evaluation Plan
The individual summative evaluation was a scorecard that identified specific performance
metrics. The various measurements created more complex metrics and group metrics. The
metrics ensured the contribution to each stakeholders’ needs. The performance assessment
provided a summary of performance and growth trajectory.
The team scorecard was more important to the individual evaluation since it directly
analysed the departments’ performance. Even though it was generated from the individual
scorecards the formulated metrics guaranteed the fulfillment of the stakeholders’ needs and
analyzed specific skills may be needed later. It allowed the directors to see which skills specific
engineers needed.
Performance Appraisal Method for Software Engineers
92
Disseminating Results
The individual scorecards were available only to the HR manager, director, and the
employees. The scores were readily available from the reports dashboard. Other aspects of the
performance evaluation were available on a quarterly basis. For that reason, the employee
needed to maintain regular feedback from other teammates.
The department scorecards were available every week, but the full performance
evaluation was available every quarter. The weekly scorecards were discussed during the one of
the weekly meetings, but also posted on the company’s bulletin board system. The department
scorecards were only available to internal employees.
Performance Appraisal Method for Software Engineers
93
Post Implementation Support and Issues
The post-implementation required some additional support. Some personnel outside of
the development and QA departments needed to stay involved. The HR department needed to
ensure the consistency in review quality and provided training.
Post Implementation Support
The first step sustaining the performance appraisal process was providing monthly checkins through the dashboard the data scientist created. The report was regularly filed by employees
to ensure that they are staying on track. Employees requested coaching sessions on a monthly or
quarterly period. Directors and employees were required to have a minimum of quarterly
coaching. At the end of the year, the department will have an annual impact celebration. It will
show the best stuff the employees made over the year.
Finance, accounting, HR, and payroll & benefits were needed for additional support.
Finance and accounting department had more invoices for the work environment benefits. The
HR manager ensured the employee records were regularly recorded. There was a bit more
paperwork for payroll & benefits. A formal compensation package needed to be completed for
new hire employees to ensure salaries matched competitors. Recruiters also needed to
familiarize themselves with salary ranges.
Post Implementation Support Resources
The only required resource was the server. It was the AWS Cloud server for $800 per
year. The configuration needed to remain consistent in order for the data to be processed
correctly and all of the stakeholders had access their software.
Performance Appraisal Method for Software Engineers
94
Maintenance Plan
The appraisal metrics and methods were stored and improved incrementally. The
directors and employees needed to ensure that the duties they were assigned are in line with
needs of the stakeholders and within the employee’s job design. If tasks fell outside the job
design, then it needed to be assigned to another person, hire a new employee, or transition the
employee to the new position.
A primary objective of the maintenance plan was to maintain the termination, retention,
promotion, and compensation packages processes. HR needed to ensure that the review
processes were utilized and enforced. After each quarterly and annual review, the supervisors
audited the reviews for consistency, progress, and quality. The reason was that part of the
primary objective was ensuring directors and employees are utilizing the data effectively. The
dashboards and reports were regularly accessed, and employees were continually improving.
Performance Appraisal Method for Software Engineers
95
Conclusion, Outcomes, and Reflection
Evaluating an engineer’s contributions was a complicated ordeal. There were many different
facets to a high-quality developer. Within the last few years, it had become more challenging as
there have been more roles that fuse others’ together. Business intelligence analyst, front-end
developer, information systems analyst, maintenance integration engineer, secOps engineer, and
sysOps engineer are just a few.
Project Summary
The initial design of the project was more complex than initially assessed. The difficulty
in evaluating software engineers was comparable to measuring a hybrid architect, designer,
engineer, and mathematician. Since the skill sets were different, a different approach was
needed. Traditional psychometric properties, applied psychology, human resource development,
human resource management, performance assessments, and performance measurements were
not sufficient. Computer science and mathematics theories helped assess the activities,
complexity, difficulty, and effort of software engineering but still lacked the substance for
utilizing the theories in practical environments. Business administration and project
management identified the particular skills into practical measurements. Information systems
theories assisted in structuring the necessary data, priorities, and values to stakeholders for the
appraisals’ effectiveness.
The testing methodology met with positive results. When the CEO tested the appraisal
method in the office, he had utilized most of the appraisal method with a heavy emphasis on the
scorecard metrics. Within three months his overall ability improved by 9% and his productivity
improved by 17%. Other firms had used the scorecard method that ranged between 7% to 30%
improvement over three months.
Performance Appraisal Method for Software Engineers
96
During development, the directors were initially apprehensive, but since it addressed their
issues, they were relieved. The engineers also liked the appraisal method since it put their skills
and contributions in ways they could relatable. The HR department was a bit surprised that the
development and QA departments had their own evaluation methods and were being
implemented so quickly. When the data scientist was working on the reports she had introduced
a couple of metrics that fit well. Meta-objectives scale and the impact force ratio addressed two
issues. The meta-objectives added specifications value weights to objectives. When the business
objectives were rated in a 0 to 100% ration, the meta-business objectives might be 103/92. The
metric shows that the team had not only achieved 100% of the objectives, but also had 11
specifications accomplished ahead of schedule. Impact force ratio is similar to meta-objective
metrics. It suggested that a specifications the team created required minimum effort, but
increased the revenue by 3%.
Deliverables
The deliverables included many financial and non-financial assets. The first are the
policies and procedures for compensation packages, promotions, and terminations. These are the
consequences of employee behavior and contributions.
The next asset is the organizational alignment figure. Within a traditional environment,
the software engineering teams are separated by development, operations, and quality assurance.
Each team creates their strategies and tactics with a unifying coding standard. A part of the
problem we encountered was the slight misalignment between directors and employees.
utilizes best practices, strategies, and tactics challenges performance standards, and
then distributes it within the teams.
Performance Appraisal Method for Software Engineers
97
The knowledge aspect of employee performance is the skills matrix. The CEO included
the modified skills matrix template based on primary design, engineering, and security skills.
The particular weights for each position were outside the scope of the document. It is the
directors’ individual decision for the employees’ position.
The skill aspect of the performance appraisal is the scorecard. The template includes the
best practices metrics. There are five sections in the scorecard. The CEO included a template as
a spreadsheet to utilize. The spreadsheet includes the tasks that are completed, the power of the
team, the utility of the team, offensive impact (o-impact), and the temperature of the team.
The last deliverable is the relationship mapping template. It addresses the issues of
employees’ relationships and traits. Traditional human resource development, cognitive
psychology, social psychology provide the best support for relationships.
Outcomes
The outcomes for the project was outstanding. The average employee productivity had
increased by 11%. The employee performance index had increased by 5%. Employee
satisfaction is at 98%. People are working longer and loving it. The work environment and
support for employees allow them to stay longer without worrying. Since the CEO and directors
changed the metrics, the product quality had increased by 8%. The quarterly revenue increased
24% higher than the profit projections. The organization is headed for a consistent 250%
revenue growth for the second year in a row. The first quarter resulted in a 54% revenue
increase. The projected profit earned an estimated $10.8 million. The project saved an estimated
$2.8 million. The organization received massive publicity for the reforms. The company was
placed on four local magazines for best places to work.
Performance Appraisal Method for Software Engineers
The project did come with some downsides. After the publications,
98
were
receiving so many resumes they did not know how to manage them all. The implementation and
support were more time consuming than anticipated. The dashboards required specialized
visualization to express complexity. The reports required additional time to develop. When
other departments had heard about the performance appraisal, they thought there was favoritism
for the departments. An unexpected outcome was that engineers felt the system was “too fair.”
Everything in the appraisals was accurate and considerate, but there were no surprises. All
aspects of the financial compensation packages and incentives were very predictable.
Reflection
The most difficult aspect of the project was within the software engineering metrics.
Evaluating complexity is the most complex aspect of the performance appraisal. Most of the
aspects of complexity can be expressed mathematically, but there are other aspects that require a
soft understanding of engineering. When planning a project there was no software repository or
tool as a reference for complexity. Only programming several apps and breaking them down into
Pomodoros or WBSs works for evaluation. After the programming was complete then there
were tools for reviewing complexity.
The largest aspect of the project was compensation packages. It was an entire discipline
by itself. There were a substantial amount of information on models, non-financial incentives,
risk analysis, and threshold management techniques that fell outside the scope of the project. It
was challenging to quantify the correlation from engineering to fiscal worth.
There were assessment and automated tools that would be excellent for potential and new
hires. It may not work well with existing employees since employees already had familiarity
with our app’s details.
Performance Appraisal Method for Software Engineers
99
No engineer will write code without documentation that includes objectives,
requirements, and specifications. Many American engineers are provided vague requirements
and then utilize a waterfall project management style that wastes a significant amount of time.
The impact of neglected documentation is massive.
The supervisor or stakeholder must create the documentation for objectives,
requirements, and specifications from user experiences. The purpose of making an app was for
the consumers’ benefit. For many engineers, the technology was available before the demand.
Scrum is based on the demand.
Post-Mortem Directions
The organization must move to the next stage. It was based on scaling our success for the
future by determining how the organization needs to change. If
continues its success
then it must create a rapid onboarding program. It would be a training program that provides the
new engineer with the knowledge and skills for the position.
The other aspect that needs to be improved is to discover ways to make the reports more
effectively. It starts with gathering the data faster and more effectively. The steps for gathering
data and creating the reports needs to be more automated.
Performance Appraisal Method for Software Engineers
100
References
AICPA, & CIMA. (2012, November). How to Develop Non-Financial KPIs. Chartered Global
Management Accountant. Retrieved from
http://www.cgma.org/Resources/Tools/DownloadableDocuments/How-to-develop-nonfinancial-KPIs_CGMA.PDF
Akampurira, A. (2014). Performance Appraisal (1st ed.). Anchor Academic Publishing.
Retrieved from http://site.ebrary.com/lib/westerngovernors/reader.action?
docID=10856489
Alexander, J. (2011). CoderMetrics. Oreilly & Associates.
Alexander, J. & Bethke D. (2011). Agile Tasks and Codermetric Tracking [TABLE]. Retrieved
from https://codermetrics.org/2011/08/24/agile-tasks-and-codermetric-tracking/
Alexander. (2008, January 30). Performance Reviews Are A Big Fat Waste Of Time. Retrieved
from http://positivesharing.com/2008/01/performance-reviews-are-a-big-fat-waste-oftime/
Anonymous, Diogenes, D., Lanfear, T., Squillace, R. (2016) Azure Identity Management and
access control security best practices. Retrieved from https://docs.microsoft.com/enus/azure/security/azure-security-identity-management-best-practices
Aphnet. (n.d.) Recruitment & Retention - How to Determine Your Retention, Turnover, and
Vacancy Rates [2.1.1.a.2]. Retrieved October 26, 2016, from
http://toolkit.ahpnet.com/Building-a-Recruitment-and-Retention-Plan/Step-1-GatherOrganizational-Baseline-Information/Gather-Organizational-Baseline-Info-QuickTool/How-to-Determine-Retention-Turnover-Vacancy-Rates.aspx
Performance Appraisal Method for Software Engineers
101
Arkowitz, H., & Lilienfeld, S. O. (2010, January 1). Why Science Tells Us Not to Rely on
Eyewitness Accounts. Retrieved October 25, 2016, from
https://www.scientificamerican.com/article/do-the-eyes-have-it/
Armstrong, M. (2003). A Handbook of Human Resource Management Practice. Kogan Page.
Retrieved from https://books.google.com/books?id=2AGbuhlTXV0C
AtmanCo. (2014a, August 8). How to Encourage Employee Engagement [Image]. Retrieved
from https://atmanco.com/blog/hcm/how-to-encourage-employee-engagement/
AtmanCo. (2014b, August 29). Internal Promotion: Which Employee Should be Promoted
[Image]. Retrieved from https://atmanco.com/blog/hcm/internal-promotion-whichemployee-promoted/
AtmanCo. (2015a, November 3). Why Managers Must Develop Emotional Intelligence [Image].
Retrieved from https://atmanco.com/blog/leadership/why-managers-must-developemotional-intelligence/
AtmanCo. (2015b, November 9). 6 Common Mistakes to Avoid During Employee Evaluations
[Image]. Retrieved from https://atmanco.com/blog/hcm/6-mistakes-to-avoid-inemployee-evaluation/
AtmanCo. (2015c, November 23). 11 Performance Management Best Practices [Image].
Retrieved from https://atmanco.com/blog/hcm/11-performance-management-bestpractices/
AtmanCo. (2015d, December 15). How to Develop Leadership Skills [Image]. Retrieved from
https://atmanco.com/blog/leadership/how-to-develop-leadership-skills/
AtmanCo. (2016a, April 4). Factors To Consider When Building Trust In Teams [Image].
Retrieved from https://atmanco.com/blog/team-building/9-things-building-trust-in-teams/
Performance Appraisal Method for Software Engineers
102
AtmanCo. (2016b, May 24). 13 Ways to Lose Your Best Employees [Image]. Retrieved from
https://atmanco.com/blog/hcm/13-ways-lose-your-best-employees/
Azeri, I. (2012, November 4) How Expensive is it to Hire Startup Engineers?. Retrieved October
27, 2016, from https://venturefizz.com/blog/how-expensive-it-hire-startup-engineers
Baggelaar, H. (2008). Evaluating Programmer Performance. Retrieved from
http://homepages.cwi.nl/~paulk/thesesMasterSoftwareEngineering/2008/HiddeBaggelaar.
pdf
Barry, L., Garr, S., & Liakopoulos, A. (2014, March 4). Performance Management Is Broken:
Replace “Rank And Yank” With Coaching And Development. Retrieved October 21,
2016, from http://dupress.deloitte.com/dup-us-en/focus/human-capital-trends/2014/hctrends-2014-performance-management.html
Bergersen, G.R. (2015). Measuring Programming Skill- Construction and Validation of an
Instrument for Evaluating Java Developers. Retrieved Nov 23, 2016 from
http://folk.uio.no/gunnab/publications/Bergersen2015_PhD_thesis.pdf
Biro, M. (2016, January 14). Performance Reviews are Dead: Use Technology to Develop
Employees. Retrieved October 21, 2016, from http://www.talentculture.com/theperformance-review-is-dead-use-technology-to-develop-employees/
Brooks, F. P. (1995). The Mythical Man-Month, Anniversary Edition: Essays On Software
Engineering. Pearson Education. Retrieved from https://books.google.com/books?
id=Yq35BY5Fk3gC
Brynjarsdóttir, E. M., & Sigvaldason, G. (n.d.). Appraisal of people and dehumanization. In
Budapest. Retrieved from
Performance Appraisal Method for Software Engineers
103
https://philosophy.ceu.edu/sites/philosophy.ceu.edu/files/attachment/event/598/abstractsd
ehumanization-conffin160331.pdf
Caldwell, C. M. (2002). Performance Management: EBook Edition. Saranac Lake, US: AMA
Self-Study. Retrieved from http://www.ebrary.com
Cisco. (n.d.) CCNP Security. Retrieved from http://www.cisco.com/c/en/us/trainingevents/training-certifications/certifications/professional/ccnp-security.html
Danis, C., Thomas, J., Richards, J., Brezin, J., Swart, C., Halverson, C., Bellamy, R., Malkin, P.
(2008). Towards Applying Complexity Metrics to Measure Programmer Productivity in
High Performance Computing. SE-CSE 2008: 1st International Workshop on Software
Engineering for Computational Science and Engineering (Carver, et. al.)
Dias, L. P. (n.d.). Appraisal Methods. Retrieved October 22, 2016, from
http://2012books.lardbucket.org/books/beginning-management-of-human-resources/s1502-appraisal-methods.html
Engber, D., & Martinelli, M. (2016, August 28). Sad Face. Retrieved from
http://www.slate.com/articles/health_and_science/cover_story/2016/08/can_smiling_mak
e_you_happier_maybe_maybe_not_we_have_no_idea.html
Florentine, S. (2015, January 14). How to Improve Employee Retention. Retrieved October 24,
2016, from http://www.cio.com/article/2868419/careers-staffing/how-to-improveemployee-retention.html
Garvin D.A. (December 2013). How Google Sold Its Engineers on Management. Retrieved
November 14th, 2016 from https://hbr.org/2013/12/how-google-sold-its-engineers-onmanagemen
Performance Appraisal Method for Software Engineers
104
Google. (2014). Gradually Sunsetting SHA-1. Retrieved December 6, 2016 from
https://security.googleblog.com/2014/09/gradually-sunsetting-sha-1.html
Greene, J. (2014). Head first PMP. Sebastopol, CA: O' Reilly Media.
Gustafsson, J. (2011). Model of Agile Software Measurements. Retrieved from
http://publications.lib.chalmers.se/records/fulltext/143815.pdf
Havelka, D., & Merhout, J. (2009). Toward A Theory Of Information Technology Professional
Competence. Journal of Computer Information Systems, (Winter), 106–116.
Heathfield, S. M. (2016, August 14). The Good, the Bad, and the Ugly: 360 Degree Feedback.
Retrieved October 23, 2016, from https://www.thebalance.com/360-degree-feedbackinformation-1917537
Hruska, J. (2015). Details on Google’s Massive Cloud Infrastructure Revealed. Retrieved
December 6, 2016 from http://www.extremetech.com/extreme/208587-details-ongoogles-massive-cloud-infrastructure-revealed
Human Resource Management Team. (2010, July 24). Performance Appraisal Methods.
Retrieved from http://www.hrwale.com/performance-management/performanceappraisal-methods/
IEEE Computer Society. (2014). Guide to the Software Engineering: Body of Knowledge. (3rd
ed.). Piscataway, NJ: IEEE.
Investopedia. (2010, September 13). Management By Objectives - MBO. Retrieved October 25,
2016, from http://www.investopedia.com/terms/m/management-by-objectives.asp
Joseph, S. (2009). Programmer Competency Matrix. Retrieved from
http://sijinjoseph.com/programmer-competency-matrix/
Performance Appraisal Method for Software Engineers
105
Kanban Flow. (N.D.). The Pomodoro Technique and Kanban. Retrieved October 25, 2016, from
https://kanbanflow.com/pomodoro-technique
Kearney, O. K., Sedlmeyer, R. L., Thompson, W. B., Gray, M. A., & Adler, M. A. (1986,
November). Software Complexity Measurement. Communications of the ACM, 29(11),
1044-1050. Retrieved from
http://www.lsmod.de/~bernhard/cvs/text/dipl/dipl/papers/kearney.pdf
Kerzner, H. (2013). Project Management Metrics, KPIs, and Dashboards: A Guide to Measuring
and Monitoring Project Performance (2nd ed.). Hoboken, NJ: Wiley.
Kjerulf, A. (2015, May 29). 9 Reasons Performance Reviews are a Waste of Time. Retrieved
October 22, 2016, from https://www.linkedin.com/pulse/9-reasons-performance-reviewswaste-time-alexander-kjerulf
Kokemuller, Neil. (n.d.). What Is a Subjective Performance Evaluation?. Retrieved October 22,
2016, from http://smallbusiness.chron.com/subjective-performance-evaluation20453.html
Leviticus, J. (n.d.). The Effect of Negative Performance Evaluations on Future Performance.
Retrieved October 23, 2016, from http://oureverydaylife.com/effect-negativeperformance-evaluations-future-performance-37676.html
McCabe, T. (1976) A Complexity Measure. Retrieved from
http://www.literateprogramming.com/mccabe.pdf
McQuerry, L. (N.D.). The Effect of Negative Performance Evaluations on Future Performance.
Retrieved October 23, 2016, from http://yourbusiness.azcentral.com/effect-negativeperformance-evaluations-future-performance-14668.html
Performance Appraisal Method for Software Engineers
106
MeasuringU. (n.d.) 3 Ways to Combine Quantitative and Qualitative Research. Retrieved from
http://www.measuringu.com/blog/mixing-methods.php
Michael, V., Takafumi, Y., Yuta, G., & Kenta, I. (2013). Task Fidelity: a new metric for
measuring task complexity involving robots. Bulletin of the IEEE Technical Committee
on Learning Technology, 15(4). Retrieved from
http://www.ieeetclt.org/issues/october2013/Vallance.pdf
Microsoft. (n.d.) Silverlight Product Lifecycle. Retrieved December 6, 2016 from
https://support.microsoft.com/en-us/lifecycle/search/?c2=12905
Microsoft. (n.d.) Text files that are larger than 65,536 rows cannot be imported to Excel 97,
Excel 2000, Excel 2002 and Excel 2003. Retrieved December 6, 2016 from
https://support.microsoft.com/en-us/kb/120596
Milkovich, G. T., Wigdor, A. K., & Broderick, R. F. (Eds.). (1991). Pay for Performance :
Evaluating Performance Appraisal and Merit Pay. National Academies Press. Retrieved
from http://site.ebrary.com/lib/westerngovernors/reader.action?docID=10056863
Mind Tools editorial team. (n.d.) Building Good Work Relationships. Retrieved Nov 23, 2016
from http://mindtools.com/pages/article/good-relationships.htm
Murphy, J., Grynoviki, J. O., & Kysor, K. P. (2001, June). Case Study of a Prototype Set of
Behaviorally Anchored Rating Scales for C2 Assessment. Washington, DC. Retrieved
from http://dodccrp.org/events/8th_ICCRTS/Pres/track_6/2_1600Murphy.pdf
Nicolette, D. (2015) Software Development Metrics. Manning Publications.
Niehaus, M. (2014). How can you hire top engineers on a startup budget? Like this. Retrieved
October 26, 2016, from http://venturebeat.com/2014/09/25/how-can-you-hire-topengineers-on-a-startup-budget/
Performance Appraisal Method for Software Engineers
107
O’Hanley, R., & Tiller, J. S. (2013). Information Security Management Handbook, Sixth Edition.
CRC Press. Retrieved from https://books.google.com/books?id=a1rSBQAAQBAJ
Open Textbooks. (2015, October 28). Behaviorally Anchored Rating Scale (BARS). Retrieved
October 23, 2016, from http://www.opentextbooks.org.hk/ditatopic/32257
Perez, S. (2014) Apple Developers Must Now Agree To Ad Identifier Rules Or Risk App Store
Rejection. Retrieved December 6, 2016 from https://techcrunch.com/2014/04/11/appledevelopers-must-now-agree-to-ad-identifier-rules-or-risk-app-store-rejection/
Project Management Institute. (2013). A Guide to the Project Management Body of Knowledge:
(PMBOK® guide). Newtown Square, PA: PMI.
Rajput, V. (2015). Performance Appraisal System. Asian Journal of Nursing Education &
Research, 5(2), 287–292.
Reuters. (2016, August 12). SAP, Maker of Performance Review Software, Ditches Performance
Reviews. Retrieved October 21, 2016, from http://fortune.com/2016/08/12/sap-endsperformance-reviews/
Revesencio, J. (2015, July 22). Why Happy Employees Are 12% More Productive. Retrieved
October 24, 2016, from https://www.fastcompany.com/3048751/the-future-ofwork/happy-employees-are-12-more-productive-at-work
Rummler, G. A., & Branch, A. P. (1990). Improving Performance: How to Manage the White
Spaces on the Organization Chart. San Fransisco: Jossey-Bass
Šalková, A. (2013). Theoretical Approaches To Employee Appraisal Methods. Scientific Papers
of the University of Pardubice. Series D, Faculty of Economics & Administration, 20(28),
91–101.
Performance Appraisal Method for Software Engineers
108
Sauro, J. (2015, April 29). 3 Ways to Combine Quantitative and Qualitative Research:
MeasuringU. Retrieved October 24, 2016, from
http://www.measuringu.com/blog/mixing-methods.php
Schwaber, K., & Sutherland, J. (2016). The Scrum Guide: The Definitive Guide to Scrum : The
Rules of the Game [July 2016]. Retrieved November 23, 2016, from
http://www.scrumguides.org/docs/scrumguide/v2016/2016-Scrum-Guide-US.pdf
Smith, C.U. & Williams, L. G. (1998) Software Performance Engineering for Object-Oriented
Systems- A Use Case Approach. Santa Fe: Performance Engineering Services
Spolsky, J. (2000, August 30). Fog Creek Compensation - Joel on Software [Blog]. Retrieved
October 22, 2016, from http://www.joelonsoftware.com/articles/fog0000000038.html
Spolsky, J. (2004). Joel on Software: And on Diverse and Occasionally Related Matters That
Will Prove of Interest to Software Developers, Designers, and Managers, and to Those
Who, Whether by Good Fortune or Ill Luck, Work with Them in Some Capacity. Apress.
Retrieved from http://skillport.books24x7.com/assetviewer.aspx?
bookid=36726&chunkid=202993081&rowid=372
Spolsky, J. (2005, November 22). Reading List: Fog Creek Software Management Training
Program - Joel on Software. Retrieved October 22, 2016, from
http://www.joelonsoftware.com/articles/FogCreekMBACurriculum.html
Spolsky, J. (2009, January 1). Thanks or No Thanks. Retrieved October 22, 2016, from
http://www.inc.com/magazine/20090101/how-hard-could-it-be-thanks-or-no-thanks.html
Stevenson University. (2012). Malware Detection, Analysis, and Prevention. Retrieved from
http://catalog.stevenson.edu/graduate12/3603.htm
Performance Appraisal Method for Software Engineers
109
Sullivan, J. (2011, January 31). The Top 50 Problems With Performance Appraisals. Retrieved
from https://www.eremedia.com/tlnt/the-top-50-problems-with-performance-appraisals/
TechTarget. (2014). Access Control. Retrieved from
http://searchsecurity.techtarget.com/definition/access-control
TechTarget (n.d.) What is a Risk Assessment?. Retrieved October 26, 2016, from
http://searchcompliance.techtarget.com/definition/risk-assessment
Trauth, E. M., Quesenberry, J. L., & Huang, H. (2009). Retaining Women In The U.S. IT
Workforce: Theorizing The Influence Of Organizational Factors. European Journal of
Information Systems, 18(5), 476–497. https://doi.org/10.1057/ejis.2009.31
Trubiani, C. (2011). Automated Generation Of Architectural Feedback From Software
Performance Analysis Results (Unpublished PhD thesis). Universit ` a di L’Aquila.
Retrieved from http://www.di.univaq.it/catia.trubiani/phDthesis/PhDThesisCatiaTrubiani.pdf
University of Texas at San Antonio. (n.d.) Department of Information Systems and Cyber
Security. Retrieved from
http://catalog.utsa.edu/undergraduate/business/informationsystemscybersecurity/#cyberse
curity
University of Washingon. (n.d.). Core Courses. Retrieved from
https://www.uwb.edu/cybersecurity/curriculum/core-courses
Ward, P. (1997). 360-Degree Feedback. [Books24x7 version]. Retrieved from
http://common.books24x7.com/toc.aspx?bookid=4514Wilson, R. (2009, December 29).
The Assumptions Behind Performance Appraisal. Retrieved October 24, 2016, from
http://rwwilson.com/art45.shtml
Performance Appraisal Method for Software Engineers
Zaks, A., & Science, N. Y. U. C. (2007). Formal Verification Using Static and Dynamic
Analyses. New York University. Retrieved from https://books.google.com/books?id=30GIR1HWdwC
Zillman, C. (2016, February 1). IBM Is Totally Revamping Its Annual Performance Review.
Retrieved from http://fortune.com/2016/02/01/ibm-employee-performance-reviews/
110
Performance Appraisal Method for Software Engineers
Appendix A: Employee Hierarchy
111
Performance Appraisal Method for Software Engineers
Appendix B: Organizational Alignment
112