In  Part 1 of this article, I discussed the Capability Maturity Model (CMM) and how prevalent it has become in assessing an organization's performance. This is good news in terms of raising awareness of the importance of processes. However, for some organizations that will never go above level 1, it makes me wonder if it makes sense to measure them against the five-level CMM.

We discussed the nature of organizations, the nature of assessments, and the nature of models in part 1. I related the example of a simple hotel survey that read "How are we Doing?" at the top of the survey card. Although assessing your IT organization's performance may be more involved than assessing a hotel's performance, there are some points in common. The main point is that the survey does not have to the involved to be effective.

The Assessment Process

Self-assessment is Not Enough

There are two views to this assessment - Internal (Our View of How We are Doing) and External (Our Customers' and Users' View of How We are Doing). It is important to consider both views as we may think we're doing great, but those we serve may have a different perception. This assessment can also identify where the perceptions differ.

While self-assessments can help us identify strengths and weaknesses in our performance, the self-view is inherently biased, even if performed by an independent third party. An independent assessment is more objective than one performed by the organization, however, the assessor(s) are at some points dependent on the information they obtain from within the organization.

Points of Measurement

In this informal assessment, there are more internal than external measures. This is because we want to keep the assessment short and simple for the customer, but also because there are many things the customer can't rate, such as problem solving.

Keep in mind that this assessment is a sample starting point. You can add or delete criteria as appropriate to your organization's needs. The overall objective is for the assessment to be simple, especially for the customer.

Internal Measures

Responsiveness

• Timeliness
• Delivering what was requested

Teamwork

• Cooperation

Communication

• Lines
• Accessibility
• Common Terminology
• Directions

Problem Solving

• Definition

Learning

• Measurement
• Reflection/Review

Practices

• Consistency

• Awareness

 Competency

• Management
• Team

Leadership

• Vision
• Influence
• Commitment to Goals
• Commitment to Quality
• Commitment to Customers

External Measures

Responsiveness

• Timeliness
• Delivering what was requested

Communication

• Lines
• Accessibility

Teamwork

• Cooperation with customer

Competency

• Management
• Team

Practices

• Consistency

Designing the Survey

Simplicity is the goal, so with that objective in mind, here is how I have created a sample survey.

How Are We Doing?

An Internal Performance Survey

	Score 1 = poor, 5 = wow!	Comments
Responsiveness Was the project delivered when promised?	1    2     3     4      5
Did the project deliver what was requested?	1    2     3     4      5
Teamwork Did the project team exhibit a spirit of cooperation?	1    2     3     4      5
Communication Were the lines of communication understood by the project team?	1    2     3     4      5
Were people accessible to speak with during the project?	1    2     3     4      5
Was there a common set of terminology used?	1    2     3     4      5
Did communication occur both to and from people?	1    2     3     4      5
Problem Solving How well was the problem initially defined?	1    2     3     4      5
Learning Were key items measured on the project, such as time and defects?	1    2     3     4      5
Was there a set-aside time for people to reflect and review how the project was performed?	1    2     3     4      5
Practices Did the project follow a method consistent with other projects performed in this organization?	1    2     3     4      5
Did the project team show an awareness of the value of consistency with other projects?	1    2     3     4      5
Competency Did management demonstrate competency in running the project?	1    2     3     4      5
Did the project team demonstrate competency in performing the project?	1    2     3     4      5
Leadership Did the leadership on the project clearly communicate their vision of what was to be accomplished?	1    2     3     4      5
Did project leadership exhibit the ability to influence people?	1    2     3     4      5
Did project leadership show a commitment to achieving project goals?	1    2     3     4      5
Did project leadership show a commitment to delivering a quality project?	1    2     3     4      5
Did project leadership show a commitment to meeting customers' needs?	1    2     3     4      5

Did anyone in particular on the project exhibit an attitude that made a positive difference in the outcome of the project? If so, who?

Did anyone in particular in the customer area exhibit an attitude that made a positive difference in the outcome of the project? If so, who?

What one thing would you suggest doing differently on the next project?

How Are We Doing?

Customer Survey

	Score 1 = poor, 5 = wow!	Comments
Responsiveness Was the project delivered when promised?	1    2     3     4      5
Did the project deliver what was requested?	1    2     3     4      5
Teamwork Did the project team exhibit a spirit of cooperation?	1    2     3     4      5
Communication Were people accessible to speak with during the project?	1    2     3     4      5
Did communication occur both to and from people?	1    2     3     4      5
Problem Solving How well was the problem initially defined?	1    2     3     4      5
Learning Was there a set-aside time for people to reflect and review how the project was performed?	1    2     3     4      5
Practices Did the project follow a method consistent with other projects performed in this organization?	1    2     3     4      5
Competency Did management demonstrate competency in running the project?	1    2     3     4      5
Did the project team demonstrate competency in performing the project?	1    2     3     4      5
Leadership Did the leadership on the project clearly communicate their vision of what was to be accomplished?	1    2     3     4      5
Did project leadership exhibit the ability to influence people?	1    2     3     4      5
Did project leadership show a commitment to achieving project goals?	1    2     3     4      5
Did project leadership show a commitment to delivering a quality project?	1    2     3     4      5
Did project leadership show a commitment to meeting customers' needs?	1    2     3     4      5

Did anyone in particular on the project exhibit an attitude that made a positive difference in the outcome of the project? If so, who?

Did anyone in particular in the customer area exhibit an attitude that made a positive difference in the outcome of the project? If so, who?

What one thing would you suggest doing differently on the next project?

How to Gather Results

The sample survey questions are written to be answered in the context of a single project. The ideal time to gather this kind of information would be in a project retrospective or post-project review. To track progress, an assessment could be performed at the end of every project.

However, if your organization doesn't work on a project basis, or if you just want to get an overall snapshot of your performance, you can change the wording of the questions to a present tense. I recommend the project assessment, if possible, as it gives people something tangible to think about in answering the questions.

You can distribute the survey to people individually, or have people fill them out during a project review or retrospective.

How to Compile the Results

After you have gathered the survey results, you can use a spreadsheet to record all of the responses. The spreadsheet will give you the capability to average and chart the responses.

How to Make Sense of the Results

For this kind of survey, a simple bar chart is fine to show the results. Each major point of assessment can be a bar (Figure 1). It is easy to show the results of several assessments on a single bar chart (Figure 2). You can also show your goals on the chart.

Figure 1 - The First Assessment Results on a Bar Chart (Baseline)

Your first assessment will be your initial baseline. Your initial baseline might be low, but that's alright. The important thing is to be able to have a way to compare future assessments. I would recommend not looking to other organizations to benchmark with on this assessment because their environments and customers are different than yours.

Figure 2 - Comparison of Assessment Results to a Baseline

A Final Word of Warning

It is possible to go overboard with measurements and this technique is no exception. One thing that really irritates me is to be the respondent of a survey where the surveyor seems more concerned with getting the numbers recorded instead of listening to how I feel about a recent situation. You may find that listing to a single customer complaint reveals more information than this little survey scorecard.

Summary

There are many ways to measure your organization's performance. You may choose the CMM, CMMI, or any number of other frameworks and models. The assessment in this article is to give a way to those who want a simple way to know how they are doing and want to improve.

If you decide to use this assessment, be sure and let me know how it goes for you. You can write me at the contact us page.