This is Part II of a three-part overview series of IIL’s newest product, the 360° Project Management  Competency Assessment (PMCA™) System 2.0.  In Part I, I discussed the basic concepts behind the product, the assessment experience, and touched briefly on the basic 360° Individual Assessment Report.   Please read Part I before proceeding.

In this post I will cover the default project management competency framework, the extensive configuration and customization options available in version 2.0, and the 360° Individual Assessment Report.

The Project Management Competency Framework

As discussed in Part I, the competency framework (domains, competencies, and a system of measurement)  is the foundation of the system.  PMCA 2.0 ships with a default competency framework with the following domains (below you will find a sampling of specific competencies as well):

1. Technical - Not to be confused with technology, this domain houses seventeen core project management competencies including Business Case Justification, Budgeting, Project Organization, Initiation Management, Project Plan Development, Time Management, Resource Allocation and Management, Cost Management and others.
2. Leadership - The Leadership domain contains six competencies regarding a project manager’s ability to lead, including Coaching, Conflict Management, Negotiation and others.
3. Personal - Networking, Written and Oral Communications, and three other competencies comprise Personal, indicating a project manager’s overall effectiveness as a contributing individual.
4. Business - Understanding how well a project manager understands her organization’s business is a critical metric.  There are four competencies in this domain, including Knowledge of Organization and Legal Aspects.
5. Technology- Many projects, and certainly most organizations, have a technical ecology that services a broad range of requirements from desktop productivity to enterprise reporting.  The Technology domain contains four competencies, including Systems and Technology Integration and Application Standards, Procedures, and Policies.

The competencies above are simply labels for detailed competency statements.  Competency statements are constructed such that an individual can rate experience, ability, or skill with respect to the competency.   For example,  the Technical domain contains the Integrated Project Control and Reporting competency, for which the competency statement reads:

Application of knowledge and experience with tools, techniques and processes for collecting, analyzing and reporting project information based on the project baselines and using all PM processes. Measuring progress and identifying deviations from the project management plan, initiating corrective actions, preventative actions and change requests accordingly.

The final component of the project management competency is a system of measurement.   As mentioned in Part I, PMCA employs a scaled continuum ranging from least experience, ability or skill to most.  Known as a Likert scale, this approach allows for a standard scale to be applied to all competency statements across all domains.  Furthermore, PMCA incorporates scale item descriptions for each competency statement, providing additional detail to the assessor regarding the specific experiences, skills, or abilities required to rate at a certain level.

For example, the image below is from the assessment itself, and lays out how the domain, competency, Likert scale, and scale item descriptions are related.

Covered in detail later in this post, the default project management competency framework is just that: the default.  The system is designed to accommodate a client’s existing framework or a combination of client competencies and default competencies.  This flexibility extends throughout the framework modeling environment, allowing for a variable number of domains, competencies, and a configurable system of measurement (as long as the system is derived from a Likert scale).

The default project management competency framework offers a glimpse into the thinking that went into designing the default project management competency framework.  Industry experts from a variety of backgrounds were brought together to create a broadly applicable framework, drawing from a combined experience totaling many decades of time on the job as a project manager.

Please contact  Christian Damiba for details on the complete project management competency framework:

Christian Damiba , PMP
Tel: +1-212-515-5150 or
1-800-325-1533, ext. 5150
email: Christian.Damiba@iil.com

Configuration Options

Defining the project management competency framework (or deciding to use the default framework) is the first step towards performing an assessment.   The assessment is delivered to participants by way of a session.  Consider the following scenario:

1. A large construction company has defined a project management competency framework that is broadly applicable across multiple project manager types, such as information technology project manager, construction project manager, and human resources project manager.
2. There are various attributes that need to be collected at the time of assessment about the project manager, including Title, Job Grade, and Location.
3. The team managing the assessment process has decided that four types of assessors will participate in the 360° process:  Self (the project manager), Manager, and two Team Members.
4. In the future, attributes and assessor types will change.

A session provides a mechanism by which attributes and assessor types can be defined for a project management competency framework at a given point in time (for example: Round 1 Assessment, followed by a Round II Assessment).   The session defines the assessment period (start and end date).   Finally, when a session is defined, all of the configurable elements such as the Welcome Screen, Invitation Emails, and 360° Assessment Report are designed and created. 

It is expected that many sessions will be defined and delivered for an organization’s competency framework.  In fact, it may be that a single competency framework is insufficient to meet an organization’s requirements (as may be the case when there are distinct competencies for specific types of project managers).  The combination of one or more competency frameworks and sessions allows for a great variety applications.

Sessions are also important in reporting.   Once a session has been delivered and the data collected, an organization may decide to deliver another session the next quarter or following year, and perform gap analysis to determine if the project managers are improving over time.    An important aspect of PMCA is IIL’s ability to help our clients develop training programs to help close any gaps uncovered by the system.

Targets

Many organizations have organizational performance levels, grades, or expected performance implied by job title.   For example, a Level I Project Manager may be expected to perform (or have different responsibilities)  differently than a Level IV Project Manager.  In fact, this scenario is  quite common.  

In PMCA, this performance-related-to-level is known as a target.  Targets are a configuration option that allows a client to specify particular performance criteria for different groups of project managers.  The target is coupled to one of the attributes discussed above (for example, Title, Job Grade or Level).  For each competency a value is specified within the Likert scale’s minimum and maximum.   For example (competencies are in no implied order or importance):

Of course, this is a very simple example.   Meaningful targets are selected based on a detailed analysis performed either by the client organization or in conjunction with IIL subject matter experts.

In the 360° Individual Assessment Report (covered in detail at the end of this post), the individual’s target it displayed much the same way her manager or team member’s results are displayed.  This allows the project manager to view her performance related to target against any of the participants who rated her.

Furthermore, customizations can be made to the report so the project manager sees as single number (essentially a variance from target) that indicates whether or not improvement is required. 

Designer Elements

In addition to creating custom project management frameworks and defining attributes, assessor types, and targets on a session, PMCA provides for many configurable designer elements.  An example of a designer element is the Welcome Screen, seen below:

In a subsequent post, I will provide details on designer element options, concepts, and maybe show you some of the customizations we have done to date. 

Reporting Options

The essential purpose of PMCA is to solicit feedback about a project manager’s competencies and reflect that back to the project manager and her organization.  PMCA 2.0’s primary output is the 360° Individual Assessment Report.   PMCA also generates organizational reports that roll up averages by Job Grade, Location, or Product Group (as an example).

The default 360° Individual Assessment Report looks like the image below.  In the grid area, the project manager’s results are organized top to bottom by assessor type  (Self, Manager, two Team Members, and Target) and by competency left to right (1-37 in the default project management competency framework).    At the very button of the report, you can see that a Team Average is provided (it is the average of the Team Members, not including the manager), along with a Ration-to-Target that indicates at a single glance whether the project manager is on target, below target, or above target.

Other report highlights include the ability to switch chart types from a column, to a line, or to a scatter chart.  Also, the checkboxes in the sample above allow the project manager to filter results by assessor type (show me how I did according to my manager and my target, for example).

Because the report is a Microsoft Excel workbook, the potential reporting and charting scenarios are endless.  In addition to the report above, the report workbook also contains a competency reference (so the project manager can review the domains and competency right from the report) as well as proficiency definitions (what does it mean that I rated a "4" in Budgeting?).

That’s it for today!  Next up is a discussion about how organizations manage and track ongoing assessments.

Introduction

This is Part I of what will be a series of posts which will give you a good understanding of how 360° PMCA™ Version 2.0 (PMCA) works.

PMCA is a competency assessment system designed to collect feedback regarding an individual’s project management competencies from multiple sources.  The multi-source assessment method is known as the 360° assessment method. 

By collecting a range of viewpoints regarding an individual’s competencies, the system offers interesting avenues for the analysis and planning of project management training programs, standardization initiatives, and personnel improvement programs (PMCA supports the traditional manager-subordinate assessment approach as well). Go here for more information on benefits and highlights.

Competencies are the core of PMCA and are defined as the skills, experiences, behaviors, or abilities required to perform a job or activity.  Related competencies are arranged in domains.  Domains provide two benefits: first, the thinking undertaken to define competencies and group them by domains increases focus on the competency framework’s underlying logic.  Second, reports that "roll-up" assessment results by domain assist in analysis.

Next, a PMCA competency framework defines a system of measurement.  PMCA’s measurement system is based on a configurable Likert scale coupled with optional competency-specific descriptions of abilities, behaviors, or skills associated with performing at a particular level.   See Figure 1 below for an example.

Figure 1: Sample Competency Statement and Rating Scale

Organizations with a mature project management ecology often have project management competencies defined and may even have considered domains as an organizing principle.  A key design goal (design goals will be discussed in detail in another post) was to provide a framework modeling environment that would accommodate a customer’s framework, IIL’s framework, and any blend of the two.   Towards this goal, PMCA allows for a variable number and structure of domains, competencies, a configurable Likert scale, and a wide array of other configurable elements.  Visit http://www.iil.com/360pmca/customization.asp for information about other configuration options.

The Assessment Experience

The individual at the center of the 360° experience is known as the assessee.   Participants assessing the assessee are known as assessors.  PMCA supports configurations with as few as two participants (assessee plus one assessor, typically used in traditional manager-subordinate assessments) or as many as ten participants.  Assessor types are configurable.  An assessment might have a manager assessor, three team member assessors and a customer assessor.  Assessors are invited by the assessee to participate and are sent an customizable email invitation to kick off the process.

In addition to configurable framework elements such as domains and competencies, PMCA provides for the definition and collection of custom attributes.  Custom attributes are data elements associated with the assessee such as business unit, location, job grade, or any other data element appropriate for a customer’s scenario.  Custom attributes are used in reporting and analysis to group assessment results.  Questions such "what is the average rating in the Personal Skills domain for project managers with 5 years or more of experience?" can be asked of the system because of these custom attributes.

The screen shots below take you through the assessee’s experience of signing up, filling out their profile (specifying values for the custom attributes), inviting others to assess, and starting the assessment.  Note that this example used a simplified Likert scale rather than the detailed competency-specific descriptions mentioned previously–yet another configuration option.

Figure 2: Sign-Up

Figure 3: Complete Profile

Figure 4: Invite Others

Figure 5: Self-Assessment

Invitations are sent via email to the individuals for whom email addresses where provided for in Figure 5.  Figure 6 shows a sample of the invitation email (also customizable).

Figure 6: Sample Invitation Email

Clicking the embedded link takes the assessor to a Start Page where basic information is collected before starting the assessment.  Figure 7 shows the assessor Start Page.

Figure 6: Sample Invitation Email

We will end off today’s post with a sample report.  What you see below is generated for the assessee (again, the person at the center of the 360° assessment).  The report is highly configurable.  What you see below shows the assessee’s self-rating, two of his team members’ rating, his manager’s rating, and a target (we will discuss targets in a subsequent post).

Part II will cover specifics regarding the default project management competency framework, configuration options, administration, and reporting.