CHOOSING 360
A GUIDE TO EVALUATING MULTI-RATER FEEDBACK INSTRUMENTS FOR MANAGEMENT DEVELOPMENT
CHOOSING 360
A GUIDE TO EVALUATING MULTI-RATER FEEDBACK INSTRUMENTS FOR MANAGEMENT DEVELOPMENT
Ellen Van Velsor
Jean Brittain Leslie
John W. Fleenor
Center for Creative Leadership
Greensboro, North Carolina
The Center for Creative Leadership is an international, nonprofit educational institution founded in 1970 to advance the understanding, practice, and development of leadership for the benefit of society worldwide. As a part of this mission, it publishes books and reports that aim to contribute to a general process of inquiry and understanding in which ideas related to leadership are raised, exchanged, and evaluated. The ideas presented in its publications are those of the author or authors.
The Center thanks you for supporting its work through the purchase of this volume. If you have comments, suggestions, or questions about any CCL Press publication, please contact the Director of Publications at the address given below.
Center for Creative Leadership
Post Office Box 26300
Greensboro, North Carolina 27438-6300
336-288-7210 • www.ccl.org
©1997 Center for Creative Leadership
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher.
CCL No. 334
Library of Congress Cataloging-in-Publication Data
Van Velsor, Ellen
Choosing 360 : a guide to evaluating multi-rater feedback instruments for management development / Ellen Van Velsor, Jean Brittain Leslie, John W. Fleenor.
p. cm.
Update ed. of: Feedback to managers, vol. 1. 1991.
Includes bibliographical references (p. ).
ISBN 1-882197-30-5 [ISBN-13: 978-1-882197-30-9]
1. Organizational effectiveness—Evaluation—Methodology. 2. Feedback (Psychology). 3. Executives—Rating of. I. Leslie, Jean Brittain. II. Fleenor, John W. III. Morrison, Ann M. Feedback to managers. IV. Title.
HD58.9.V36 1997
658.4'03—dc21
97-18853
CIP
Table of Contents
STEP 1: FIND OUT WHAT IS AVAILABLE
STEP 2: COLLECT A COMPLETE SET OF MATERIALS
STEP 3: COMPARE YOUR INTENDED USE TO INSTRUMENT CHARACTERISTICS
STEP 4: EXAMINE THE FEEDBACK SCALES
STEP 5: FAMILIARIZE YOURSELF WITH THE INSTRUMENT-DEVELOPMENT PROCESS
STEP 6: LEARN HOW ITEMS AND FEEDBACK SCALES WERE DEVELOPED
STEP 7: FIND OUT HOW CONSISTENT SCORES TEND TO BE
STEP 8: ASSESS BASIC ASPECTS OF VALIDITY—DOES THE INSTRUMENT MEASURE WHAT IT CLAIMS TO MEASURE?
STEP 9: THINK ABOUT FACE VALIDITY
STEP 10: EXAMINE THE RESPONSE SCALE
STEP 11: EVALUATE THE FEEDBACK DISPLAY
STEP 12: UNDERSTAND HOW BREAKOUT OF RATER RESPONSES IS HANDLED
STEP 13: LEARN WHAT STRATEGIES ARE USED TO FACILITATE INTERPRETATION OF SCORES
STEP 14: LOOK FOR DEVELOPMENT AND SUPPORT MATERIALS
STEP 15: COMPARE COST—VALUE FOR THE PRICE
STEP 16: CONSIDER LENGTH A MINOR ISSUE
Instrument Evaluation Checklist
Foreword
At the end of 1991, CCL published the two-volume Feedback to Managers (Volume I: A Guide to Evaluating Multi-rater Feedback Instruments; Volume II: A Review and Comparison of Sixteen Multi-rater Feedback Instruments). Since that time, there has been a notable increase of interest in multi-rater, or 360-degree, instruments: More are available; they are being used for a wider range of purposes; and much has been learned about their use. A revision is thus in order. What you will find here is an updated edition of the first volume (with a new edition of the second currently in progress).
The text has been thoroughly reviewed by the authors—Ellen Van Velsor and Jean Brittain Leslie, who wrote the original edition, and John Fleenor, who joined the team for this version—and changes have been made to reflect current understandings in the field—for instance, new information on the proper use of norms and on “item banks” (a collection of instrument items that have been tested for reliability and validity) has been added, and the section on validity has been redone.
You will note that this work has been retitled and that it is no longer referred to as the first volume of a set. We wanted to make this guide, which can help practitioners begin to make sense of the complexity and proliferation of instruments, as visible as possible. The update of the second volume, also to be a stand-alone publication, will aid practitioners by providing in-depth descriptions of selected instruments, including some that were not included in the previous edition, and discussion of key issues in their use. The two can still, of course, be used together, and we invite people to do so.