Extension Evaluation Matters: 2nd Edition

EVALUATION TECHNIQUES

Introduction

When collecting primary data for the purposes of program evaluation, there are multiple techniques to consider. These include:

Surveys

Direct Observation

Focus groups and interviews

Selecting the right technique depends on the evaluation purpose and type of data of interest (quantitative vs. quantitative). Figure 9 below provides additional information.

Methods

Useful for Providing

Advantages

Limitations

Key Informant Interviews

-general descriptive data

-provides in-depth, inside information -flexibility permits exploring unanticipated topics

-does not generate quantitative data

-understanding of attitudes and behaviors

-susceptible to interviewer and selection bias

-suggestions and recommendations

-easy to administer

-information to interpret quantitative data

-relatively inexpensive

-takes 4-6 weeks

Focus Group Interviews

-customer views on services, products, benefits -information on implementation problems -suggestions and recommendations for improving activities -data on physical infrastructure, supplies, conditions - information about an agency’s delivery systems, services

-can be completed rapidly (5 weeks)

-does not provide quantitative data

-very economical

-discussion may be dominated by a few individuals

-group discussion may reduce inhibitions, allowing free exchange of ideas

-susceptible to moderator bias

Direct Observation

-phenomenon can be examined in its natural setting

-susceptible to observer bias

-act of observing can affect behaviors -distortions can occur if sites selected are not representative

-may reveal conditions or problems informants are unaware of

-insights into behaviors or events

-can be completed in 3-4 weeks

Surveys

-quantitative data on narrowly focused questions -when probability sampling is difficult -data on attitudes, beliefs, behaviors of customers or partners

-can generate quantitative data

-susceptible to sampling bias

-reduces non-random sampling errors

-requires statistical analysis skills -inappropriate for gathering in-depth, qualitative information

-requires limited personnel

Figure 9: Methods and uses.

42

Powered by