Diaries of Senior UI/UX Designer Desarae Veit who loves to read, write, travel, and so much more. Hobbies include aviation, yoga, scuba diving, barre, and art or craft projects. Blog includes Book Reviews and Gadget Reviews.
“A distinguishing feature of evaluation is the universal focus on stakeholder perspectives, ......,and evaluation are judged on if and how stakeholder perspectives are included” (Matheson, 2008).
So how to engage stakeholders and include the stakeholder perspectives during the evaluation process? (You can share your own experiences/stories or talk about the approaches you have heard of/read)
Page 194 of Mathison (2007), "A distinguishing feature of evaluation is the universal focus on stakeholder perspectives, a feature not shared by social science research. And evaluations are judged on if and how stakeholder perspectives are included. While evaluation models are based on different epistemological foundations, all evaluation models attend to stakeholder perspectives-the assessments, values, and meanings of the evaluand's stakeholders are essential elements in any evaluation." .... "When research participants are referred to as stakeholders, it is often a reference to whom the data are collected from rather than a consideration of the stakeholder's vested interests. "
How to engage stakeholders and include the stakeholder perspectives during the evaluation process?
Factors: Internal vs. External
How to engage with stakeholders depends on how you know the stakeholders. Evaluators, as discussed in Fitzpatrick, J., Sanders, J. & Worthen, B. (2011), can be internal or external evaluators. If you work within a company and are evaluating your own company, you are more likely to have a vested interests and therefor more likely to have bias, unless it's (ideally) for a client or other department (which would make you an external evaluator).
Evaluators also come into a project or evaluation with some knowledge of a proposal, business requirements, vision document, or other criteria. Often, or in the best case scenario, an evaluator would be part of writing, strategizing, and reviewing these documents. These documents are the foundation or backbone of what is known about stakeholders and the criteria for evaluation.
In practice - Factors
There are a lot of ways to engage stakeholders and include stakeholder perspectives in the evaluation process. Most of these will likely depend on a number of factors like:
The type of business (B2B or Business to Business or B2C Business to consumer)
The industry (advertising agency, government, retail, IT, education, policy/HR)
Availability of quality assurance (QA) testers
Background of the evaluator (social science, technology, design, level of training, government, policy, science, health care, specialized expertise and willingness to learn more about the project/program)
Ability to learn more about the program/put the evaluator in the shoes of the stakeholder
Whether or not the stakeholders are minors
Type of data being evaluated
Laws and legislature - and whether or not those laws/legislature change during the evaluation (this happens A LOT in health care)
Stakeholders with financial/political/bias interests opinions
I've used a number of methods to engage client stakeholders and users for user experience testing, design, and improvement of software systems. Some of those methods include:
Meetings - for financial stakeholders
Brainstorming sessions - for financial stakeholders
Surveys - for user stakeholders you can contact
Behavioral flows - for user stakeholders you might not be able to contact easily
Analytics - for user stakeholders you might not be able to contact easily
Data tracking - for user stakeholders you might not be able to contact easily
Eye tracking software
Giving users a list of tasks and watch them complete the tasks, then ask some users to talk about their struggles and video the process
Training tools and walk-throughs with tracking software (scripted use of product)
Natural use of a product (paid shoppers)
Tracking search forms
Personas - This is a great way to always keep stakeholders who are not present or can not represent themselves or a large few groups of people easily.
Ethnographic field studies
First click testing
Heuristic evaluation/expert review
Mathison, Sandra (2007). What is the difference between evaluation and research? And why do we care? In N.L. Smith & P. Brandon (Eds.). Fundamental issues in evaluation. New York: Guilford Publishers. https://books.google.com/books?id=l6oEU80PWwsC&pg=PA194&lpg=PA194&dq=A+distinguishing+feature+of+evaluation+is+the+universal+focus+on+stakeholder+perspective&source=bl&ots=7lmxhjF5vO&sig=5McBFL0N_DyEBAP9jc1T6qyXeU8&hl=en&sa=X&ved=0ahUKEwjbmp2S9dfKAhXDXhoKHSeDDgAQ6AEIHTAA#v=onepage&q=A%20distinguishing%20feature%20of%20evaluation%20is%20the%20universal%20focus%20on%20stakeholder%20perspective&f=false
Fitzpatrick, J., Sanders, J. & Worthen, B. (2011). Program Evaluation: Alternative Approaches and Practical Guidelines (4th ed.), Chapter 1. New York: Pearson.