Since 1991, Andrea L.K. Johnston has been employed full-time in working towards a future that changes the relationships between Indigenous Peoples and funding agents to operationalize an environment in which Indigenous Peoples can set their own parameters. In 2016, she launched the first-in-the-world training curriculum in Indigenous evaluation practice, titled “Honouring Reconciliation in Evaluation.” Andrea seeks to deconstruct the language and assumptions behind evaluation to support the unfolding of a reconciliation process that changes the ways evaluations are operationalized.
Andrea L. K. Johnston continues to develop and launch Tools for Change, to meet the goal of Indigenous-led evaluation. Andrea has worked on Indigenous programs and services evaluations, full-time, for 20 plus years. Andrea L.K. Johnston is a Credentialed Evaluator with the Canadian Evaluation Society (since 2011), descendant of Chippewas of Nawash, graduate of the University of Toronto, and member of the Canadian Council for Aboriginal Business.
This paper was written for the Department of Justice Canada for a project that explores, considers, and identifies Indigenous approaches and methods for evaluating services and supports for Indigenous victims and survivors of crime. This project also explores principles and methods that ought to be considered in designing evaluation frameworks and methods (including tools and processes). There is also interest in understanding how these approaches could be used to inform social science research studies.
There is a need to reconsider our approaches and methods to evaluative practices. There have been many adaptations made to the field of evaluation; however, these have been made by Western technicians. Current publications around Indigenous evaluation have centered on the augmentation and Indigenization of evaluation practice. However, what we really need are transformative and wholistic Footnote 6 [sic] conceptualizations of Indigenous evaluation. When we speak of reconciliation, we are referring to a two-way street or river, upon which the two-worlds glide by, each strong and independent of the other. At the crux of the issue is the fact that evaluations are still done to Indigenous Peoples. Western practitioners can craft an evaluation and mold it into whatever form they want, but at the end of the day, if it is Western practitioners steering and directing the evaluation – then evaluations are still being done to Indigenous Peoples, and reconciliation is never realized. The difficulty is that many Indigenous Peoples cannot untangle their minds from Western thought – many wear a colonized mind, largely from the Residential School System and its intergenerational impacts. However, the Elders say that without Indigenous culture and traditions, Indigenous Peoples cannot become whole, independent, and strong again. This paper will discuss several ways to bring Indigenous knowledge and culture into evaluation practice, including the Waawiyeyaa Evaluation Tool Footnote 7 . Ithas changed the focus of data collection to an intervention method, since it incorporates Indigenous knowledge and culture and encourages individuals to share their experiences while they sit in the driver’s seat of the data exchange exercise.
Working from a place of reconciliation in evaluation methodology starts with discussions on what constitutes truth. Truth occurs from many perceptions and dimensions. Truth is the essence of what evaluation is designed to uncover. Scientific inquiry seeks to uncover the physical truths of an object that is being studied. However, Indigenous Peoples are challenging evaluation to go beyond the physical understanding of whether an intervention is deemed successful or not. Given the complexity of understanding truth, evaluation too must challenge itself to embrace this complexity. By embracing the complexity of truth, evaluation can begin to operate from a place of reconciliation. It is within the path of reconciliation practice and understanding that we can begin to truly understand why and how some programs are well-oiled machines and others struggle.
Incorporating reconciliation practice brings a unique understanding to every evaluation. It is these special insights which ignite the alternative experiences of individuals impacted by the evaluation. Reporting must reflect this diversified knowledge. This paper describes a unique Indigenous framework of understanding from approach, evaluator roles, program design, and management, while contextualizing each area in the context of evaluating Indigenous specific victims’ services.
When thinking about how we bring reconciliation into evaluation practice, one must first recognize Indigenous evaluation practice not as a destination, but rather a journey. This exercise is not easy. It is about having a deep, grassroots understanding of ways of knowing, ways of being, ways of experiencing, and ways of doing. While knowing refers to a frame of mind, being refers to trauma informed relationships and practices, experiencing is contextualized within one’s spiritual connections, and doing refers to the actions one is willing, able and competent to undertake. Decolonization, even for me, an Anishnawbe-que, is a journey, an ongoing process. The more we decolonize, the more we can support and implement evaluations that honour reconciliation.
One method of decolonization is coming to understand how communities view evaluation and change the ways evaluation impacts upon community members. Sitting together and discussing the aspects of a program is a rare luxury in an environment that is challenged by continual crisis management. Many communities are under-staffed, under-resourced, and find evaluation confusing and taxing on their time. Many staff resent evaluation practice since they view it as taking time away from the clients they are serving. In many cases, staff have to book off one day of the week for administrative duties and evaluation is one of many items on that day’s agenda.
Another challenge is that while there is a lot of knowledge built-up documenting evaluation, it is an evolving practice. This makes it difficult for communities to keep up-to-date and learn new concepts. Another reality is that evaluation has only been a requirement built-into funding agreements of the Canadian federal government for the past 20 years. Evaluation took on a narrow definition of conduct with the onset of the logic model; however, it took about 10 years to popularize the logic model.
To understand what changes need to happen in evaluation practice, we must first understand how we got to the point evaluation practice is at right now. Evaluation has held tightly onto logic models and the ‘theory of change’ Footnote 8 for the past 20 – 30 years. These evaluation methods have the pre-conceived notion that the logic model must be built prior to program implementation, rather than built at the program development stage. Many federal government programs have a logic model developed in a centralized office, which has its input facts primarily based within a theory of change model. The theory of change which helps to predict the path of behaviour change in program participants is often developed without sufficient insight from those delivering the program or service.
The field of evaluation changed drastically in the early 1990s when the theory of change (1990) and the logic model (1997) became exciting, new and interesting topics at the American Evaluation Association conference in Chicago, Illinois, 1997. While, Chen in 1983 was among the earliest to document an evaluation that applied the theory of change, in 1991 Rush and Ogbourne were among the first to publish the use of logic models. It was more common for articles to discuss theory of change from 1998 onwards (Francis 1998, Barley & Phillips 1998). As well by 1997 numerous articles on logic models had been published (Alter 1997, Funnel 1997, Julian 1997, McEwan & Bigelow 1997, Moyer 1997). Subsequently, Carol Weiss published the 2 nd edition of “Evaluation” in 1998, a text that supports the logical reasoning of the logic model and defines a common approach for theory of change.
While Canada published logic models in the early 1990s (Corbeil 1986, Wong-Reiger & David 1995, 1996), logic models were not yet adopted into federal government systems. It was not until the early 2000s when the Canadian federal government introduced the logic model and its reasoning into program work plans and requests for proposals. This created an environment for conformity to the new logical sequential reasoning of logic modelling. It is also significant to report that the early publications on logic models used language such as “program’s performance story” (McLaughlin & Jordan 1997), “focus health services on population health goals” (McEwan & Bigelow 1997), “strengthen service program development” (Hermann 1996), and “adaptable tool for designing and evaluating programs” (Funnell 1997). By 1998, the publications on logic modelling evolved to focus on outcomes and indicators (Francis 1998, American Cancer Society 1998, Bell & McLaughlin 1998).
Starting from a place of reconciliation and understanding provides an in-depth understanding of the role and function of the program in the larger community and national environments. The focus of a single evaluation needs to be bigger than a single program and a single community. In Indigenous teachings there is the idea of a Spirit of a People and a National Spirit. There is also a global spiritual connection and even connections to the universe and across time. The majority of current evaluations assess programs from a funder’s perspective, which values the funder’s perspective over the community’s values and priorities. The reconciliation of evaluation demands that the spiritual essence of a program is the starting point for evaluation practice (Public Safety Canada 2014; Health Canada and the Public Health Agency of Canada 2013). Taking an approach that starts in a place of reconciliation is truly wholistic and ensures that every step in the evaluation is related to this basic onset. This beginning stage also considers many variables and factors influencing the program. However, this exceeds the current methodology of typical evaluation practice. There are a few instances of evaluations from a place of reconciliation; however, there are few evaluations that are based in a broader community or national context. It is these evaluations that we want to discuss in terms of a model for the next wave of evaluation transformation.
We are at a precarious stage of examining Indigenous evaluation. The following are questions and actions that come from a place of reconciliation. These would guide an evaluation process to be more inclusive of Indigenous thought and actions.
It is important to consider who the key evaluation contributors are – these are a variety of individuals and those who receive some form of compensation for their “professional services.” If one does not value such expertise and experience within a bartering relationship that expresses value for their time and effort, the productivity will suffer, as it has in the past. Instead of igniting creativity and ingenuity, we will produce a mediocre evaluation project that does little to create change and inspire transformation.
The rest of the paper will focus on the previously mentioned eight questions. These eight questions are discussed as a set of guiding principles for evaluation practice. These principles each discuss a step-by-step descriptive guide for bringing reconciliation practices into evaluation methodologies and approaches.
1. Identify the key evaluation contributors that can be involved in the design of the overall evaluation approach and methods.
There is no question that an Elder, knowledge keeper, traditional healer must have a highly significant role in the evaluation process. For example, any evaluation department that has any involvement with Indigenous Peoples must employ an Elder, knowledge keeper, traditional healer to inform not only their evaluation practice but also the interpretation of results and application of results into actions and follow-through activities. Many organizations and government departments conduct evaluations internally, as well as contract out evaluation projects. When these evaluations intercept with Indigenous Peoples the inclusion from a place of spiritual knowledge must be in effect.
There are many different levels of colonization among Indigenous Peoples. As well, many levels of cultural sensitivity and humility among Western evaluators. However, the goal of this paper is not to determine what level of decolonization is necessary, but rather to focus on what Indigenous evaluation can do and contribute to Western evaluation. In terms of what Indigenous evaluation can accomplish, my goal is to see the focus of evaluation transform until the primary outcome is at the national level of all Indigenous Peoples, united on several indicators of success. These would speak to what is valued most and provide a roadmap for reaching those goals. For practical terms, this work would be undertaken by the evaluation contributors, recognizing that the evaluation can occur in many ways. For example,
2. Learn about and document the overall and broader story of the program.
This is a critical step, particularly if there is a logic model in-place for the program. This is not a lip service activity. This is undertaken with a determination to discover the essences of the program and truly understand not just what it does but rather a larger concentration on how it does what it does. These questions ask about the management style, perceptions of staff under that management style, procedures for the program, and how the program operates. Thus, these process questions are the real story. Indigenous Peoples have called for the focus of evaluation to shift from outcomes to a focus more centered on process (NCCAH, n.d., Saini & Quinn, 2013, Van der Woerd, 2010; Chouinard & Cousins, 2007; Fetterman & Wandersman, 2004).
There is a sense among communities with significant challenges that outcomes are extremely difficult to measure, particularly the fact that it is unreasonable to expect decisive outcomes from a community that is classified as being a small population, such as 500 or fewer residents. In some of these communities the birth rate is anywhere from two babies to ten babies a year. In these cases, assessments cannot be made from numerical data. A case study approach will be necessary to differentiate between the many confounding variables. This has two major implications, 1) specific program sites should have the option to collect completely different data sets and 2) and the roll-up of information should be flexible in the manner by which it draws conclusions, where conclusions are not made on the number of xx and yy, but rather a more substantive conclusion is drawn that speaks to the varied and multiple ways individuals have experienced the program. Aiming to gather information that is experiential-based is far more valuable in determining future directions and continued funding for Indigenous Peoples.
Instead of focussing on the mechanisms (processes of programs) that give birth to creative and innovative thinking and that generate processes and policies that target these underlying issues, we have spent the past twenty years measuring outcomes in a meaningless and unrewarding manner. What have these outcomes done for Indigenous Peoples to improve their health and wellness outcomes? Many of the key damaging statistics have actually risen over the past twenty years, instead of declining. The greatest damaging legacy of the past two decades has been Indigenous Peoples and Western medicine working in silos – neither sincerely trusting the other. But evaluations over the past twenty years have not addressed these factors, instead they narrowly measure participant outcomes in a vacuum (this is a meaningless exercise). The rates of diabetes and cardio-vascular disease are not going to dramatically drop until Indigenous Health Practitioners and Western Medicine Practitioners come together in an effective and meaningful manner. Addressing the underlying mental wellness and inter-generational trauma issues are key factors in the solution. Bringing Western and Indigenous medicines together is a huge factor in initiating the needed and desired healing of a person, a family, a community and a nation.
It is these underlying and systemic issues that make the measurement of outcome data so difficult, particularly for promotion and prevention focused programs. These programs scratch at the surface, without actually addressing the systemic issues and problem solving at the heart of the issues affecting victims and their families. This significant constraint is particularly harsh on staff and must be included as a primary focus of program outcomes – not what the program has done for clients, but rather, what has the program and its constraints done to the staff and the vision and heart of the community? The question of absolute importance, then is what has the program done to support or hinder the People or the Nation, and in what ways do the staff and management implement the program that support these notions of community-building and transformation? These are the valuable questions that need to be asked and that would further support the development of community vision that can lead the People into a path of healing and strength-building. Evaluation then is a tool not only for documenting accountability to government, but it has a responsibility and the means to support the development of these visions and report on the underlying and systemic issues.
3. Identify an objective or goal against which the program is being measured
This is typically, in the era of the logic model, an outcome-related statement in the context of impacts on program participants in expectation that the intervention would yield some sort of change in socio-economic condition. However, as discussed above, while participant outcome goals and objectives are laudable, there is a time and place for such endeavours. The goals and objectives must suit the community, not the community suit the goals and objectives. Footnote 9
There is no place for participant-outcome objectives if the community involved in the program does not see their program as suitable for such measurements. And this does not mean they won’t, but the issue is, has anyone bothered to ask and ask in a way that is meaningful and makes sense to community?
But first, the ice must be broken, community members think evaluation is a black hole and it has all the answers, answers they cannot create themselves. This myth must be broken before a truly community-focused conversation can be had.
In a truly community-focused conversation, a variable goal and related objectives could be created, ones that accommodate the scale suggested and varies across communities, from zero participant outcomes to numerous participant outcomes. The priority for the objective and goal statement is that it reflects the vision for the community and the essence of the program. It is not up to one individual or a single grouping of individuals to determine how that objective should read. This is a complex task and is best sought from Indigenous Peoples participating as key evaluation contributors. They know programming and now they should also know evaluation very well.
The funder may have a notion as to the program goal and objective. However, when community is left to determine the program goal, it is far often rooted in a spiritual reality and has roots tied to the land and a significance that is grounded in the teachings and knowledge of spirit. To operate in a manner from a place of reconciliation, the goals and objectives must reflect both realities and these decisions must be made in a meaningful and truth-seeking way by community. These types of exercises can serve as a tool to bring about change and transformation.
4. Determine the indicators that will tell the story of the ways in which the program worked to address the objective(s).
The measurement of program objectives is often a complicated process regardless of evaluator. The identification of indicators differs significantly when one is coming from a place of reconciliation. Such indicators are more descriptive and have greater depth than the typical logic model indicator. Indigenous indicators must reflect the language of the People. In our experience we have witnessed community- identified goals and objectives defined in Indigenous languages. The goals and objectives therefore have elaborate meanings and are action-oriented. These goals and objectives can then be translated into indicators.
Ojibway, for example, has two-thirds verbs, whereas English has two-thirds nouns. Therefore, one rule is that the indicators must be action-oriented. As well, the Ojibway language is primarily learned orally in Anishnawbe communities, so the indicators should be easily understood from an oral perspective, in other words inspire a visual. And, the Ojibway words have elaborated and complex meanings, in other words the language is multi-dimensional. Further, the majority of Ojibway words support critical thinking, foster empathy and emotional intelligence because the language is composed of two-thirds verbs. The verbs reflect the actions of human beings and animals. All of these Indigenous language considerations are critical to identifying indicators that honour, respect, support, and aspire to bring reconciliation into the evaluation design. Once Indigenous indicators are known, new concepts can be developed for evaluation practice.
At the inaugural University of Toronto Indigenous Health Conference in 2016, Chief Wilton Littlechild spoke to an audience of 400 plus health professionals and stated that, “We need to hold each other up.” This is essential because it applies to the room of students and professionals in the health field, but it also applies to women, men, nurses, Indigenous Peoples, and the applications are almost endless. It supports critical thinking because it makes you think about the possibilities and, makes you visualize what it means to hold each other up. It can mean horizontally, vertically, and unilaterally. The statement supports empathy, in terms of caring for your fellow colleague or human beings, in general. It succeeds in spanning through time, in that it suggests we need to do something, now and in the future. The phrase is action-oriented because it refers to the act of supporting one another, and ensuring our livelihood is intact and we are fully cared for by one another. This type of objective restores honour, respect, support, and aspires to bring reconciliation into the evaluation design. The indicators that could be thought to fall from this objective can include the following set of triangulated evidence:
5. Develop tools/strategies that will serve as a record of the information that tells the story.
It is important that Indigenous tools act both as interventions and data collection devices. When the data collection tool acts as an intervention, it should support the documentation of experiences that matter in the minds of the program individuals and the whole – that they identified without the use of more than 1 – 3 very broad questions and without any probing. This step needs to be agreed to by Elders, knowledge keepers, and or traditional healers before proceeding. This process also requires an education process where such Elders etc. are exposed to a broad-based set of tools that are creative, and intervention focused. Blindly having people talk in a story-like fashion runs the risk of leaving the person hanging – rather 1 – 3 broad based questions enable a semi-structured process to engage the participant; however, these need to be carefully considered within a framework of experienced and informed individuals. In the remainder of this step an example of such an intervention-based tool is given.
Intervention-based creative tools such as the Waawiyeyaa Evaluation Tool Footnote 11 foster the development of local ideas and tools that allow respondents’ voices to be heard and inspire them to think creatively about the one or two questions at hand, instead of asking a list of 20 – 30 questions in a typical survey tool. It allows respondents to express their experiences in a manner that they wish. They are steering the data collection process.
What has been key is that people can hear their voice in the data design and the ultimate findings resonate with people’s beliefs about change. When engaged in the application of the Waawiyeyaa Evaluation Tool, respondents are drawn into a creative process of thinking about change, self-discovery of their own journey and the turns and changes they experienced.
By utilizing Indigenous-based tools, frameworks, and traditional storytelling and teachings, we have learned the following from program participants. While these can certainly be learned from other means, these were learned on a single page directed by each participant, by them answering one single broad question; whereas, in other means the following would take a detailed series of questions to obtain.
6. Document and design a system for recording, storing and analyzing/realizing the results of a program.
This step should be determined by the Elders, knowledge keepers, traditional healers and others participating as key evaluation contributors. According to statements made by the Truth and Reconciliation Commission of Canada (2015), a very important social responsibility of an evaluator is to shed their bias, engage in introspective thinking, and become an active participant in the healing and reconciliation process:
We should do no less. It is time to commit to a process of reconciliation. By establishing a new and respectful relationship, we restore what must be restored, repair what must be repaired, and return what must be returned.
Indigenous communities are very concerned with process and less interested in documenting outcomes. This reflects the focus of Indigenous systems. While individuals have different short-term to intermediate-goals and have different outcomes, an outcome conceived within an Indigenous system for a human is to nurture the spirit – that means living a life with respect for all life, and more, and to live through the lessons thrown at one’s self in order to support the spirit in its intended journey (which is largely unknown). Indigenous systems are more complex than this example alone. For example, another factor which makes outcomes less important, are the fact that Indigenous systems value opportunity seeking and engaging in new opportunities as they randomly and unpredictably arise. There is so much more to learn about Indigenous ways of being and ways of knowing. The epistemological perspective is entrenched in a dynamic complex pedagogical system. This is why it is so difficult for reconciliation practices to do justice to evaluation practices. Without a complete change in the lens and the manner by which the evaluator understands truth there is no place for the evaluator to undertake the design of a system for recording, storing and analyzing/realizing the results of a program.
One project allowed us to gather data that demonstrates this TRC recommendation in action – this information was only gathered and documented because the evaluator was experienced in the Indigenous pedagogy described next. In the north, a health authority has entrenched culture into the program and management framework. This northern organization has completely embraced the act of providing an “experience” not just for clients, but most importantly for staff as well. The organization provides its staff and management an inductive learning experience. A critical component of the approach is the culturally safe knowledge bundles each staff person is given. The physical component is the giving of a sacred bundle they carry with themselves on the job to utilize within their practice. For each of the sacred gifts they are given (such as medicines, drums), they received the teachings alongside their colleagues at the office.
The organization normalizes their traditional ways of being and ways of knowing. At this organization, the big drum greets you when you enter the building; it sits in the front in the waiting room, so clients can sit with the drum. This is the meditative process spoken by Dr. Yellow Bird -- to sit with the drum, to experience it, to feel it, to observe and to communicate with it, and bond with the drum. Yellow Bird’s work has examined brain scans that indicate prayer outcomes in the brain activity, show-up with increased creativity in the right side of the brain. Yellow Bird further explains that these ceremony participants experience significant development in the pre-frontal lobe of the brain, in addition to the occipital lobe and experience significant growth in an area of the brain that allows for connections beyond the self – such as increased empathy. Yellow Bird has validated prayer as a guaranteed outcome for increasing brain activity and brain development (Yellow Bird and Wilson, 2005)
Smudge and use of the pipe are “in the open” at this organization, and available for everyday use. The smudge and bowl are not decorations -- they are a living substance one greets when entering the room, and his or her experience is enhanced as they engage with the smudge: smelling, seeing, and opening spiritual-touch. It is not uncommon for someone to walk into the building and use the pipe, and all the staff have teachings on the use of the pipe. This is an instance of Dr. Yellow Bird’s engagement of the occipital lobe within one’s brain (Yellow Bird and Wilson, 2005).
7. Record the program’s information.
If the steps above are undertaken, the final information documented with the help of Elders, knowledge keepers, and/or traditional healers will reveal an understanding from a place of reconciliation. The remainder of this step provides a project example of the type of in-depth Indigenous pedagogy that is revealed. In the organization described in the previous section, the learning environment was described as having a deeply healing and culturally affirming impact on the employees and the management. A term such as ‘normalizing cultural practices’ was very profound in that it describes the organization as embedding cultural practice throughout the organization. In this case the cultural pedagogy directed the training approach and the whole work experience of the organization’s personnel. The managers were responsible and entrenched in the cultural ways and knowing whereby they carried their own cultural bundles and therefore could engage their staff in ongoing on-the-job inductive learning in the cultural pedagogy. The organization engages in Anishnawbe ceremonies and conducts its meeting from their entrenched cultural approach. These epistemological understandings are not easy to uncover and document, it takes a keen ear and insight from experience. So, either the external evaluator is skilled in this area and or the data collection tools support the respondent to illicit their own insights into the impacts of the program from their definitions and perspectives, entirely.
8. Share the program evaluation results broadly.
The sharing of the process information is vital to Indigenous Peoples who make requests at government organized meetings for increased time for networking to allow for the sharing of stories about each other’s programs. Networking is considered the most important function of such meetings. There is a lot to learn about community processes and Indigenous knowledge to inform evaluation practice. The concept of holism is certainly not isolated to Indigenous knowledge systems. However, despite a sense of universality across many nationalities confirming holism is a valid understanding, this seems to fail to inform evaluation indicators and outcomes. Western interventions are typically designed to treat the absence of something, therefore evaluation seeks to measure the increase in the presence of the absent variables. Rather, humans are dynamic and not unidimensional, thus when thinking about explaining the impacts of a program, yes, everyone wants to hear about the value of the program – what makes it relevant and significant. Using a wholistic format allows for program processes to be explained, but not just the physical calculations one can make about the impacts, but also the other aspects. Remembering humans are not unidimensional, factor in the emotional environment such as what it evokes and what it reminds staff and clients about; the mindful decision making around program design and day-to-day thinking that contributes to the program goals, and the spiritual connections the program creates for staff and clients. But, don’t try to discover this on your own, use tools that support this type of holistic thinking about evaluation measurement and reporting. Community does not want to hear about rigid and narrow reports, they want to hear about the good, the bad and the ugly to remind us we are human, but also reports that are fun and even humorous. The reporting needs to have rhythm and be full of life, to inspire goodness and feelings of goodness and feelings of joy and pride in their community and the people in their communities.
This elaborate definition of evaluation practice significantly expands the story that needs to be told. It goes beyond the individual, to include a complexity within one person between spirit, mind, emotion and body, and the inclusion of community impacts and influencers, and environment impacts and influencers.
In our work sharing the story supports ongoing evaluation. This step is social-justice in action. This looks at the planning and implementation of actions by evaluation collaborators from the evaluation findings. This section highlights the opportunity present for one to learn and further develop one’s evaluation practice of understanding the program from a place of reconciliation. This step calls for the evaluator to dialogue with the program, the funder, and Elders, knowledge keepers, traditional healers, and other key evaluation contributors, to determine if the vision and Spirt of a People were reflected in the final story.
We have worked on this leading innovative practice and seen it as elevating the function and role of dissemination into a validation procedure. Working with a community-based program a cultural celebration of the evaluation report included presentation, singing, raffle, dinner, and dancing. During the event about 2/3 through, children, handed-out surveys and collected 100 percent of the papers. The survey asked community members 6 questions that elevated their role and function to that of a decision maker and adjudicator. They were essentially asked if the results presented to them warranted the continuance of the program.
This path to reconciliation in evaluation is not for the lighthearted; it is a deep commitment to the course of decolonization in every means of the conduct and execution of evaluation practice. It is also understood that it is not a course of action that can be implemented instantaneously. Education is needed first and foremost, our next generation of evaluators should all be privy to such knowledge as contained in this article – such as this type of information being integrated into university courses by professors. We have several university professors that work with us keeping up to date on our innovative tool developments and educating their students on inspiring methods and approaches that honour reconciliation actions. At the very least, Elder and knowledge keeper involvement in the evaluation process should not be as a makeshift contributor but rather as a major decision maker as to its form and detailed conduct. The community voice is also an important, as to their voice being heard on what matters to them and the varied ways in which they have experienced the programs and services being evaluated. This involvement matters when executed at the start of the program and particularly in the design stage. While it is a common belief that program and evaluative practices are separate entities, they are in fact significantly intertwined. Take the case of the evaluation tool called the logic model, it is continually used to inform program design. Second, the evaluation practice should be used as a positive reinforcing tool of change. This can be in its overall conduct but also in its tool for data collection. The evaluation, therefore, is a means for informing and detailing program improvement and transformation. Finally, evaluation is an activity that communities look forward to and want to participate in, not because it rewards them with a form of payment for participation, but rather they want to participate because it leads to positive and ongoing growth and change in their community at-large.
The Indigenous Elders that discussed this term with the author insist that it must remain with the ‘w’ included as not to lose its connection to wholeness and the general sense of being whole and circular in nature, in that it resembles a circular path that can be repetitive and long-lasting.
Theory of change refers to how and why a desired change is expected in a particular context.
Adapted from a conversation with Standup, Geraldine (Standup, 2002), Elder in Residence, Aboriginal Health Toronto, Ontario, Canada.
Tool Bundle refers to not only physical items such as a drum, but also spiritual connections such as spirit helpers, emotional referring to skillsets that focus on maintaining a balanced composure that among other things reflects wisdom, and mental skillsets that support a balanced way of being and way of doing.