2. Evaluation purpose, scope, and objectives

Since the Project is at its final stage of implementation, the FINAL EVALUATION (FE) exercise aims to inform UNDP Afghanistan, the donor (JICA), and partners of the lessons learned, outcomes achieved, value for money ensured, and recommendations for similar programing in future. The FINAL EVALUATION will draw out progress and achievements toward project deliverables, identify gaps in programming, and identify efficient use of resources, impacts, and key takeaways for future programming. Notably, the findings and analysis of the final evaluation will inform the designing of UN/UNDP’s future interventions related to improving livelihoods, promoting the OVOP approach, and cross-border trade. The FINAL EVALUATION will produce valuable lessons and experiences, providing useful findings for other relevant projects and initiatives organized by UNDP Afghanistan, Tajikistan, and other country offices. Responding to the Theory of Change (ToC) as described in the project document, the agreed Results Framework (RF), and the approved work plans, the FINAL EVALUATION is expected to assess the relevance of the project, quality of the project design, effectiveness, and efficiency of the implementation to date, sustainability of the overall project results, impact of the intervention, and forward-looking directions for future. 

Project Outputs vis-à-vis outcomes to be evaluated include: 

•                   Output 1: Employment opportunities for communities living in border provinces of Afghanistan and Tajikistan increased

•                   Output 2. LITACA products’ access to the market enhanced

•                   Output 3: Access for cross-border trade improved

For more details on the Project, please see the Project Document (LITACA) 

The FINAL EVALUATION will serve to:

•                   Ascertain the relevance, effectiveness, efficiency, impact, and sustainability of the project results.

•                   Assess project performance and progress against the expected outputs targets, and indicators presented in the results framework, as well as their contribution to the expected outcome of the UNDP Transitional Country Programme Results Framework (TCPRF), Transitional Country Programme Strategy (TCPS) and the UN in Afghanistan and Tajikistan Sustainable Development Cooperation Framework 2023‐2026.

•                   Assess the Quality of implementation, including financial management and project efficiency.

•                   Review and document the success, lessons learned, good practices, and transferable examples. Identify challenges and the effectiveness of the strategic approaches that the project adopted to address those challenges.

•                   Outline recommendations, including potential realignments in scope and approach in line with the project’s outcome, and provide forward-looking recommendations that enhance the quality of future programming.

      

The Final evaluation will geographically focus on the provinces and districts where the LITACA project is implemented.

The evaluation should cover the following cross-cutting themes: (a) Rights-Based Approach (RBA), (b) disability inclusion, (c) gender mainstreaming, (d) environmental sensitivity and sustainability, and (e) accountability to affected populations (AAP). All the abovementioned themes are equally important to the UNDP and should be considered in this evaluation. The data collection process should address these cross-cutting themes equally and effectively. 

Furthermore, the “Leave No One Behind” (LNOB) principle is a cornerstone of the OECD’s inclusive development and evaluation approach. The Evaluation team must consider integrating LNOB into the evaluation methodology and approach to ensure that the evaluation considers the most marginalized and disadvantaged groups. This aligns with the 2030 Agenda for Sustainable Development’s commitment to prioritize the needs of the furthest behind first. By embedding LNOB in evaluation practices, UNDP advocates for a systematic focus on equity and non-discrimination, encouraging the evaluation team to identify and address disparities in project outputs and ensure that development interventions contribute to equality and empowerment for all. The evaluation team must note that this principle is vital for capturing the true breadth of UNDP intervention’s impact and for steering policies towards more inclusive and sustainable outcomes.

3.      Evaluation criteria and key guiding questions

The evaluation should adhere to the guidelines established by the Organization for Economic Co-operation and Development – Development Assistance Committee (OECD DAC), which is structured around six essential evaluation criteria:

·       Relevance: This criterion assesses whether the intervention aligns with the right priorities, policies, and needs of the target group, recipients, and donors. It considers how well the intervention responds to changing circumstances.

·       Coherence: Evaluate how well the intervention fits within the broader context. This includes its alignment with other initiatives and its consistency with global, national, and local policies.

·       Effectiveness: Examine whether the intervention achieves its stated objectives. This assessment involves measuring the actual impact against the intended outcomes.

·       Efficiency: Analyse how efficiently resources are utilized. This criterion considers cost-effectiveness and resource optimization.

·       Impact: Explore the tangible difference the intervention makes. This involves assessing the real-world changes resulting from the intervention.

·       Sustainability: Evaluate whether the benefits of the intervention are likely to endure over time.

UNDP Guidelines for Quality Assessment

It is essential to highlight that the mainstream definitions of the OECD-DAC criteria are neutral regarding human rights and gender dimensions. However, it is crucial to incorporate these dimensions into the chosen evaluation criteria. Specifically, please see page 77, table 10 of the document titled 'Integrating Human Rights and Gender Equality in Evaluations’.

The below table details the evaluation criteria.^[1] and an inexhaustive list of questions that should serve as a guide. These should be adapted/supplemented by another set of detailed and specific sub questions by the Evaluation Consultant(s) in consultation with UNDP before commissioning the evaluation.

 

 

Table 2 - Criteria and Guiding Questions

Criteria	Guiding Questions
Coherence	Assess the coherence of the project’s strategies, design, and implementation arrangements to the needs of Afghanistan and Tajikistan.   1-      Do synergies exist with other interventions carried out by UNDP, intervention partners, and stakeholders including the donor? 2-      To what extent is the LITACA III project consistent with international norms and standards to be applied to the existing context? 3-      To what extent is the LITACA III project complementary to other actors’ interventions in the same context? To what extent does the LITACA III project add value and avoid duplication in the given context?
Relevance	Relevance of the project: review the progress against project outputs and contribution to outcome level results as defined in the project’s ToC whether

 

 

	assumptions and risks remain valid. Identify any other intended or unintended, positive or negative, results using the following guiding questions.   1.       To what extent was the project in line with the UN Transitional Engagement Framework (TEF, the UNDP Transitional Country Programme Strategy (TCPRF), and the UNDP Strategic Plan and to the needs of the local population?  2.       To what extent has the project adapted to the changing external conditions following the changes that happened at the national level after the Taliban takeover in August 2021 and the recent ban on women's education and work by the Taliban interim government? 3.       To what extent were the inputs and strategies identified realistic, appropriate, and adequate to achieve the results? 4.       To what extent did the project achieve its overall outputs? Are the project’s contributions to outcomes clear, especially to gender equality and women’s empowerment?
Effectiveness	Effectiveness of implementation approaches: Review the project’s technical and operational approaches, the regionality and deliverables, the quality of results and their impact, alignment with national priorities and responding to the needs of the local populations; cover the results achieved, the partnerships established, and issues of capacity using the following guiding questions.   1.       How effective were the strategies used to implement the project and complete the activities effectively regarding quality, quantity, and timing? To what extent did the project outputs contribute to the achievement of outcomes, and what factors were contributing to it? 2.       Did the capacity support services delivered by the LITACA III project address the needs of beneficiaries and local communities, and were the target beneficiaries reached as expected? (Are there significant differences between male and female beneficiaries?) 3.       What are the key internal and external factors (success & failure factors) that have contributed, affected, or impeded the achievements, and how have UNDP and its partners managed these factors?  4.       To what extent have UNDP partners and local stakeholders been involved in project implementation following a participatory approach? To what extent are stakeholders and beneficiaries satisfied with the benefits they received?
Efficiency	Efficiency of the project management structure and added value of the project’s approach: review planning, management, social and environmental safeguarding, grievance redressing, monitoring, and quality assurance mechanisms for the delivery of the project interventions and the added value of the project setup.  How cost-effective was the project? Were the financial resources used appropriately to achieve the intended results? 1.       What systems and tools were developed and implemented for social & environmental safeguarding? What challenges were experienced during the implementation of the tools? How has it contributed to the quality of the project implementation? 2.       Did the project activities overlap, and duplicate similar interventions funded nationally and/or by other donors? 3.       To what extent did the project produce synergies within UNDP and with other development partners and play complementary roles with each other?
Sustainability	Sustainability of the project results and risks along with opportunities related to future interventions: review and assess if the current project setup has plans for future resource mobilization, synergy, long-term partnership, and / or considering institutionalization of the project impact for continued support after the project ends using the following questions.   1.      Did the LITACA III project activities take specific measures to guarantee sustainability? Is LITACA III project activities supported by the local stakeholders and communities and well-integrated into local social and economic structures? 2.      What exit strategies are in place, and how effective are they in ensuring sustainability beyond the project life cycle? Are structures, resources, and processes in place to ensure the benefits generated by the LITACA III project are continued after the completion of the Project? 3.      How do beneficiaries at the individual and institutional levels perceive the sustainability of the LITACA III project for capacity-building support? Do they plan to continue making use of the services/products produced? 4.      Describe key factors that will require attention to improve the prospects of sustainability of Project outcomes and the potential for replication of the approach. What social or political risks may jeopardize the sustainability of project outputs and the project’s contributions to UNDP Country Programme outputs and outcomes?
Impact	1.      To what extent is the project likely to contribute to the improved capacity of communities and OVOP groups? 2.      What positive and/or negative changes are the beneficiaries experiencing due to their participation in LITACA III project activities? 3.      To what extent did the local communities benefit from the quick-impact projects implemented?
Human Rights, Gender Equality, and Leaving No One Behind	1.      How adequately were cross-cutting themes such as human rights, gender equality, disability, age and social & environmental standards considered in the LITACA III project? 2.      To what extent has the project promoted positive changes in gender equality and women's empowerment? Were there any unintended effects? How can the project further broaden its contribution to enhancing diversity and inclusion in a future phase? 3.      To what extent have local communities, women, youth, people with disabilities and other disadvantaged groups benefited from the project either direct or indirectly?

 

4.      Methodology

 

The evaluation will apply the standard OECD/DAC criteriausing a mixed-methods approach, i.e., qualitative and quantitative data collection. The evaluation Consultant(s) should further develop this and approve it by the UNDP Evaluation Manager during the inception phase of the evaluation. 

This document envisages a general approach for conducting the evaluation, as well as data sources and tools that will likely yield the most reliable and valid answers to the evaluation questions within the limits of resources. However, final decisions about the specific design and methods for the evaluation should emerge from consultations among the UNDP Evaluation Manager, the LITACA III Project Team and the Evaluation Consultants about what is appropriate and feasible to meet the evaluation purpose and objectives and answer the evaluation questions, considering the limitations of budget, time, and data. 

The evaluation will employ qualitative and quantitative methods to assess the project’s impact. Evaluation Consultants are expected to adopt a participatory and consultative approach, engaging closely with the evaluation manager, responsible parties, and male and female direct beneficiaries. Suggested methodological tools and approaches may include but are not limited to the following:

o Desk review. At the beginning of the assignment, the Evaluation Consultants will need to review key documents, such as, but no limited to:

·       Project Document and all annexes – Please see page 19-21 for Results Framework

·       Quarterly/Annual Progress Reports,

·        Assurance/Field Monitoring Reports,

·       Previous Evaluation Reports

·       Knowledge Products

·       Relevant Policies of UNDP and the de facto Authorities.

·       UNDP TCPRF and TCPS Documents

·       UNSFA Joint wok plan

·       UNDP Strategic Plan

Note: The complete list of documents alongside attachments will be provided as soon as the Consultants are onboarded.  

•        Development of evaluation questions and surveys around relevance, effectiveness, efficiency, and sustainability and designed for different project beneficiaries, local stakeholders, and service providers to be interviewed.Surveys to capture diverse perspectives and stories should be tailored based on the specific interventions and beneficiaries. 

•        Data collection, done in the form of the following: 

o   Semi-structured interviews with key representatives of the partners and their employees, community representatives, local stakeholders, and service provider

o   Key informant and focus group discussions with the project beneficiaries, including men and women. 

o   Interviews with Project Board members and strategic partners

o   Field visits and on-site validation of key tangible outputs and deliverables.

o   Other methods include outcome mapping, observational visits, group discussions, etc.

•        Data review, validation and analysis:

o   To ensure maximum validity and reliability of data quality, the evaluation team should:

▪  Triangulate information from various data sources to strengthen the validity of findings and conclusions.

▪  Apply a gender and human rights lens to all evaluation products.

▪  Address gender, disability, and human rights issues in the LITACA III Project Final Evaluation Report.

•        Key Informant and Focus Group Discussions:

o   Conduct anonymous discussions with beneficiaries (men & women) and stakeholders. While selecting the respondents, the evaluator should ensure gender balance and ascertain voices of the most vulnerable are included in this assessment.

o   Conduct meetings with respective UNDP staff including the evaluation reference group.

o   Ensure that specific comments are not attributed to individuals in the final evaluation report.

•        Surveys and Questionnaires:

o   Administer surveys to male and female participants, capturing diverse perspectives.

o   The Questionnaire should be tailored based on the specific interventions and beneficiaries.

•        Field Visits and On-Site Validation:

o   Conduct field visits to verify key tangible outputs and interventions. These visits will be carried out in accordance with safety and security protocols and guidelines. Present a clear methodology for data verification and triangulation.

All conclusions, judgments, and opinions must be qualified by evidence and not be based on opinions.

Post-data collection debriefing: After the completion of data collection, the Evaluation Consultants will debrief project key stakeholders on the preliminary findings. The meeting will also serve as an opportunity to identify areas requiring further analysis and any missing information and evidence before the consultant enters a full synthesis and drafting phase.

The Evaluation Consultant is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, service providers, and direct beneficiaries. Gender-sensitive approaches to collecting and analysing data should be ensured. All interviews should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals.

Following a participatory approach, evaluation consultants must approach the following stakeholders and key partners of this evaluation for the survey, interview, and feedback where needed. These partners include but are not limited to UNDP senior management, UNDP Development Effectiveness Unit, UNDP Program Quality Assurance, and the JICA as the project's donor who will be engaged in the design of the TOR for evaluation, evaluation oversight, and quality assurance.

As part of the requirement, the evaluation must include an assessment of the extent to which the project's design, implementation, and results have incorporated a gender equality perspective and rights-based approach. The Evaluation Consultants will make sure to Integrating Human Rights and Gender Equality into the Evaluation during the inception phase. In addition, the methodology used in the evaluation, including data collection and analysis methods, should be human rights and gender-sensitive to the greatest extent possible, with evaluation data and findings disaggregated by sex, ethnicity, age, etc. – with a focus on people with disabilities. Detailed analysis of disaggregated data will be undertaken as part of the final evaluation, from which findings are consolidated to make recommendations and identify lessons learned for the enhanced gender-responsive and rights-based approach to the project. This evaluation approach and methodology should consider diverse groups in the project intervention – women, youth, vulnerable groups, etc.

The final methodological approach, including the interview schedule, field visits, and data to be used in the evaluation, should be clearly outlined in the inception report and fully discussed and agreed upon between UNDP and the Evaluation Consultants.

 

5.      Evaluation Products (deliverables)

•        Evaluation inception report (7-10 pages). The inception report should be carried out after the desk review and based on preliminary discussions with UNDP. It should be produced before the evaluation starts (before formal evaluation interviews, survey distribution, or field visits) and before the country visit for international Evaluation Consultants. 

•        The work plan and methodology. should be included in the Inception Report, and a clear timeline of how and when each evaluation step will be undertaken should be provided. Considering the travel restrictions due to safety and security in Afghanistan. The work plan can be annexed to the inception report or shared with the evaluation reference group/ evaluation manager in advance.

•        The updated Evaluation matrix. should be included in the inception report. The evaluation matrix is a tool the evaluator creates as a map and reference in planning and conducting an evaluation. It also serves as a useful tool for summarizing and visually presenting the evaluation design and methodology for stakeholder discussions. It details evaluation questions that the evaluation will answer, data sources, data collection and analysis tools or methods appropriate for each data source, and the standard or measure by which each question will be evaluated. Below is a sample of the evaluation matrix template.

 

 

•        Evaluation debriefings. Immediately following completion of fieldwork and data collection. the Evaluation Consultants are expected to provide a preliminary debriefing of findings to UNDP.  Draft evaluation report (40-50 pages excluding annexes). The content of the reports should be provided as per the provided template.  United Nations Development Programme - Evaluation Guidelines (undp.org) SECTION 4 EVALUATION IMPLEMENTATION AND USE – Page 56

•

UNDP will coordinate with key stakeholders to review the draft evaluation report and provide comments to the evaluator within an agreed period (within two weeks after receiving the document), addressing the content required (as agreed in the TOR and inception report) and quality criteria as outlined in these guidelines.

 

•        Evaluation report audit trail. This document serves as a repository for feedback provided by the evaluator in response to the comments and feedback by the Evaluation Reference Group on the draft Evaluation Report. It will be retained by the evaluator to demonstrate how these comments have been addressed. The document can take the form of a simple matrix, including the following details:

o   Chapter and Section Number

o   Paragraph Number/Line Number

o   Comments provided by

o   Date

o   Evaluation Team Responses and/or Actions Taken

o   Date of Response

 

•        Final evaluation report, reflecting the achievements and success stories of the project. 

•        Presentations to UNDP (Evaluation Reference Group) and JICA.

 

Expected Deliverables and Timelines:

#	Deliverables	Description	Due date
1	Final evaluation inception report	•        The inception report should include a proposed schedule of tasks, activities, and deliverables, building on what has been provisionally proposed in this ToR.  •        It should be prepared by the Evaluation Consultant/s before going into the full-fledged exercise.  •        It must also outline reviewers’ understanding of what is being reviewed and why, showing how each area of inquiry will be answered by way of: proposed methods; proposed sources of data; and data collection procedures.  •        The inception report should provide UNDP and the Evaluation Consultant/s with an opportunity to verify that they share the same understanding about the assignment, the same understanding of the ToC and clarify any misunderstandings at the outset.	15 days after the contract signed
2	Final evaluation briefing	After completion of data collection or before sharing the draft report, the Evaluation Consultant should present preliminary debriefing and findings to UNDP and evaluation reference group.	One months after the contract signed
3	Draft Final evaluation report	•        The Final evaluation report will be reviewed by the UNDP Evaluation Reference Group including respective CO teams to ensure that it meets the required quality standards and covers all agreed components and contents. Detailed comments and feedback on the draft report will be provided to the Evaluation Consultant/s, and discussions may be held to provide clarifications, as necessary.  •        Evaluation Consultant should submit a comprehensive draft report consisting of major findings and recommendations for future course of action.	Two months after the contract signed
4	Final draft of the Final evaluation report with achievements and success stories of the LITACA III project	•        The final report will be produced by the Evaluation Consultants based on feedback received on the draft report. The Evaluation Consultants should include two rounds of feedback from UNDP. The final report will be shared with UNDP LITACA III project management team.  •        The final draft report should be submitted within the given timeline with enough detail and quality.	3 months after the contract signed
5	Audit Trail Form	The comments and changes by the Evaluation Consultant/s in response to the draft report should be retained by the Evaluation Consultant in form of audit trial to show they have addressed comments.  This document can be submitted as an Annex to the final evaluation report.    The key partners engaged in the evaluation and audit trial would be the followings:  1.       UNDP senior management. 2.       UNDP relevant departments such as Program Quality Assurance and the Development Effectiveness Unit. 3.       The World Bank as the LITACA III project donor.	25 days after the contract signed

 

6.      Evaluation team composition and required competencies

 

For national consultant: 

 I. Academic Qualifications: 

• A minimum of a bachelor’s degree or equivalent in public administration, development studies, economics, or other relevant social science.

 

II.      Years of experience:

•        At least 5 years of professional experience in the provision of policy, analytical, and technical advisory support for international development organization. 

•        At least 3 years of proven experience in development, risk assessment, and/or evaluation of programmes or projects in area-based development, capacity building of private sector, government entities and civil society organizations. 

•        Experience in the result-based management, evaluation methodologies and programme/project monitoring approaches with development partners 

•        Evaluation experience with UN and/or UNDP is preferred. 

•        Sound understanding of the UN system and of UNDP’s mandate and role.

III.    Language:

•        Excellent English knowledge, both oral and written. 

 

IV.   Competencies:

Functional Competencies:

•        Ability to perform tasks in a timely manner and produce quality final product.

•        Strong interpersonal, communication. 

•        Openness to change and ability to receive and integrate feedback.

 

Responsibilities of the FINAL EVALUATION consultant (National):   

•        Review the relevant project documents including the Project proposal, M&E and GRM manuals, TPMA guideline, quarterly reports and other relevant documents deem necessary to be reviewed to help set the scene about the project nature and activities. 

•        Maintain a sound working relationship with the international consultant and team in the country office as well as in the provinces, seek support where needed and coordinate regularly. 

•        Undertake data collection through the team and directly in relation to the evaluation criteria and questions, ensure the validity and accuracy of the data and compare with the historic information collected by the LITACA III project M&E teams. 

•        Undertake field visits to the project locations, interview with the beneficiaries and stakeholders and collect evidence of the project progress and outcomes to feed the final evaluation. 

•        Implement the ethical considerations throughout the process and ensure to comply with the UNEG’s evaluation standards. 

•        Assist with the international consultant to develop and present a draft evaluation report and findings to the UNDP management and LITACA III project team, take into account all the relevant discussions and comments, and the track record of the changes being proposed/made on the next versions of the evaluation report.

•        Support the international consultant on the submission of the final evaluation report with due consideration of quality and effectiveness in line with the timeline that is initially agreed and approved by the UNDP.

 

7.      Evaluation ethics

 

This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’ The consultant must safeguard the rights and confidentiality of information providers, interviewees, and patterners through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The info

Submission Guideline :

Qualified candidates should submit a Cover Letter and Resume through Jobs.af.

Submission Email :