Programme Evaluation – Quality Education and Livelihoods Support for Syrians under Temporary Protection in Turkey At Concern Worldwide

Concern Worldwide with funding from EUTF is seeking bids from bidders for providing Final evaluation service for project in Turkey.

Bidders who are interested can request the bid documents by sending their request to turkey.procurement@concern.net.

TOR

1. BACKGROUND

Concern Worldwide is an international, non-governmental, humanitarian organization dedicated to the reduction of suffering and working towards the ultimate elimination of extreme poverty in the world’s poorest countries. Concern began working in Turkey in 2013 due to the high volume of refugees arising from the Syrian crisis. It is estimated that Turkey is hosting more than 3.6 million Syrian refugees.

Concern Turkey has been implementing a EUFT MADAD funded Education and Livelihood programme in four provinces of South-east Turkey; Sanliurfa, Hatay, Kilis and Gaziantep. Concern received approval from the Ministry of National Education for the programme implementation and worked closely with Provincial Departments of Education in the four provinces. The programme targeted primary school children (aged 06 to 13), high school children (aged 14 to 18) and adults from Syrian and host communities. The programme started on 16 December 2017 and is expected to complete by 30 September 2021. Concern Turkey is looking for a suitable team of consultants or firms to undertake an external evaluation of the programme.

The programme has two modifications accompanied by no-cost extensions (NCE). The first modification request (MR1) was mainly based on shift in implementation areas from originally designated 7 provinces (Adana, Ankara, Izmir, Konya, Kahramanmaraş, Mersin and Sanlıurfa) to existing four provinces (Gaziantep, Hatay, Kilis and Sanlıurfa) due to operational permission challenges and, the original end date 31 August 2019 was extended to 30 September 2020. The MR2 was required to compensate for the loss of time in fulfilling the implementation requirements of the action due to consecutive changes within MoNE staff and approvals. Therefore, the EUTF granted a third NCE until 30 September 2021, the existing end date. The BT programme experienced challenges in conducting activities since the Covid-19 related restrictions started in March. Training and psychosocial support (PSS) provided to Outcome 1 beneficiaries through Public Education Centres (PECs) moved to emergency remote training and remote support modality. Outcome 2 beneficiaries, students of vocational and technical high schools, were provided monthly stipends without looking for classroom attendance criterion. Training for teachers’ professional development was conducted remotely. The programme partially started in-class training for Outcome 1 and 3 in late June, July, August, September, October and early November 2020 following the Government of Turkey’s normalisation plan. The pandemic outbreak and related restrictions caused delays in some activities. Social cohesion activities have not been possible to implement.

2. PURPOSE OF THE EVALUATION

The final evaluation will contribute to lessons learned and knowledge on resilience, education and livelihoods in the protracted crisis, response in urban settings and the findings will be disseminated to the EU and relevant stakeholders.

The evaluation aims to assess the programme for three main purposes:

· Accountability: Evaluate if the programme has been implemented in an accountable manner to all stakeholders including communities, beneficiaries, MoNE and EU and recommend learnings to apply in new programme development

· Learning: Document learnings from the implementation of the programme in partnership with MoNE and recommend how these learnings could be applied to inform new programme development

· Increasing Impact: Identify the impacts that the programme brought to the lives of three main target groups (Children 6 – 13years, adolescents 14 – 18 years, and adults) and the extent these changes are sustainable and recommend learnings to apply in future programme development.

3. DESCRIPTION OF THE SUBJECT OF THE EVALUATION

‘Quality Education and Livelihoods Support for Syrians under Temporary Protection in Turkey’ or shortly called Building Tomorrow Programme, funded by EUTF, aims to improve resilience and strengthened social cohesion amongst targeted vulnerable Syrian and Turkish communities through improved access to quality education and livelihood opportunities.

In response to the identified problems of Syrian refugees, this intervention was designed to facilitate the integration and enrolment of Syrian students into Turkish public schools through education, outreach and learning support in Public Education Centers (PECs). In particular, assisting students who have been out of school for an extended period and need Turkish language support, as well as those who are enrolled in the Turkish formal school system, but require help for catching up with their peers. The action enhances the capacity of 30 Public Education Centers (PECs) and 20 vocational high schools managed by the Ministry of National Education (MoNE) to cater for Syrian and Turkish students. These government centers and schools supported programmes to integrate refugees into primary education and vocational training programmes. Working in and with the PECs, this programme supports 12,000 school-aged children (6 – 13 years) by bridging them to the formal education system or providing those already in school with homework and Turkish language support.

Additionally, 1,800 out of school or school youth (1,260 Syrian and 540 Turkish 14 – 18 years old) will be supported to enroll and attend vocational high schools for two semesters. Moreover, adults will be supported to access vocational training and Turkish Language classes to increase their employability and income opportunities.

The overall objective of this programme is improved resilience and strengthened social cohesion amongst targeted vulnerable Syrian and Turkish communities through improved access to quality education and livelihood opportunities.

To achieve the proposed impact, the programme has three key outcomes:

Outcome 1: Improve access to formal educational opportunities for school-aged Syrians under temporary protection through outreach, learning support programmes and provision of learning and psychosocial support materials

Outcome 2: Improve the capacity of vulnerable Syrian and Turkish youth to access vocational high schools to prepare them to enter the labour market

Outcome 3: Improve the capacity of vulnerable Syrian and Turkish adults through access vocational training courses

The outcomes and outputs of the programme are measured through several indicators presented in Annex-1 (Programme Logical Framework).

4. EVALUATION OBJECTIVES AND KEY QUESTIONS

The main objective of the evaluation is to evaluate the overall performance of the Building Tomorrow Programme and the degree to which the anticipated improvements in people’s lives have been achieved. Final questions will be agreed upon during the Inception.

The specific objectives of the evaluation are set according to the Development Assistance Committee (DAC) criteria and should answer relevant questions listed below against the DAC criteria:

Relevance

· To what extent, did the contextual analysis carried out to inform programme design was based on Concerns Understanding of Extreme Poverty or Turkey Programme Country Strategic Plan, and priorities of the EUTF?

· How appropriate were the chosen interventions and programme design to the situation and needs of different stakeholders at different levels (micro, meso, and macro, and considering the needs of girls, boys, men, women and others identified as vulnerable to situation in the programme area)?

· How the programme adapted changes during COVID-19 and addressed the needs of target groups effectively?

Coherence

· To what extent was the programme outcomes relevant, appropriate and strategic to national goals regarding the EU – Turkey joint priorities addressing the needs of the SuTPs and other persons in need of international protection, and Concern policies and guidelines?

· To what extent was, the invention carried out in a way that external coherence considers the consistency of the intervention with other actors’ interventions in the same context. This includes complementarity; add value while avoiding duplications, harmonisation and coordination with others.

Efficiency

· How efficient were resources (both human and financial) used? Were there things that could have been done differently and if so, how?

· How efficient was the programme M&E system in considering adaptation to the external changes?

Effectiveness

· What steps were taken to address issues of inequality to ensure the interests of the most marginalised people were taken on board during programme planning, implementation and monitoring? How effective was this?

· To what degree did the programme experience unintended and unexpected consequences, and if so have these been addressed in the programme and how?

Impact

· What indications are there of significant changes taking place beyond the programme – both positive and negative?

· How have the programme interventions impacted differently on men and women, boys and girls, people with disabilities (and other vulnerable groups as identified) in the programme area?

Sustainability

· Are the results sustainable? Will the outputs and outcomes lead to benefits beyond the life of the existing programme?

· Which findings may have relevance for future programming or other similar initiatives elsewhere? How might the programme do things better in the future?

5. METHODOLOGY

Concern will contract external evaluators according to Concern’s Evaluation Guideline to undertake the final evaluation independently to meet Concern’s and the donor’s requirements.

An external evaluation team will independently assess the programme’s relevance, effectiveness, efficiency, coherence, impact, and sustainability in line with DAC evaluation criteria. This final evaluation will be participatory and seek input from all parties involved including beneficiaries, government partners, contracting authority and other relevant stakeholders.

The methodology will be a combination of primary and secondary data collection for analysis and review. For primary, qualitative data collection, tools like Focus Group Discussions (FGDs), Key Informant Interviews (KII) or interviews with beneficiaries, interviews/visits of partners and government stakeholders and relevant programme staff will be used. Concern will also provide access to the data collected during the programme implementation, which may be validated and used as a resource for the final evaluation. Secondary data will be collected through a process of desk review of programme documents, reports, and monitoring data will be included.

The Programme Director (PD) and the Area Managers will brief the evaluators on the programme, and introduce the focal points for the external evaluation who will be nominated by the PD/CD. Concern’s focal points will provide all required documents, information and local contextual knowledge throughout the evaluation. The evaluators and the focal persons will agree on the design and process of the evaluation in detail, finalising dates, methods, tools, and sampling for the evaluation. Cooperation between the Concern focal persons and the evaluators is expected to increase the quality and relevance of the evaluation. However, the Concern’s focal person will not influence the independence of evaluation by their level of involvement.

1. RESPONSIBILITIES AND KEY TASKS OF EVALUATORS

· Review programme documents (proposal, reports, plans, agreements etc.)

· Prepare an evaluation plan, tools and methodologies and share with Concern for comments and inputs

· Collect all qualitative and quantitative data from all relevant stakeholders (government functionaries, EU, beneficiaries, Concern staff) in all programme locations

· Conduct workshop/meetings to discuss and agree upon findings and recommendations with the Concern Programme team and CMT

· Provide an in-country debriefing to Concern country management and programme teams

· Prepare a comprehensive evaluation report in line with the above-mentioned objectives of the evaluation and recommendations for future programme design and implementation.

· Adhere to Concern’s Code of Conduct and Associated policies including compliance with Concern policies on security and any other relevant policies

7. EXPECTED PRODUCTS/OUTPUTS

Evaluator/s are expected to provide the below outputs:

· Inception report (maximum 6 pages) consisting of the methodology, proposed data collection tools and consolidated Work Plan for the evaluation illustrating timelines, roles and responsibilities. This is to be submitted within 2 weeks of award of contract.

· Presentation of initial findings to the Concern Country Management team, Area Managers, MEAL Coordinator for input, feedback and recommendations.

· A draft report to be submitted to Concern for review and feedback two weeks after the fieldwork has been completed

· A final report (outline below) incorporating comments on the draft report and presenting final key findings and recommendations as explained above, is expected one week after receiving feedback on the first draft.

· The evaluators will score the programme against each of the DAC criteria according to the below table:

4

Outstanding performance

3

Performance in line with what would be expected of a well-functioning organisation

2

Generally acceptable performance but with some clear, and documented, shortcomings

1

Barely acceptable performance with some major shortcomings and reservations

Totally unacceptable performance or insufficient data to make an assessment

8. REPORTING LINES AND STRUCTURE

The evaluation team will report to PD/CD in the country and will coordinate their movements and arrangements with assigned focal points by Concern. The evaluation report should be maximum 25 pages (minus Annexes) and structured as follows:

· Executive Summary (Max 1 pages)

· Introduction and Programme Overview (Max 2 pages)

· Methodology and Limitations (Max 2.5 pages)

· Findings and Discussion on specific questions highlighted in the TOR under the relevant DAC criteria (Relevance; Coherence; Effectiveness; Efficiency; Impact; Sustainability) including a score under each criterion (Max 15 pages)

· Conclusions and Lessons Learnt (1.5 pages)

· Recommendations and Management Responses (Max 2 pages)

· Report Annexes

· Evaluation schedule including lists of site visits/discussions (Max 2 pages)

· Final terms of reference (attached as annex)

9. COMPOSITION, SKILLS, AND EXPERIENCE OF THE REVIEW/EVALUATION TEAM

The evaluation team may consist of up to three consultants (to be proposed by bidders), led by a team leader, to undertake the evaluation process and they should be selected as per the below essential criteria:

• More than 5 years’ experience in undertaking independent evaluations of EU funded programmes and/or using DAC criteria, preferably in education and livelihood sectors
• Have technical skills appropriate to the particular piece of work
• More than 10 years’ experience in participatory approaches in programme implementation, evaluation, planning.
• Strong understanding of the Turkey context, GDPR and Turkish data protection regulations
• Understands key issues of economy, efficiency, effectiveness, impact, design, relevance
• Have strong analytical, facilitation, listening and writing skills in English
• Proven ability to produce a high-quality written report.
• PLAN FOR EVALUATION IMPLEMENTATION (INCLUDING TIMELINES)

The evaluation is expected to be completed and final evaluation report submitted by the end of November ’21 (two months after the completion of the programme).

1. DOCUMENTS REQUIRED TO ASSESS CONSULTANTS EXPRESSION OF INTEREST

Applicants should submit the following documents:

· Proposed evaluation methodologies and work plan

· Profile/CVs of the evaluation team

· Samples of previous work and list of evaluations done

· Cost of evaluation proposal (all inclusive)

· Confirm ability to provide formal invoice (providing registration and other related documents)