Evaluating public policies requires a great deal of technical knowledge. Yet it is not only a matter for experts. While the work on the data, and the studies themselves are entrusted to specialists, the formulation of questions of public interest is above all a major democratic issue, and must be regarded as such. This ideal is stated in the Declaration of the Rights of Man and of the Citizen of 1789 in its article 15: "Society has the right to require of every public agent an account of its administration".
If citizens are to develop a "culture of evaluation", the work of evaluation must answer questions of interest to citizens. France Stratégie, therefore, sought to develop the participative dimension of its public policy evaluations.
As part of the evaluation of the "National Strategy for the Prevention and Fight against Poverty", a "citizens' panel" was established, responsible for reflecting in turn on the evaluation questions and on the interpretation of the evaluation results, when these become available. Acting in support of and in consultation with the "Evaluation Committee" this panel constitutes an innovation in the field of socalled "participatory" evaluation generally reserved for "stakeholders" in a given field of public policy [1]. The panel members were invited to reflect "as citizens" and thus to adopt from the outset the point of view of the general interest, regardless of their personal relationship to the issues of fighting poverty.
A new framework for participation
Without an identifiable precedent, it was necessary to innovate, and to create a framework in which citizens participating in this exercise would be informed, as in a citizens' conference, about the issues at stake, as well as about the major evaluation approaches and methods, while being guided in the drafting of their contribution. To this end, France Stratégie consulted various specialists in citizen participation, and with the help of the IFOP Institute - a service provider selected for this type of citizen consultation process within the framework of public contracts for the Government Information Service-built an original system adapted to the needs and timetable of the operation.
The panel was made up of thirty citizens recruited by telephone in the Fall of 2019, a selection based on socio-demographic diversity that aimed to reflect the French population, notably chosen according to criteria of gender, age, region and type of municipality of residence, occupation and level of education. These thirty persons were asked to take part in two sessions of meetings in Paris; these were organised to enable them to form a collective and conduct their work, while connecting them between those meetings to an online platform that would permit them to pursue their reflections on the evaluation of the poverty strategy at a distance. Their mission? To help the Evaluation Committee define the questions that the strategy evaluation will seek to answer.
A weekend for training
Coming from the different regions of metropolitan France, the thirty panel members met in Paris at their hotel and at the France Stratégie offices for forty-eight hours during the weekend of 30 November and 1 December 2019 to "step into the shoes of the evaluator". In addition to internal discussions within the group, their work consisted of consultations with various experts responsible for introducing them to the themes they needed to understand to tackle the issue. Among these experts were specialists from INSEE, the Conseil d’Etat, the École d’économie de Paris or the Conseil national d’évaluation du système scolaire (CNESCO).
Many questions were addressed, in particular: what are the different ways of measuring poverty? how has it evolved in France and in comparable countries over the last few decades? what policies have been implemented to combat it during this period? what is the new "National Strategy"? and how is it applied in the different territories? who is it aimed at in particular? The citizens were also able to exchange with representatives of anti-poverty associations to understand better the expectations and fears these persons might have, with regard to the poverty strategy. .
The training further focused on the evaluation process itself: what types of questions can it answer? What are the main methods in this area? Finally, the group talked for almost two hours with three members of the Evaluation Committee, including its chairman Louis Schweitzer, to gain a better idea of what the Committee expected of them, and how it could benefit from their contribution once it had been prepared.
During this first weekend, the panelists showed an interest in all these issues, displaying a great curiosity about the data, and at the same time expressing certain difficulties in grasping the issues and tools for evaluating public policies.
One day to draw up an opinion
Left at home with a set of documents and a series of questions, the citizens were then able to exchange with one another on an online platform. This opportunity allowed them to share information from current events, and to reformulate for themselves and for the group as a whole the reflections they had developed during the first weekend, and to prepare for a second physical meeting scheduled for Saturday 18 January 2020.
This second session was devoted to drawing up the "opinion statement" that would be forwarded to the Evaluation Committee and later made public. It was a matter of organising the citizens' messages about the overall strategy - its design and implementation - and on the methods of evaluation. The principal objective was for citizens to express themselves about what they thought to be most significant, and to request what information they needed to form an opinion about this policy – and that it will then be a question of expressing in the form of evaluative questions to shed light on the subject with pertinent insights.
All the work was coordinated at the time by the IFOP without any intervention from France Stratégie or the Evaluation Committee. Its purpose was to develop a common vision enriched by the proposals of each participant, and to prioritize the issues, stressing citizens' requests for information, their "alert points", and what they considered as priorities in the evaluation process. At the end of an intensive day, the written version was prepared, and then validated at a distance by e-mail by all participating citizens during the following week. It was then sent to the Evaluation Committee to be considered in its first annual "progress report".
A long-term project
What did the citizens say? In summary, they asked for clarification on how to measure poverty, and on the objectives of this strategy. They also called for particular attention to be given to those persons who could be overlooked, especially the "invisible" ones that may go unnoticed in many statistics. They insisted, moreover, on the need for a qualitative approach, and on the contribution of actors on the ground. Finally, they drew the Evaluation Committee’s attention to policies not included in the strategy but that may have an impact on the condition of poor people.
These messages were sent to the Evaluation Committee. Louis Schweitzer met with two of the panel members, designated as rapporteurs by their peers, to discuss the citizens' requests, and how the Committee could respond to them. As can be read in the first progress note of the Committee, several of the citizen’s recommendations were, indeed, taken into account, and will, therefore, feed into the evaluation work throughout the project.
This is indeed one of the original features of this project: it does not end with this first exchange but will continue for several years, at least until 2022. As the evaluation is carried out, the same citizens will again be mobilized to formulate their reflections from the results obtained, and thus contribute to the interpretation and the development of the recommendations that will result. An extended period of time, indispensable for a serious evaluation, is also a necessary condition for building a relationship of trust between citizens and public authorities. The next steps will, therefore, be essential to the success of this process.
read the full opinion in french
Participatory evaluation: the voice of people in poverty
1] It is in this sense that the Conseil national d’évaluation du système scolaire (Cnesco) has widely developed participatory evaluation, involving the entire "educational community" in its work.
Auteurs
