Government evaluations of the effects of industrial policy are not reliable
With few exceptions, outcome evaluations made of industrial policy are not of a high enough quality to draw conclusions on causal links. Only 2 out of 37 audited evaluations reach the required level for the evaluation to be considered reliable, according to the Swedish National Audit Office’s (Swedish NAO) audit.
Central government spends about SEK 60 billion annually on various industrial policy initiatives. In addition, there are EU-funded initiatives of a further SEK 10–15 billion.
To ensure that these resources are used effectively to achieve the industrial policy objectives, it is important that the policy effects are evaluated. This has also been requested by the Riksdag on several occasions.
Among the government agencies, it is the Swedish Agency for Growth Policy Analysis (Growth Analysis), the Swedish Agency for Economic and Regional Growth and Vinnova that produce outcome evaluations of industrial policy initiatives. The Swedish NAO has audited the policy outcome evaluations made by these agencies in the period 2015–2018.
In addition, the annual “effectiveness measurements” carried out by Statistics Sweden on behalf of Almi Företagspartner AB (Almi) are included. They are presented by Almi as policy outcome evaluations, although according to Statistics Sweden they should not be interpreted as such.
The audit shows that the majority of the policy outcome evaluations have significant shortcomings. Of the 37 audited evaluations that make claims on causal links, only 2 are of sufficient quality to be reliable.
– The policy outcome evaluations represent a relatively small proportion of the volume of evaluations, inquiries and analyses produced in the industrial policy area. However, they have an important role to play and development work is needed to increase reliability,” says Auditor General Helena Lindberg.
However, there are differences between the agencies. Policy outcome evaluations from Growth Analysis are in several cases lacking in their analysis. Despite this, they show a much higher quality than reports that state opinions on outcomes from the Agency for Economic and Regional Growth, Vinnova and Almi/Statistics Sweden – which only to a small extent satisfy the most basic requirements to be expected of a policy outcome evaluation.
– Policy outcome evaluations from the Agency for Economic and Regional Growth and Vinnova, as well as Almi’s effectiveness measurements carried out by Statistics Sweden, generally have such major shortcomings that they should not be used for guidance on causal links in industrial policy,” says Niklas Kaunitz, project leader for the audit.
It is difficult to design policy outcome evaluations. The Swedish NAO notes that procurement of consultants has not resulted in significantly higher quality of the policy outcome evaluations, and that cooperation with universities only rarely has resulted in better standards.
Recommendations in brief
The Swedish NAO recommends that the Government use Growth Analysis to a greater extent to evaluate the outcomes of industrial policy initiatives. The Government should further expand and clarify its reporting to the Riksdag on the results of well executed policy outcome evaluations.
Growth Analysis is recommended to further increase the quality of its policy outcome evaluations.
The Agency for Economic and Regional Growth and Vinnova are recommended to obtain help from Growth Analysis in policy outcome evaluation. Furthermore, they should ensure that other types of evaluations and follow-ups do not make claims on causal links, and that their purposes and limitations are clearly communicated.
See the report for full recommendations.
Press contact: Olle Castelius, phone: +46 8-5171 40 04.
Presskontakt: Olle Castelius , telefon: 08-5171 42 06.
Share in social media and by e-mail
Send your questions or comments via the form below and we will make sure that they reach the right member of staff. Please state if your question concerns the information on this particular page.