INNOVATIVE ACTIVITY OF INDUSTRIAL ENTERPRISES: MEASUREMENT AND EFFECTIVENESS EVALUATION

INTRODUCTION

Industrial companies are a key element of the national innovation system. The volume of their investments into research and development (R&D), innovative projects is significant enough to have a decisive influence on the technological development of the relevant industries. The mechanisms of such influence can be the transfer of technology and commercialization of in-house developments, the ordering of R&D and innovative products (works, services), the creation of competitive environment on the markets where the company operates, making other entities carry out similar innovations. The role of industrial companies as drivers of innovative development should be taken into account when developing their technological strategies, identifying priority technological areas on which resources should be concentrated.

However, in making strategic decisions it is necessary to correctly assess the innovative activity of industrial companies.

There are many studies devoted to the measuring of companies’ innovative activities. However, a unified approach to understanding the assessment, which makes it possible to fully assess the level of innovative activity, has not been developed. Most often the literature contains some approaches based on the following indicators: R&D costs, number of patents, citation of patents, the number of new products introduced into the market, etc.

The purpose of this study is a deeper research of the indicators of innovative activity of industrial companies and the formation of multidimensional aggregate assessment of innovative activities of industrial enterprises. The formation of such an indicator will make it possible to carry out more informative assessment of the companies’ innovative activity and the transformation of its innovative behavior.

1. EFFICIENCY OF INNOVATION PERFORMANCE: KEY METRICS AND THEIR EVOLUTION

The studies devoted to the measurement of innovation activity can be divided into three main areas:

• studies dedicated to the indicators measuring the resources and the results of innovative activities of companies;
• studies describing the metrics of innovative activities and the methodology of their measurements;
• studies aimed at the formation of multidimensional indicators for assessing various signs of innovation.

1.1. INDICATORS MEASURING THE RESOURCES AND THE RESULTS OF INNOVATIVE ACTIVITIES OF COMPANIES

There are many studies devoted to the measuring of input resources and output results as effectiveness indicators of innovative activities of companies in various industries and fields of activity. Thus, based on the data of 3 247 Spanish companies, the work of Potters (Potters, 2009) analyzes the influence of innovative resources (inputs) on innovative behavior. It is concluded that the same set of resources leads to different results of innovative activity depending on what innovative regime the company belongs to.

Muller et al. (Muller, A., Valikangas, L., Merlyn, P., 2005) identified three areas of assessment (resources, possibilities and leadership), while Milbergs and Vonortas (Milbergs and Vonortas, 2006) classified indicators according to four stages (indicators of incoming flows, indicators of outgoing flows, indicators of processes and indicators of innovation). Around the same time Adams et al. (Adams, R., Bessant, J., Phelps, R., 2006) in their review of the literature on innovation management identified six key aspects of assessment: incoming flows, knowledge management, innovation strategy, organization and organizational culture, portfolio management, project management and commercialization.

The work of Spanish researchers Vega-Jurado et al. (J. Vega-Jurado, A. Gutierrez-Gracia, I. Fernandez-de-Lucio, and L. Manjarres-Henriquez, 2008) analyzed the activity of 6094 manufacturing companies in Spain and concluded that technological innovation is a key factor in the effectiveness of innovations. Later these findings became the basis for the Spanish innovation policy.

The report of the 21st Century Economy Advisory Committee (USA) devoted to innovations (M. Reffitt, C. Sorenson, N. Blodgett, R. Waclawek, and B. Weaver, 2007) concludes that most studies are focused on input (resources) for innovative activities, while there are not enough studies devoted to the effectiveness of innovative activities. The key conclusion of the report was about the need to improve the measurement of innovative activities by analyzing as many new indicators of innovation as possible, especially those characterizing performance. This conclusion is also made in the work of Gold (Gold, 1973) “The Impact of technological innovations - concepts and measurement,” which finds the research on innovative measurements unsatisfactory and suggests that models are needed that link the inputs and outputs of innovative activities.

In further studies this leads to the formation of models that determine the relationship between productivity and investments of companies into innovation. For example, Griffith et al. (R. Griffith, E. Huergo, J. Mairesse, and B. Peters, 2006) by using the statistical data of four countries (France, Germany, Spain and the United Kingdom) determine that the results of innovative activities, expressed in the share of innovative products, depend on the size of the company’s investments in research and development calculated per one employee in the field of research and development.

1.2. METRICS AND METHODOLOGIES FOR MEASURING INNOVATIONS

The second area of research is focused on metrics and methodologies for measuring innovative activities of companies.

For example, in the study of L. Morris (L. Morris, 2008) the formation of metrics of innovation activities is tied to

each stage of the innovation process. In total Morris identified nine key processes for the emergence of innovations - strategic thinking, portfolio management and indicators, research, imagination, understanding, planning, innovative development, market development and sales. At each stage Morris identified some possible quantitative and qualitative metrics. The possibilities of using the process approach to assessing the effectiveness of innovative activities are described in the work of Trachuk and Tarasov (Trachuk, Tarasov, 2015).

In his work I. Pallister (Pallister I., 2010) also identifies the metrics for measuring innovations and divides them into the metrics characterizing innovation processes and the metrics characterizing innovative activities of companies. At the same time Pallister notes that different measurement mechanisms, input indicators and performance indicators will differ depending on the industry in which the company operates. Indeed, there are many studies evaluating the effectiveness of innovations in companies of various industries in which different indicators are applied. For example, a study by Hansen et al. (E. Hansen, H. Juslin, and C. Knowles, 2007) interviewed forestry managers and identified indicators and factors that stimulate innovative activities. The study of Ivanova (A. Ivanova, 2015) is also devoted to determining the metrics of effectiveness of innovative activities of companies specializing in forest biotechnologies. The author proposed a system that includes the following indicators: availability of production capacities to create innovations; the state of preparatory processes; provision of material and technical resources; personnel potential. B. Pozdnyakov (Pozdnyakov, 2009), analyzing the effectiveness of innovative activities in the flaxseed complex, justified the need to take into account an increase in costs of fiber contained in flaxseed. The author also proposed to take into account not only the direct effect of innovative activities, but also the indirect ones related to environmental consequences.

These studies confirm the validity of the thesis put forward by Pallister about the need to use different mechanisms for measuring innovations and their effectiveness in companies of various industries. B. Glassman (Glassman, 2009) comes to the same conclusion. He identifies a single data set for assessing innovative efficiency and concludes that it cannot be used for all companies in different industries. Therefore, Glassman proposes to use subjective measures characterizing the peculiar features of the innovation process of companies in specific industries and gives recommendations on how to choose metrics according to the needs of companies.

A number of researchers suggest using not individual indicators to assess the effectiveness of innovative activities, but models that make it possible to carry out a comprehensive analysis. Thus, P. Gupta (Gupta, 2007) in his study concludes that most studies measure the effectiveness of innovations by measuring the share of innovative products, which makes it impossible to conduct a full analysis of the company's innovative activity and its effectiveness. In this regard the author proposes the SIPOC model (S - supplier, I - input, P - process, O - output and C - customer), which makes it possible to analyze the process of innovation.

Coombs et al. (R. Coombs, P. Narandren, and A. Richards, 1996) have proposed a model called LBIOI for evaluating innovative activities. This model helps assess the degree of novelty of products manufactured by companies, as well as their technologies. At the same time, this model is not applicable for evaluating process and organizational innovations.

Cordero (Cordero, 1990) creates a model for measuring the effectiveness of innovations by using resource and product indicators. At the same time, he measures every stage of the innovation process: designing, modeling and production of a prototype, testing and commercialization. The results of the study show that the most commonly used indicators for assessment are the quality of technical products, the level of achievement of goals and the amount of work completed on time.

Verhaeghe and Kfir (Verhaeghe, A., Kfir, R., 2002) presented a management and evaluation model consisting of 10 separate aspects: leadership, resources’ provision of innovations, systems and tools, proposals of innovations, proposals of developments, transfer of technologies, absorption of technologies, market focus, innovate activity and networking.

Suomala (Suomala, P., 2004) substantiated a life cycle approach to evaluation. In his opinion, at each phase of the cycle specially selected performance indicators should be used. Suomala identified the following phases of the life cycle of an innovative product: (1) feasibility study/preparation; (2) product development; (3) launch on the market; (4) active phase; (5) support, provision and further development; (6) end of the cycle.

Ortiz et al. (Ortiz F. I., Brito E. E., and Ovalles M. L., 2007) offers a system of measurement for technological innovations of products and processes. They use sets of indicators identified by experts to determine a system of measurement. The system consists of tools, procedures and methodologies for analyzing the effectiveness of innovative activities and makes it possible to compare companies in different industries and fields of activity.

Finally, a number of researchers use econometric modeling to analyze the relationship between innovations and company performance. For example, Mairesse and Mohnen [Mairesse J. and Mohnen P. A., 2004] by using econometric modeling show that the company's investment into R&D has a positive effect on the effectiveness of innovative activities and the company’s productivity in general. They also maintain that an overall elasticity of investments in research and development is higher in low-tech industries compared to high-tech industries. Trachuk and Linder (Trachuk., Linder., 2018) used econometric modeling based on the data of the Russian industrial companies and concluded that the impact of investments into innovations of productivity depends on the “intensity” of investments in R&D; the relationship between investments in innovation and productivity growth is non-linear and has a stable positive relationship only after a certain critical mass of investments into R&D has been achieved; the characteristics of industries in which companies operates have significant impact on the relationship between investments into innovations and productivity. Companies operating in high-tech industries not only invest more in R&D and innovations, but also have higher productivity due to research and development.

Loof and Heshmati (Loof and Heshmati, 2002) conduct econometric studies based on the data from Swedish manufacturing companies. This study is also focused on the relationship between innovations and productivity growth in industrial companies. They examine the existing econometric methods, propose a new model and apply it to the generated set of data. The results show that the level of effectiveness of innovation activity significantly grows with an increase in investments in R&D.

1.3. THE FORMATION OF MULTIDIMENSIONAL INDICATORS TO ASSESS THE EFFECTIVENESS OF INNOVATIONS

The third group of studies is devoted to the use of multidimensional indicators that make it possible to evaluate various aspects of the company's innovative activities.

Sood and Tellis (Sood and Tellis, 2009) study the growth in the costs of innovations at each stage of the innovation process. The result of the study was the conclusion that the growth in the costs of innovations is affected more by the phase of innovation development than by the phase of commercialization. Therefore, in order to attract more market attention, companies should advertise their research and development as much as possible.

Choi and Ko (2010) have proposed an integrated assessment of innovation activity, which makes it possible to measure both the effectiveness of investments in R&D and their impact on the results of innovation activities. The assessment consists of four groups of innovative metrics measuring all innovative processes of companies. An integrated assessment of innovative activity was also proposed in the work of Trachuk and Linder (Trachuk, Linder, 2016).

The study of Salomo et al. (S. Salomo, K. Talke, and N. Strecker, 2008) for the first time defines the concept of innovative orientations of companies, which are understood as a strategy of managing the company's innovative activities. At the same time, such areas of management as joint research and development, customer value, formation of consumer groups, technological leadership and formation of unique competencies of the company are highlighted. The finding of this study is the conclusion that only those companies that manage innovative areas are more successful and have portfolios of more innovative products than others.

Rosaermeland Hess (Rothaermeland Hess, 2007) develops a multi-level theoretical model to evaluate innovativeness on three different levels: the level of an individual company, the industry level and the level of networks. They collect data from pharmaceutical companies and base the dependence of the level of innovative products on independent variables (factors) - investments in R&D, company size, number of employees, etc. This study shows that intellectual human capital significantly influences the innovativeness of companies.

In his writings Shapiro (A. R. Shapiro, 2006) concludes that it is impossible to determine the innovativeness of a company by any single measure. He proposes the use of “fixed” and “variable” indicators in pairs in order to effectively analyze the level of innovative activity.

The “fixed measure” refers to the percentage of income from the sale of new products while variables include technologies, production, organizational innovation, etc., which also demonstrates the degree of novelty and innovativeness. The author believes that the level of innovative activity of the company will be measured better by combining these indicators.

Our study is focused on the study of factors contributing to the improvement of innovation performance of industrial companies and the formation of an aggregated assessment of innovative activities.

2. RESEARCH METHODOLOGY

Innovations in industrial companies are a complex phenomenon that is difficult to evaluate only from secondary sources. Therefore, to form an aggregate indicator for the assessment of innovative activity we used a consistent approach, which includes:

• analysis of the factors of effectiveness of innovative activities;
• collection and analysis of information regarding the indicators used to assess the innovative activity of industrial companies taking into account the identified factors of innovation activity;
• compilation of an aggregate and a short list of indicators evaluating innovation activity taking into account the identified factors of effectiveness of innovative activity;
• formation of an aggregate indicator of innovative activity of industrial companies.

The collection of information included:

• the analysis of domestic and foreign literature on the measurement of innovative activity of industrial companies;
• the analysis of published reports of consulting companies on the indicators used to assess innovative activities of large industrial companies;
• the analysis of data on the assessment of innovative activity of large industrial companies of the American Productivity and Quality center (APQC);
• the collection and analysis of materials of the websites of large industrial companies, as well as the analysis of innovative development strategies and innovative development programs of state corporations and companies with state participation in terms of indicators used to evaluate innovative activities;
• interviews with experts about the indicators for assessing innovative activities of industrial companies.

For the analysis of literary sources we selected the scientific literature according to the following criteria: relevance, reliability of the source (scientific validity of the work is confirmed), peer-review of the published materials.

After that we analyzed innovative development programs of state corporations and companies with state participation such as:

1. SC "Rosatom"
2. JSC “RusHydro”
3. JSC “FSK EAS”
4. JSC “Gazprom”
5. JSC “Russian Railways”
6. OJSC “System Operator of the Unified Energy System”
7. FSUE "Rosmorport"
8. FSUE Uralvagonzavod.

We also analyzed the strategies for innovative development of large private industrial companies (a total of about 57 companies).

We studied the reports of big consulting agencies on innovative companies. Such reports combine the experience of the leading world practices in all areas of innovative development and describe the tools for the analysis of recommendations for the development of innovative companies containing aggregated information on the latest trends in innovations and breakthrough technologies that have an impact on the global economy.

We also analyzed the experience of the American nonprofit organization APQC (American Productivity & Quality Center) specializing in research in the field of business process efficiency, conducting a comparative analysis of the practices of various companies and determining the best practices.

The research of APQC is based on information provided by APQC member organizations. Currently, over 320 global companies are members of APQC. Therefore, the indicators obtained by APQC in conducting research and interpreting the results of innovative activities were also interesting for us.

Interviews with experts and data processing: for preliminary testing of the selected aspects of innovative activities of companies in a small sample 16 interviews were conducted with managers of companies working in various industrial sectors. The sample includes both the Russian and foreign companies operating on the Russian market. The materials of interviews were analyzed using content analysis.

Selection of indicators: the selection of indicators was carried out with reference to the factors that evaluate the key factors of the effectiveness of innovative activities in industrial companies.

Analysis and aggregation of indicators: the initial list, including 103 indicators of innovative activities, was aggregated and reduced to a single methodology taking into account the minor differences in formulations and units of measurement. Then, on the basis of the aggregated list, a short list of indicators was formed, on the basis of which an aggregated indicator of innovative activity was formed.

The research methodology is shown in Figure 1.

3. FACTORS, WHICH CONTRIBUTE TO THE INCREASED EFFICIENCY OF INNOVATIONS

The analysis of the existing literature identifies four classic factors in the effectiveness of companies' innovative activities: access to financial capital, qualified personnel, effective organization of innovation processes and formation of a corporate culture aimed at creativity (see, for example, Calantone., Chan, Cui, 2012; Galende and de la Fuente, 2013; Gatignon, Tushman, Smith, and Anderson, 2012].

At the same time, based on in-depth interviews our study makes it possible to supplement them with a number of factors that determine the success of innovative activities of industrial companies.

Focus on technological innovations. As a factor of success most of the interviewed company executives noted research, the development of scientific and technological ideas, creation and development of new products (78%). At the same time, if additional investments were made in innovative activities, 69% would use them to develop new products; 61% - to introduce new technologies and to purchase equipment. When analyzing the strategies of innovative development 46% of enterprises prioritize innovations, 64% want to optimize innovations (cost reduction, customization, and improvement of product quality), 31 % are focused on creating fundamentally new products or services.

Customer focus. 74% of company executives consider customer satisfaction an important success factor, 69% - the company's ability to adapt a product or a service to customer requirements; 58% - creation of not just an innovative product, but a combination of products, services and information that can satisfy the needs of a particular client; 43 % - creation for the client the possibility to switch to a new product of the company; 51% - development of a mechanism for interaction with customers aimed at strengthening customer loyalty and satisfaction.

Training of employees. 78% of respondents consider the presence of highly professional teams of developers and design engineers as well as the training of personnel in order to form competencies for the development of new technological solutions and products as one of the most important success factors.

Company size and competitiveness. 71 % of the surveyed company executives consider the size of companies to be a significant factor in increasing innovative activities as larger sizes give more opportunities to access capital, greater market share, etc. 88% of managers consider their companies to be market leaders, of which 23 % called their companies indisputable leaders while 62 % admitted that their company is one of 2-3 leaders on the market.

Access to international markets. 74% of companies operate not only in Russia, but also on the foreign markets exporting goods and services. An average export share among the surveyed companies is 16%.

The key role of CEOs. 87% of respondents consider CEOs of companies to be the main initiator of development programs for new products and services. In 58% of companies a CEO oversaw the creation of at least one company product, and in 31 % of companies - the development of all new products.

Intercompany cooperation. 76% of top managers of industrial companies recognized the factor of intercompany cooperation as important for the success of innovative activities, while 68% considered cooperation between company divisions or within the holding more important, 44 % - cooperation with external organizations.

The ability to receive external financing was not named among the main success factors for innovative activities of companies. State support also did not play a key role in the development of innovations of the companies we studied. 82% of executives replied that at least once they received some state support, but only for 13 % the received state support significantly influenced business development and innovative activities. 52% of companies received grant financing from the state. The survey data also show that 49% of companies do not encounter significant administrative barriers in the process of developing and commercializing innovations, while 23 % believe that there are such barriers, but they have already learned to overcome them.

We have identified the following key factors that contribute to improving the effectiveness of innovations:

• customer focus;
• focus on technological innovations;
• company size and competitiveness;
• the key role of CEO;
• the volume of total investments in the company’s activity (both current and capital);
• qualified personnel and training of employees;
• intercompany cooperation.

We will consider the formation of an aggregate indicator for the assessment of innovative activity taking into account the identified performance factors.

4. RESEARCH RESULTS

4.1. ANALYSIS OF SCIENTIFIC PUBLICATIONS, EXPERT OPINIONS AND RECOMMENDATIONS OF THE LEADING PRACTITIONERS

According to the results of the literature review, the following indicators can be distinguished for assessing the innovative activity of companies (Table 1).

We also analyzed indicators with median values recommended by the American Productivity and Quality Center² (Table 2).

Table 3 presents the results of the analysis of indicators for assessing innovative activities reflected in innovative development programs of state corporations and companies with state participation, as well as innovative development strategies of private industrial companies.

The analysis of reports of consulting companies makes it possible to use the best international practices in the analysis of innovations and identification of indicators for assessing innovative activities. Examples of the key KPIs used for evaluation by consulting companies are shown in Table 4.

On the basis of the conducted analysis and identification of indicators for assessing innovative activities an extended list of indicators of 103 KPIs was made up.

The analysis resulted in an extended list of indicators that are potentially suitable for inclusion into the integral indicator for assessing innovative activities of industrial companies.

4.2. FORMATION OF AN AGGREGATE INDICATOR FOR ASSESSING INNOVATION ACTIVITIES OF INDUSTRIAL COMPANIES

103 indicators selected during the previous stage went through the aggregation process with the aim of eliminating the double counting of indicators and combining similar indicators. We analyzed the indicators for their thematic content, purpose, wording and units of measurement in accordance with the factors of innovative activities.

As a result, an aggregated list of indicators was obtained (Table 5).

Then the indicators of the aggregated list were analyzed by factors:

• availability of input data and transparency of the calculation methodology;
• mentioned more than once in information sources;
• the indicator corresponds to the factors of innovation performance we have identified (Fig. 2).

For the formation of an aggregate indicator 5 key indicators were selected that correspond to such performance factors as customer focus, focus on technological innovations, company size and competitiveness, total investments into the company’s activities, qualified personnel and employee training, as well as three indicative indicators consistent with the innovative development strategy of an industrial company (the key role for top management), and an indicator characterizing the degree of intercompany cooperation.

Table 6 shows five key indicators for assessing the innovative activity of industrial companies, which are included in the calculation of the integrated indicator of innovative activity, and three indicative indicators that are not included in the aggregate indicator, but which reflect the company's innovative development strategy and the level of intercompany cooperation, both external and internal.

Based on the conducted analysis we propose to calculate the integral indicator of innovative activity of an industrial company according to the following model:

where: X₁ X₂, X₃ X₄, X₅ are shares in the integral indicator KPI₁, KPI₂, KPI₃, KPI₄, KPI₅ respectively;
X₁ + X₂ + X₃ +X₄ + X₅ = 100%.

The choice of weighting factors X should be determined by the company’s innovative strategy and the priority of the set tasks.

5. CONCLUSIONS AND APPLICATION OF THE OBTAINED RESULTS

In order to form an integral indicator of innovative activities of industrial companies we analyzed the published reports of 57 leading companies in industrial sectors over the past 5 years and more than 40 scientific publications evaluating the innovative activity of companies. We also conducted 16 interviews with experts of big industrial companies to assess their innovative activities and analyzed KPIs of innovative activities recommended by the American Productivity and Quality Center (APQC).

The research proposed a methodology for multivariate assessment of innovative activities of industrial companies recommending 5 key efficiency indicators for calculating the integral indicator and 3 indicative indicators not included in the integral indicator. An algorithm for calculating the integral indicator of innovative activities was proposed.

The proposed algorithm for calculating the integral indicator of innovative activities makes it possible to compensate for insufficient achievement of target values for one indicator by overfulfillment of target values for other indicators. In Russian practice the value of the integral indicator can be considered satisfactory if it deviates downward by no more than 10%.

In addition to the integral indicator it is recommended to calculate three indicative indicators, the values of which do not affect the integral indicator:

1. the comprehensive indicator for the development of the scientific base and ongoing research of universities, research institutes, scientific organizations, R&D and technology platforms, percent;
2. the number of proposals for the creation of new technologies, technical and technological solutions received from subsidiaries, affiliates and external organizations - partners in the accounting year;
3. the number of new competencies in the company due to the results of innovations.

Indicative indicators should be consistent with the company's innovation program and policy, as well as with the strategic development priorities of industrial companies.