Defining and measuring the productivity of programming languages

The goal of programming support systems is to make it possible for application developers to produce software faster, without any degradation in software quality. However, it is essential that this goal must not be achieved at the cost of performance: programs written in a high-level language and intended to solve large problems on highly parallel machines must not be egregiously less efficient than the same applications written in a lower-level language. Because this has been a traditional stumbling block for high-level languages, metrics for productivity analysis must explore the trade-off between programming effort and performance. To that end, we propose the use of two dimensionless ratios, relative power and relative efficiency, to measure the productivity of programming interfaces. In this paper we define these concepts, describe their application, and explore various ways for measuring them, including both empirical strategies and expert opinion. Rather than combine these metrics into a single number representing a universal productivity, we propose that they be represented graphically in at least two dimensions so that the trade-offs between abstraction and performance are clearly depicted. However, we also introduce a single problem-dependent parameter that allows us to reason about the relative productivity of two languages for a given problem.