A procedure to construct temporally local schemes for the computation of fractional derivatives is proposed. The frequency-domain counterpart (iω)α of the fractional differential operator of order α is expressed as an improper integral of a rational function in iω. After applying a quadrature rule, the improper integral is approximated by a series of partial fractions. Each term of the partial fractions corresponds to an exponential kernel in the time domain. The convolution integral in a fractional derivative can be evaluated recursively leading to a local scheme. As the arguments of the exponential functions are always real and negative, the scheme is stable. The present procedure provides a convenient way to evaluate the quality of a given algorithm by examining its accuracy in fitting the function (iω)α. It is revealed that the non-classical solution methods for fractional differential equations proposed by Yuan and Agrawal (ASME J Vib Acoust 124:321–324, 2002) and by Diethelm (Numer Algorithms 47:361–390, 2008) can also be interpreted as applying specific quadrature rules to evaluate the improper integral numerically. Over a wider range of frequencies, Diethelm’s algorithm provides a more accurate fitting than the YA algorithm. Therefore, it leads to better performance. Further exploiting this advantage of the proposed derivation, a novel quadrature rule leading to an even better performance than Diethelm’s algorithm is proposed. Significant gains in accuracy are achieved at the extreme ends of the frequency range. This results in significant improvements in accuracy for late time responses. Several numerical examples, including fractional differential equations of degree α = 0.3 and α = 1.5, demonstrate the accuracy and efficiency of the proposed method.