Overshifted parameter-shift rules: optimizing complex quantum systems with few measurements

Gradient-based optimization is a key ingredient of variational quantum algorithms, with applications ranging from quantum machine learning to quantum chemistry and simulation. The parameter-shift rule provides a hardware-friendly method for evaluating gradients of expectation values with respect to circuit parameters, but its applicability is limited to circuits whose gate generators have a particular spectral structure. In this work, we present a generalized framework that, with optimal minimum measurement overhead, extends parameter shift rules beyond this restrictive setting to encompass basically arbitrary gate generator, possibly made of complicated multi-qubit interactions with unknown spectrum and, in some settings, even infinite dimensional systems such as those describing photonic devices or qubit-oscillator systems. Our generalization enables the use of more expressive quantum circuits in variational quantum optimization and enlarges its scope by harnessing all the available hardware degrees of freedom.