The intriguing properties of polaritons resulting from strong and ultrastrong light–matter coupling have been extensively investigated. However, most research has focused on spectroscopic characteristics of polaritons, such as their eigenfrequencies and Rabi splitting. Here, we study the decay rates of a plasmon–microcavity system in the strong and ultrastrong coupling regimes experimentally and numerically. We use a classical scattering matrix approach, approximating our plasmonic system with an effective Lorentz model, to obtain the decay rates through the imaginary part of the complex quasinormal mode eigenfrequencies. Our classical model automatically includes all the interaction terms necessary to account for ultrastrong coupling without dealing with the rotating-wave approximation and the diamagnetic term. We find an asymmetry in polaritonic decay rates, which deviate from the expected average of the uncoupled system’s decay rates at zero detuning. Although this phenomenon has been previously observed in exciton–polaritons and attributed to their disorder, we observe it even in our homogeneous system. As the coupling strength of the plasmon–microcavity system increases, the asymmetry also increases and can become so significant that the lower (upper) polariton decay rate reduction (increase) goes beyond the uncoupled decay rates, γ − < γ 0,c < γ +. Furthermore, our findings demonstrate that polaritonic linewidth asymmetry is a generic phenomenon that persists even in the case of bulk polaritons.