C.H. Spurgeons's The Treasury of David
Verse 34. My covenant will I not break. It is his own covenant. He devised it, drew up the draft of it, and voluntarily entered into it: he therefore thinks much of it. It is not a man's covenant, but the Lord claims it as his own. It is an evil thing among men for one to be a "covenant breaker", and such an opprobrious epithet shall never be applicable to the Most High.
Nor alter the thing that is gone out of my lips. Alterations and afterthoughts belong to short sighted beings who meet with unexpected events which operate upon them to change their minds, but the Lord who sees everything from the beginning has no such reason for shifting his ground. He is besides immutable in his nature and designs, and cannot change in heart, and therefore not in promise. A word once given is sacred; once let a promise pass our lips and honesty forbids that we should recall it, -- unless indeed the thing promised be impossible, or wicked, neither of which can happen with the promises of God. How consoling it is to see the Lord thus resolute. He, in the words before us, virtually reasserts his covenant and rehearses his engagements. This he does at such length, and with such reiteration, that it is evident he takes pleasure in that most ancient and solemn contract. If it were conceivable that he had repented of it, he would not be found dwelling upon it, and repeating it with renewed emphasis.
EXPLANATORY NOTES AND QUAINT SAYINGS
Verse 34. My covenant will I not break. He had said above, If the children of David break my statutes; and now, alluding to that breach, he declares that he will not requite them as they requite him, My covenant will I not break, implying, that although his people may not altogether act in a manner corresponding to their vocation, as they ought to do, he will not suffer his covenant to be broken and disannulled on account of their fault, because he will promptly and effectually prevent this in the way of blotting out their sins by a gratuitous pardon. --John Calvin.