His resignation comes bare hours after Prime Minister Theresa May announced that she had secured the backing of her cabinet for the agreement, after a five hour meeting.
Several ministers were understood to have spoken against it, with the BBC reporting that Conservative backbenchers were in talks to force a no-confidence vote in her.
Mr Raab – a Leave supporter who was promoted to the cabinet to replace David Davis when he quit in protest at Mrs May’s Brexit plans – is among a group of senior ministers thought to be unhappy with the agreement.
In his resignation letter, Mr Raab said he could not support it because the regulatory regime proposed for Northern Ireland “presents a very real threat to the integrity of the United Kingdom”.
And, he added, the “backstop” arrangements aimed at preventing the return of a hard Irish border would result in the EU “holding a veto over our ability to exit”.
“Above all, I cannot reconcile the terms of the proposed deal with the promises we made to the country in our manifesto at the last election,” he told the prime minister.
Analysts say Raab’s departure puts pressure on other cabinet members to quit, raising the prospect of a “domino effect” that could end in the break-up of the cabinet.
Conservative Brexiteer MP Anne Marie Morris told BBC she believed enough Tory MPs had now sent letters to the chairman of the 1922 committee to trigger a leadership contest.
She said there was enough time to install a new prime minister and change course on Brexit, adding: “Now is not the time for her leadership.”
Labour’s shadow Brexit Secretary Sir Keir Starmer said: “It is now clear the prime minister didn’t have the backing of her cabinet.”