Georgia operative Nick Ayers, a top aide to Vice President Mike Pence, is headed back to Atlanta soon after he declined Sunday to take the job as President Donald Trump’s chief of staff.

Ayers, 36, was long rumored to be Trump’s top choice to replace outgoing chief of staff John Kelly, a retired U.S. Marine Corps general who the president said will leave the job by year’s end.

But the two couldn’t agree on a time frame for the job, with Ayers unwilling to commit to the role deep into next year, according to an Ayers’ associate familiar with the talks.

Ayers did not return messages seeking comment, but said in a Sunday tweet that he will be departing his job with the administration at the end of the year but will work with the Trump campaign “to advance the cause.” He ended the tweet with a Georgia hashtag.

It’s not clear now who will take Kelly’s job as the top West Wing adviser as Trump prepares for his 2020 re-election bid at a precarious time.

Whoever takes the job will have to contend with an intensifying probe into Russian election interference and emboldened Democrats who will take control of the U.S. House in January. Kelly’s successor will also have to grapple with a mercurial president loath to take counsel from his advisers.

“Nick has three little kids and that chief of staff job is a nightmare - no matter who the president is,” said Alec Poitevint, a longtime Ayers confidant and influential Georgia GOP donor.

It’s not immediately certain what Ayers will do next in Georgia, but he will have plenty of options.

He and his former boss, Sonny Perdue, were the driving forces behind Trump's surprise endorsement of Brian Kemp, and Ayers helped organize Pence and Trump's recent visits to Georgia. And he's built a reputation as a wealthy and well-connected strategist after getting his start in Perdue's 2002 campaign.

He was still a student at Kennesaw State University then, where he went to school with dreams of being a banker, but got swept up in Perdue’s underdog bid to become the first Republican governor in Georgia since Reconstruction.

“I had no interest in joining the campaign. I had my career planned out. I truly did not believe Governor Roy Barnes could be beat at the time, ” Ayers said at the time. “After 10 minutes of talking to Sonny, I was one hundred percent confident he was the right person to run this state.”

He soon became the Republican’s right-hand man – part assistant, part adviser, part protégé – and got hooked on politics. He also became a part of the family: He married Perdue’s second-cousin, Jamie, in May 2005 and the couple are now raising 6-year-old triplets in metro Atlanta.

Four years after his upset victory, Perdue tapped Ayers to serve as his campaign manager for reelection against Democrat Mark Taylor.

Ayers went on to become the youngest-ever head the Republican Governors Association during a period of rapid state-level expansion for the GOP. In that role, he and then-Mississippi Gov. Haley Barbour helped transform the organization from a bit player to an influential political network.

He later built a consulting business that successfully boosted Republican candidates and made him fabulously wealthy: His federal financial disclosure pegged his worth between $12 million and $54 million. But they also raised questions about how he amassed his fortune in a short period of time, including recent complaints of running afoul of ethics laws.

Trump’s rapid rise in the Republican world opened up new opportunities for Ayers.

He became a top Pence counselor during the presidential campaign and, last year, briefly flirted with a run for Georgia governor. But he abruptly ruled out that bid moments before news broke that Pence offered him the chance to be his top staffer.

Since arriving in Washington, Ayers won Trump’s admiration for insulating Mike Pence from the chaos that’s frequently engulfed the West Wing and cultivated the support of key Trump family members, including Don Jr. and Ivanka Trump.

He’ll surely lean on those connections with national figures, as well as his ties to a universe of wealthy GOP donors, as he builds his next venture.

Brandon Phillips, a Georgia GOP consultant who was Trump’s state campaign chair, called him a “role model for political operatives who think nice guys can’t finish first.”

“His future in Georgia and national politics is only limited by his imagination,” said Phillips.