A small but mostly affluent group of Americans are about to see their mortgage payments skyrocket.

They are the more than 1.7 million owners of homes bought since 2019 with an adjustable-rate mortgage. These loans—averaging about $1 million to finance more expensive properties—are set at a rate lower than the prevailing 30-year for the first few years, then adjust once or twice a year based on current borrowing costs.

Coming out of the fixed period after interest rates soared to a two-decade high is the worst timing for an estimated 330,000 of these borrowers. Another 100,000 will see their first adjustment in the next 12 months, according to ICE Mortgage Technology. Even though they tend to be well off, the approaching reset dates are creating some stress, said Chris Stearns, a southern California-based mortgage loan advisor at Thrive Loans.

“They could run into some trouble, especially on these large loan amounts as their ARMs come out of the fixed period,” Stearns said. “Your payment’s gonna almost double and it’s not gonna be pretty.”

Unlike in other countries like the U.K. and Australia, the vast majority of mortgages in the U.S. have long-term fixed rates, which has sheltered most homeowners from the Federal Reserve’s rate-hiking campaign to fight off inflation. About 5.5% of U.S. mortgages are adjustable, although their share in dollars is bigger because they are much larger than the average.

People who take out ARMs often plan on selling their home or refinancing before the initial fixed-rate period expires, but many couldn’t in the current environment because rates were already too high and rose so quickly. They’re also commonly used to finance more expensive properties—the average loan size of an ARM is hovering near $1 million, roughly triple that of a fixed-rate mortgage, according to Mortgage Bankers Association.

Consider a five-year ARM that’s resetting this year. When it was taken out in 2019, the average size was $791,100 with a 3.3% fixed rate—equating to a monthly payment of about $3,465 before home insurance and property taxes. ARM rates are now around 6.5%, although thanks to a cap, that loan would likely reset at 5.3% initially. That would still add almost $1,000 a month.

A survey conducted this month for Bloomberg by polling firm CivicScience Inc. found that 70% of ARM holders say they’re at least somewhat concerned about making the new monthly mortgage payments since interest rates are up. Almost 1 in 10 think they might delay or default on their mortgage once it adjusts.

ARM holders do have some options to stave that off, like borrowing against non-retirement accounts, said Tait Lane, a wealth manager at Merit Financial Advisors in Issaquah, Washington. Another would be refinancing into an interest-only loan. Perhaps the most obvious would be to reduce expenses elsewhere, but Lane—who’s been in the business for 25 years—said that’s a “futile exercise.”

Another avenue is trying to convince the bank that easier terms are warranted, said Steven Ebert, a lawyer with Cassin & Cassin LLP. Those modifications—which include lengthening the amortization period—do exist, but they’re hard to come by, he said.

“You really have to show that you both are having difficulty with the new payment and at the same time you’re able to perform under the other payment,” Ebert said. “So if you just say, ‘oh, I have to cut back on my lifestyle, I can still pay the mortgage,’ that’s not gonna convince the bank. You really have to show that there’s a real affordability problem.”

Borrowers with conventional mortgages pay a premium for protection against rising rates, and former Fed Chair Alan Greenspan once said that many Americans could save tens of thousands of dollars with ARMs if they were willing to take that risk.

That sentiment is resonating with more borrowers now as current Fed Chair Jerome Powell has said that rates will likely stay high for longer, as have several of his colleagues.

The last time ARM holders really got into trouble was during the global financial crisis, when underwriting standards were much less stringent and home values sank. But the share of ARM borrowers today is a fraction of what it was in the years leading up to the crisis, so there isn’t as much cause for concern, said CoreLogic Chief Economist Selma Hepp. Average credit scores nowadays are much higher, too.

“By the time these loans are due to reset, mortgage rates are likely to be lower,” Hepp said.

With home prices up more than 40% since the pandemic, many owners who have gained equity and are in need of significant financing—perhaps to pay for a home renovation or college tuition—have taken out home equity lines of credit, in which the collateral is the borrower’s property.

But HELOCs, like ARMs and credit cards, have variable interest rates—which are now double pre-pandemic levels—so even someone with a conventional fixed rate mortgage locked in at a low level could be exposed.

The best hope for these borrowers is that interest rates come down, but that probably won’t happen until later this year at the earliest as the Fed has yet to become confident enough that inflation is on a sustainable downward trend.

“The problem is there’s not really anywhere to run right now until rates get better,” said Merit’s Lane. “We all kind of thought that was gonna happen by now, but the inflation data that keeps coming in is less than ideal.”

This article was provided by Bloomberg News.