How to Retire Before 50

Planning for retirement can feel like one unending slog to the finish line — and for many Americans, that finish line keeps moving. According to a 2014 Gallup poll, the average retirement age is now 62, up from 59 in 2010.

Don’t let this discourage you: You don’t have to be average, as long as you don’t make the same mistakes many of your peers are making.

A 2014 survey conducted by Wells Fargo found that more than a third of middle-class Americans aren’t contributing anything to a retirement account. Indeed, almost a third of American workers have less than $1,000 in savings, period, according to the Employee Benefit Research Institute.

In fact, not only is it possible to retire wealthily, you can do it at 50. We talked to a number of financial experts — including one man who saved enough money to retire at age 45 — to learn what you need to do now to make early retirement a reality.

Read: You Don’t Really Need $1 Million to Retire

How One Man Saved Enough to Retire by Age 45

“When I was in my early 30s, I decided I wanted to retire at 45,” said Chris Carosa, veteran investment adviser and author of several books, including “Hey! What’s My Number? How to Improve the Odds You Will Retire In Comfort.” “At the time, I had a huge 30-year mortgage, a young family and a job that paid a modest salary. I knew I had to address four key areas if I was going to be able to retire at 45.”

Carosa created an action plan to achieve his early retirement goals.

1. Pay Off Your Mortgage Early

“I did this by first refinancing (to take advantage of lower rates) to a 15-year mortgage,” Carosa said. “Then I began a regimen of paying two months of payments every month. This proved successful and the mortgage was paid off in my early 40s.”

2. Get a Handle on Your Budget

“At the time, I read many books about living frugal,” Carosa said. “Unless you’re a trust fund baby (I’m not), you’re going to have to live modestly both heading into retirement and subsequent to retirement. To accomplish this, you need to be very good at tracking expenses.”

3. Maximize Your Savings

“Excess cash flow immediately went into savings, but not into a bank account — rather, into a long-term growth fund,” Carosa said. “First it was the Magellan Fund (then a very popular investment), but that quickly morphed into an investment club I started with some college classmates. Any big one-time bonus I got went directly into retirement savings.”

4. Find a Source of Passive Income

“Retirement can be filled with surprises — usually extra expenses that you don’t count on,” Carosa said. “This means all those plans to hit a certain retirement savings target can disappear with one unexpected expense. To help mitigate this, I spent a lot of time trying to determine the best way to create a royalty-based income stream that would continue to generate income throughout my retirement.

“For many people, this involves buying real estate, with the rental income being the revenue stream. But I was never one to like property management, since, after all, that entails work I was not passionate about. No, I wanted something that was ‘one-and-done,’ meaning I would exert a defined unit of energy to create something, then sit back and watch the cash come in.”

Carosa said he could only think of one way he could do this: writing books.

“A single book only produces a small amount of steady income — even best-sellers, most of which give you a lot of money quickly, but then peter out,” he said. “With a stable of books, now you’re talking real money.”

Read More: How Ultimate Cheapskate Jeff Yeager Retired at 47

How to Invest Your Retirement Accounts

Once you’ve decided to ramp up your retirement goals, you need to figure out the best vehicles for growing your savings — and how you can make sure they get you the best dividends.

1. Use Fixed Income Annuities

“Fixed Income Annuities are a savings vehicle that does not lose value, even when the market takes a downturn — you will never lose your principal,” said Jim Poolman, executive director of the Indexed Annuity Leadership Council. “The value of an FIA can increase due to index growth, but it will never decline, even if the index does. And they offer the opportunity for guaranteed income for life — really critical right now as people are concerned about running out of money in retirement.”

2. Use Exchange Traded Funds

Sally Brandon, senior vice president of client services and an investment advisor at Rebalance IRA, recommends investing in exchange-traded funds.

“It sounds complicated but really it’s not complicated at all,” Brandon said. “An ETF is simply a way to own a large group of stocks or bonds without having to buy and sell hundreds or thousands of individual holdings. Most often, ETFs are created to represent a specific index, such as the S&P 500 or the total bond market.”

3. Diversify Your Portfolio

“It is important to diversify your portfolio, based on your tolerance of risk and to rebalance your portfolio on a yearly basis,” Poolman said. “We all go through life cycles, and it is easy to put your money away and forget it. Those savings still need your attention, and as you get older, your tolerance for risk will and should change.”

4. Stock Up on Low-Risk Investments

Brandon recommends choosing low-risk investments that offer steady growth.

“People often place a lot of emphasis on making specific investment choices at specific points in time,” Brandon said. “It’s understandable. They hope to ‘time the market’ and get ahead of the game by making a large amount of money in a short period of time. The problem is that those kinds of outsized risks are counterbalanced by equally large or larger losses. Emotions begin to take a role in the decision-making process and from there it’s incredibly easy to lose even more money. The smart way to grow a retirement plan is to rely on steady growth, diversification and compounding while keeping costs minimized.”

How to Jump-Start Your Retirement Savings

If you want to retire at age 50, you can’t start saving too soon, said Wes Moss, chief investment strategist at Capital Investment Advisors and author of the book, “You Can Retire Sooner Than You Think.”

“Most people that retire at 50, in a traditional sense, are those who start accumulating assets and investing in their teens or 20s — excluding business owners who cash out for millions, tech entrepreneurs and lottery winners,” he said. “Age 20 to 50 will give you 30 years of compounding. At 7 percent per year, you’ll need to save about $11,000 per year for 30 years to hit $1 million.”

To generate money to save, Moss recommends purchasing rental properties, getting a part-time job, paying off your mortgage and taking out a private health care policy to bridge you until Medicare. You also shouldn’t count on income from Social Security and pensions, he said, as it won’t kick in until your 60s, or later.

3 Important Tax Considerations for Early Retirement

If you’re looking to retire early, there are a number of factors you need to consider — including how you’ll be affected by taxes. Here’s why:

1. Watch Out for Early Withdrawal Penalties

If you take an early withdrawal from a retirement plan, such as a 401(k), before you reach age 59 and a half, you could be subject to an additional 10 percent tax on your withdrawal (this tax typically does not apply to nontaxable withdrawals).

Transferring a withdrawal from one qualified retirement plan to another within 60 days counts as a rollover and can save you from having to pay both income tax and the additional 10 percent tax.

2. Tax Rates Could Rise

There’s no telling what the future might hold in terms of taxes, so be prepared for rates to rise during your retirement. This could especially be an issue if you have a substantial amount of savings in tax-deferred retirement accounts.

3. Future Tax on Social Security Income

When the time comes to collect Social Security benefits, keep in mind that some people do have to pay federal income taxes on them. If you have substantial income from other sources coming in, you might be forced to pay taxes on up to 85 percent of your Social Security income.

For example, if you file a federal tax return as an “individual” earning between $25,000 and $34,000, you could have to pay income tax on up to 50 percent of your benefits — increasing to 85 percent if you earn more than $34,000 per year.

Retiring at age 50 can absolutely become a reality if you play your cards right. Create a savings strategy now and vow to stick with it, so you can get the most from your golden years.

Read: Save Money by Retiring Abroad: How to Retire Overseas for Less

This article originally appeared on GOBankingRates.com: How to Retire Before 50

This article by Laura Woods first appeared on GoBankingRates.com and was distributed by the Personal Finance Syndication Network.