The iconic US motorcycle brand is stopping manufacturing and massively scaling back its sales operations.
Harley's decision comes weeks after Toyota said it wouldn't expand further in India due to the country's high tax regime.
The exit is a blow for Indian Prime Minister Narendra Modi's efforts to lure or retain foreign manufacturers.
Harley's departure involves $75m (£59m) in restructuring costs, around 70 redundancies and the closure of its Bawal plant in northern India.
The plant was opened in 2011 but Harley-Davidson has struggled to compete with local brand Hero as well as Japan's Honda.
About 17 million motorcycles and scooters are sold each year in India.
While it is cheaper than many other developing economies, India has proven a tough market to crack for foreign automakers.
General Motors pulled out of the country in 2017 while Ford agreed last year to move most of its assets into a joint venture with Indian vehicle giant Mahindra & Mahindra.
US President Donald Trump has previously complained about India's high taxes, specifically mentioning the levies placed on Harley-Davidson bikes.
India's import tariffs were slashed by 50% but the brand has still struggled in the competitive market.
Harley has also been suffering its own problems and recorded its first quarterly loss in more than a decade between April and June this year.
It has been cutting hundreds of jobs under its new chief executive Jochen Zeitz and focusing on core markets and models.
By Nikhil Inamdar, BBC News' India business correspondent
Harley Davidson came into India with much fanfare a decade ago. But it has since, struggled to find a foothold in one of the world's most lucrative two-wheeler markets.
With sales averaging under 3,000 units every year, the iconic American brand simply couldn't capitalize on the big Asia opportunity it was betting on.
Auto experts put the blame on the brand's failure to drive up volumes and derive cost efficiencies by leveraging local tie-ups.
This, coupled with prohibitively high taxes, a slowdown in discretionary spending and an inability to compete with Indian brands may have prompted the decision to call it quits.
"India is a high volume, low margin market. They weren't structured to play that game, being at the very pointy end of the pyramid," Hormazd Sorabjee, Editor of Autocar India told the BBC.
"The lifestyle element that goes with owning a Harley bike is also not fully developed in India yet."
Incidentally the bike maker has had a better run in other Asian markets like Thailand and Korea, precisely because the market and cost structures are more favourable.
Harley's exit may not have a significant economic impact, but is bad optics for the Modi government, which is preparing to roll out a $23bn package to lure global manufacturers to set up base in India as part of the country's 'Make In India' policy.
High import tariffs on Harley have also been a flashpoint in India's trade negotiations with the US.
Mr Trump has previously cited the example of Harley to call India a "tariff king", and its decision to exit the Indian market could well set off another diplomatic tussle with the US with whom India is negotiating a free trade agreement.
The iconic US motorcycle brand was founded in 1903 and has built a very loyal customer base. It has owners' clubs all over the world.
It hit the global stage in 1969 thanks to the classic road movie Easy Rider starring Dennis Hopper, Peter Fonda and Jack Nicholson.
Its bikes, nicknamed "hogs", are also made in factories in the US, Brazil and Thailand.
Harley has been looking to grow the brand beyond baby boomers in the US, with smaller models and all-electric versions.
Don't be afraid to give up the good to go for the great.