Workers at a New York warehouse voted 55% in favour of joining the Amazon Labor Union.
The group is led by former Amazon worker Chris Smalls, who made his name protesting against safety conditions at the retail giant during the pandemic.
Mr Smalls' victory marks a major defeat for Amazon, which had fiercely fought against unionisation.
However, in Alabama, where Amazon was facing a separate union drive, the company appeared to have fended off activists in a tight contest in which challenged ballots could yet overturn that result.
Together, the two elections mark a milestone for activists, who have long decried labour practices at Amazon, the country's second largest employer.
Mr Smalls emerged from the vote count looking tired but jubilant, and popped open a bottle of champagne he was handed by supporters.
"We did whatever it took to connect with these workers," he told the crowd, recounting an against-the-odds campaign that started with "two tables, two chairs and a tent" and relied on an online fundraiser for money.
"I hope that everybody's paying attention now because a lot of people doubted us."
In a statement, Amazon said it was disappointed by the loss in New York and that it was evaluating how to proceed. It also accused regulators of improperly influencing the vote.
"We believe having a direct relationship with the company is best for our employees," the company said. "We're evaluating our options, including filing objections based on the inappropriate and undue influence by the [National Labor Relations Board]".
Rebecca Givan, professor of labour studies at Rutgers University, said Amazon's defeat by Mr Smalls and his team of worker-organisers was a "really big deal", calling it a "David and Goliath story" that upset the odds.
But she warned he will be facing another tough fight when it comes to contract negotiations.
"Amazon will do everything it can do undo this success, to break up these workers and to try to stop the momentum that will inevitably come from this victory," she said.
Amazon has already poured resources into fighting the unionising efforts, which it sees as an obstacle to business flexibility and warehouse efficiency.
Officials have said the company offers competitive pay and benefits and believes it is better to work directly with workers. In meetings about the vote, representatives questioned union leaders' ability to win more in contract negotiations.
Amazon said it may challenge the results, citing the timing of a decision by the National Labor Relations Board to sue the company last month alleging unfair labour practices in New York.
As part of its statement, it shared comments by two of the country's most powerful business lobbies also raising objections, including a letter from the National Retail Federation to congressional leaders that called for an investigation of the matter.
Mr Smalls said he was undaunted by the challenges ahead.
The 33-year-old had worked for Amazon for more than four years before the company fired him in 2020, after his coronavirus protest, citing quarantine violations.
He established the Amazon Labor Union last year, calling for higher pay, stronger medical benefits, anti-discrimination policies and better leave among other changes.
The 2,654 to 2,131 victory gives his group the right to negotiate a contract with Amazon for the roughly 8,000 workers at the Staten Island warehouse.
His team is also behind a second union campaign at a smaller warehouse in the same industrial park, which is scheduled for a vote later this month.
"There's no doubt in my mind we're going to be successful in that one as well," he said.
Organisers said Amazon made critical mistakes ahead of the vote, such as pushing to reverse policies relaxed under Covid, including allowing workers to keep their mobiles with them while working.
Still they said their win in some ways surprised even them, despite the intense efforts they had made to reach out to workers and make their case ahead of the vote.
"Watching all of this come true is pretty crazy," said Karen Ponce, Amazon worker and secretary of the Amazon Labor Union.
The union drives at Amazon come amid an upswell of labour activism in the US.
In recent months, organisers have led successful campaigns at coffee chain Starbucks, media outlets, smaller retailers and others.
Analysts say the pandemic galvanised workers worried about their safety. The rapid rebound of the economy since the coronavirus shock has also produced an unusually tight labour market, giving workers more leverage.
But across the US, just 6.1% of private sector workers belong to unions.
"It continues to be very difficult for workers to organize collectively," Prof Givan said.
In Alabama, workers at the Bessemer warehouse voted 993 to 875 against joining the Retail, Wholesale and Department Store Union, regulators said on Thursday.
More than 400 challenged ballots have yet to be counted and could change that result.
The unofficial tally showed a much closer election than the lopsided victory that Amazon claimed after last year's election, which regulators later declared invalid, ordering a do-over citing labour rule violations by Amazon.
Turnout was also lower, with less than 40% of the more than 6,000 people working at the Bessemer warehouse casting votes in the election.
Stuart Appelbaum, president of the Retail Wholesale and Department Store Union, which led the unionising drive in Alabama, said he was excited by the possibility of a union victory in New York.
Crediting Mr Smalls and his team, he also said their win in part reflected differences between New York - where one in five workers is unionised - and Alabama, which is a notoriously anti-labour state.
He said his union's initial union effort at the Amazon warehouse in Alabama last year had "ignited a movement".
"Regardless of the final outcome, workers here have shown what is possible," he said at a press conference on Thursday following the Alabama count. "This is a sign of more to come."