WASHINGTON--The Senate on Monday rejected a procedural motion to advance legislation to approve the Keystone XL pipeline, prolonging the chamber's debate over the project.

Amid a partisan fight over congressional procedure and absences due to weather and other conflicts, Senate Republican leaders couldn't amass the 60 votes needed to end debate on the bill and move toward a final vote on approving the pipeline. The procedural vote was 53-39.

While enough Senate Democrats support the pipeline to clear the 60-vote hurdle, several of them are fuming over a decision last week by Senate Majority Leader Mitch McConnell (R., Ky.) to vote against taking up Democratic amendments and to prevent Democrats from delivering one-minute floor speeches on their amendments.

Their effort to defeat the motion was helped by absences among some members who probably would have voted for it.

The Senate is now expected to continue working on the bill this week, with more amendment votes, and a final passage vote isn't expected until early next week at the earliest. The legislation has 60 co-sponsors, including six Democrats, so once both parties agree on how to move forward on the procedure, it is expected to pass and eventually reach President Barack Obama's desk. The Senate bill is nearly identical to one passed by the House earlier this month.

Keystone XL, a TransCanada Corp. project, would move as many as 830,000 barrels of oil a day from Canada's oil sands to Steele City, Neb., where it would connect with existing pipelines to Gulf Coast refineries. TransCanada first submitted its permit application in 2008 to the State Department, which oversees cross-border pipelines. The pipeline has become a proxy for a much broader debate over the economy and Mr. Obama's efforts to address climate change.

The White House, citing a continuing review process by the State Department over the pipeline, has said Mr. Obama would veto the Senate bill. Neither the House nor the Senate is expected to have enough votes to override such a veto.

The Democratic anger erupted last week over the amendment process. Mr. McConnell had already allowed votes on several Democratic amendments, including votes on climate change that were politically uncomfortable for Republicans.

But Democrats wanted more time to offer additional amendments, and their leaders were successful in rallying even some Keystone supporters to vote against the procedural motion as a protest against Mr. McConnell, saying he wasn't fulfilling his promise to open up the Senate floor to a debate over a broad range of amendments.

Republicans say Democrats had plenty of opportunity to offer their amendments and had taken advantage of it last week with votes on amendments covering a number of energy-related topics.

"This is really disappointing," Mr. McConnell said, saying that Democratic complaints were misleading and that the Senate had already had more amendment votes on one bill than the Senate had all of last year.

Sen. Barbara Boxer (D., Calif.), who opposes the pipeline, said five Democrats who had asked to speak for one minute "were told sit down; we're gagging you."

"This has been anything but an open amendment process," she said on the Senate floor Monday shortly before voting no.

Write to Siobhan Hughes at siobhan.hughes@wsj.com and Amy Harder at amy.harder@wsj.com

Subscribe to WSJ: http://online.wsj.com?mod=djnwires