By Gregor Stuart Hunter 

Stocks in Hong Kong tumbled after Greeks rejected bailout terms, while China's shares stabilized amid Beijing's efforts to prop up markets.

Hong Kong's Hang Seng Index shed 3.2% Monday, its worst one-day performance since 2012. The index is down 11.3% since its April high, entering correction territory, defined as a drop of more than 10%. A gauge of Hong Kong-listed Chinese companies, known as H-shares, is down 3.6%. Hong Kong had largely avoided the roller-coaster trading that wiped out about $2.4 trillion in value from Chinese shares during a three-week decline.

"Given the 'No' vote in the Greek referendum and the latest developments in the mainland A-shares market, there may be increased volatility in the Hong Kong markets," said the Hong Kong Monetary Authority in a statement Monday. "The banking system in Hong Kong is highly liquid and is well equipped to handle any such volatility. The HKMA stands ready to provide liquidity support to the banking system should it become necessary to do so."

Investors whisked the most capital out of China via the Stock Connect than on any day since the trading link between Hong Kong and Shanghai launched in November. A total of 12.5 billion yuan ($2.01 billion) was withdrawn over the course of the trading day.

China's indexes also cooled after an initial spike earlier Monday. The Shanghai Composite ended up 2.4% compared with earlier gains of as much as 7.8%. The smaller Shenzhen market is down 2.7% and the ChiNext board, composed of small-cap stocks, is down 4.3%. All indexes still are off more than a quarter from highs reached in June.

Chinese officials have turned to an array of tools to prop up the market in recent days: from encouraging stock buying with borrowed money to rallying state-affiliated firms to invest. Now, China's central bank indirectly will help investors borrow to buy shares and regulators over the weekend also agreed to halt all new initial public offerings.

"The intervention of [the People's Bank of China] is unprecedented," said Li Bin, a Shanghai-based analyst at Capital Securities, and shows "the government is highly concerned about potential market stampede caused by margin calls." While margin trading can amplify returns on the upswing, losses can pile up quickly when investors are forced to sell holdings to pay back brokerages from whom they have borrowed.

Some say the measures won't have much staying power. "These policies are aimed at stabilizing market confidence for the short-term, but still fall short of expectation to push the market to a higher level," said Jacky Zhang, an analyst at BOC International. "Investors may remain cautious about long-term investment."

Other brokerages are more hopeful. Regulators have plenty of options at hand to stabilize the Chinese market, and the unwinding of margin positions could encourage more risk taking in the future, said HSBC.

Regulators are "committed" to preventing further falls in mainland A-shares, and "more favorable policies are expected to be rolled out to stabilize the market if volatility remains high," analysts from the bank wrote in a research report. "We estimate that the worst of deleveraging and forced selling in the A-share market could be behind us."

Moreover, a big source of liquidity hasn't yet been tapped for stock investments, said analysts at Bernstein Research. Liquidity from wealth-management products and money-market funds, rather than bank deposits, drove much of the earlier rally. "This is good news for the broader market, as the equity market rally has yet to tap into the largest liquidity pool in the system, i.e. bank deposits, so future liquidity supply is not yet a constraint," they say.

Late Sunday, the top securities regulator said the People's Bank of China would "provide liquidity assistance" to China Securities Finance Corp., a company owned by the stock regulator. The company will use the money to lend to brokerages, which could then make loans to investors to buy stocks. It marks the first time central-bank funds will be directed to institutions other than banks.

Earlier in the weekend, China's big state-controlled securities firms, mutual funds and a unit of China's giant sovereign-wealth fund also pledged to buy shares. The Securities Association said that 21 brokerages pledged to try to increase investments in the stock market as long as the Shanghai Composite Index stays below 4,500.

Investors elsewhere in Asia braced for bumpy trading. In Malaysia, political pressures have helped push the ringgit to its weakest level against the U.S. dollar since September 1998. The attorney general said an official investigation into a troubled state investment fund has uncovered documents related to allegations that money was transferred into the personal bank accounts of Prime Minister Najib Razak. The Wall Street Journal reported on Friday that Malaysian government investigators looking into the activities of 1Malaysia Development Bhd., or 1MDB, had traced almost $700 million in deposits into what they believe are Mr. Najib's personal accounts. Mr. Najib has denied wrongdoing. The ringgit hit 3.8080 on Monday, and is Asia's worst-performing currency.

Asian shares also are lower after results of Greece's referendum Sunday show a victory for the "no" campaign, which rejected austerity policies set out by the eurozone and the International Monetary Fund. Creditors have said the outcome imperils future compromise and puts Greece closer to leaving the currency bloc.

Japan's Nikkei 225 Stock Average shed 2.1% while Australia's S&P/ASX 200 was down 1.1%. South Korea's Kospi was down 2.2%.

The euro sank 0.6% to $1.1053 against the U.S. dollar and fell 0.8% against the Japanese yen as investors sought safer assets. The yen also rose 0.2% against the dollar. Gold prices rose 0.2% to $1,166.20 per troy ounce, while Brent crude futures dropped 1% to $59.69.

Yifan Xie, Bradford Frischkorn and Grace Zhu contributed to this article.

Write to Gregor Stuart Hunter at gregor.hunter@wsj.com