A construction worker fell about 40 stories to his death yesterday at a skyscraper being built by the developer of the World Trade Center, authorities said.

The victim, identified as Anthony Esposito, a rigger on a crew dismantling a crane, fell either from the crane or a 20-foot, movable walkway linking it to the glass-walled midtown Manhattan skyscraper, said Deputy Fire Chief Anthony DeVita.

An impromptu memorial service was held inside the fenced construction lot where Esposito landed, apparently killed instantly.

"This happens. We live with that," said stoic ironworker Luis Guzman, who had been getting his tools together to head to the roof when Esposito fell nearby. Another ironworker, Jim Bradfield, said it appeared that Esposito had on a safety harness.

The accident happened at 600 W. 42nd St., where trade center developer Larry Silverstein is building two 60-story luxury apartment buildings. Construction began last fall on the development, called Silver Towers at River Place.

The general contractor, Gotham Construction Co. LLC, identified the victim as an employee of DFC Structures Inc., a subcontractor.

A woman answering the phone at DFC Structures said only that "everybody is grieving." Gotham said that "everyone is cooperating" with government agencies investigating the accident.

"We are deeply saddened by this tragic accident, and all our thoughts and prayers are with the family and friends of Anthony Esposito," it said in a statement.

The Department of Buildings halted work at both towers while the accident is investigated but allowed the crane dismantling to continue to a safe point. DeVita said there were no structural problems with the crane or the tower.

More than 20 people have died in city construction accidents this year, including nine in two crane collapses. A window worker died last week after falling 12 stories from a Manhattan building.

The city's buildings commissioner was replaced and dozens of new construction safety rules created after the spate of accidents.