• Significance: Governors Island derives it significance largely from its role as a major component. in the defense system of one of the country's most important harbors, New York Harbor. Defense works on the island can be dated with certainity as early as 1776. A major earthen work was constructed in the 1790s and later demolished. The present fortification, Fort Jay, a square four bastioned fort, and Castle Williams (building #501), a circular casemated work, erected under the Second American System of fortifications in the years preceding the War of 1812, were two of the harbor's largest defense works. Both structures, are fine examples of their types and exhibit excellent design. They are among an important and innovative period of American military construction. Both structures are on the National Register of Historic Places.

The island is also significant for its role as a major Army administrative center for nearly a century. Beginning in 1878 when the Military Division of the Atlantic and the Department of the East were transferred to the island, the post served as headquarters for major Army commands until 1966. The last of these was the First United States Army, one of the country's most important field armies and the only one to fight in both WWI and WWII. The First Army was headquartered on Governors Island from 1933 to 1943 and again from 1946 to 1966 when the First Army headquarters was moved to Fort Meade, Maryland under a major Army reorganization. It is not headquarters for both the Third Coast Guard District and the Atlantic Area and is the largest Coast Guard base in the world.

• Unprocessed Field note material exists for this structure: FN-365
• Survey number: HABS NY-5715