‘What can a man do?’ The legacy of James F. Brown at Mount Gulian
In a wooded neighborhood less than a half mile off Route 9D in Beacon, the Mount Gulian Historic Site preserves an unusual legacy: the work of James F. Brown, one of the country’s first Black master gardeners.
Brown was born as Anthony Chase or Anthony Fisher into enslavement in 1793. Growing up in post-revolutionary War Maryland, he was denied an education, although he learned to read and write probably through the churches of the region, said Myra Young Armstead, the Lyford Paterson Edwards and Helen Gray Edwards Professor of Historical Studies at Bard College and author of “Freedom Gardener: James F. Brown, Horticulture, and the Hudson Valley in Antebellum America.”
Brown worked primarily as a “quasi-free” wage laborer and eventually saved enough money to purchase the freedom of his wife, Julia. When his enslaver, Susan Williams, reneged on a deal to manumit him in 1827, the couple fled — but only after James left behind a note promising payment for his freedom.
“What can a man do who has his hands bound and his feet fettered?” he wrote. “It will certainly get them loose in any way that he may think the most advisable.”
Escape to the North
The couple resurfaced in lower Manhattan, where Brown found employment as a coachman and waiter with the Verplanck family, wealthy Dutch merchants and New York landowners. While the Verplancks were not abolitionists, they “had an evolution in their thinking ” and emancipated their enslaved servants, according to Elaine Hayes, executive director of Mount Gulian.
Brown’s newfound freedom was nearly derailed by a dinner guest from Maryland who recognized him and reported him back to Williams. But in an exchange of letters between Daniel Crommelin Verplanck, a congressman and family patriarch, and Williams, it’s clear Brown’s former enslaver “wanted to be done with the business of chasing (him),” said Armstead. The congressman purchased Brown’s freedom for $300, and he became a valued employee for almost a halfcentury.
By 1829, the Browns relocated to Beacon, where the Verplancks had built Mount Gulian, their summer estate. He became the manor’s coachman and gardener, and began keeping what would become a 10-volume diary documenting nearly 40 years of his everyday existence in the Hudson Valley.
The evolution of a writer and master gardener
The journals are hardly ripping yarns — those are more commonly found among slave narratives, a literary genre that rose from Northern abolitionist societies, who discovered that first-person retellings of the enslaved experience proved persuasive in winning sympathizers.
But Brown’s diaries, which are available online through the New York Historical Society, provide rare firsthand documentation of Black life during the early years of emancipation — a painfully gradual process, even in the North, where some landowners still enslaved people into the 1840s.
Brown’s diaries cover a range of minutiae, from the weather to chores, trips to church and around the region, and local and national happenings gleaned from newspapers. While he didn’t offer his own commentary on current events, language choices give clues to his state of mind. For example, when radical abolitionist John Brown was executed in West Virginia in 1859, James Brown refers to him in his diary as “our hero John Brown.”
Brown also chronicled his maintenance of the 8 acres of formal English and agricultural gardens at Mount Gulian, where he grew vegetables, flowers, fruit trees and shrubs, as well as 140 rose bushes.
The entries show Brown’s evolution into a master gardener, a designation that to this day represents in-depth training in and knowledge of horticulture. Mount Gulian’s location along the Hudson River afforded him the opportunity to work with a variety of early American landscape designers, including the renowned Andrew Jackson Downing, who established a botanical garden and nursery in Newburgh. Brown even exhibited his own plants at botanical shows, a rarity for African Americans, who weren’t permitted among the membership in most horticultural societies.
Along with Brown’s many gardening entries are hints of the toll of nearly a half-century of manual labor. At various points in time, he mentions a back injury incurred while moving lumber, cuts on his hands from pruning and an ankle sprain severe enough to require surgery.
Social and civic advocacy
Still, Brown recorded a subtle pride in the dignity of his work and his position in Hudson Valley society. As his profile rose, he became involved in several local initiatives.
After the passage of a law excluding free Black men from voting in 1821, Brown advocated for land ownership among African Americans and the subsequent right to vote. “It meant a lot to him, so much that he memorialized it in his journals,” said Hayes, noting that Brown began recording each exercise of his civic duty beginning in 1837. “I think he was asserting the fact that he was a first-class citizen, and felt it was important to carry out the rights and responsibilities of one.”
In the 1850s, Brown spearheaded the antebellum equivalent of a crowdfunding campaign to purchase a 50-by-130foot land parcel to establish the Union Burial Ground, a cemetery for people of color in the southern portion of the Methodist Cemetery on North Walnut Street.
While researchers don’t know exactly how many people are buried there, it contains several headstones of Black Civil War veterans.
Brown’s diaries end in 1866, two years before his death at age 74.
He was buried not in the cemetery he helped establish but across town at Saint Luke’s, his primary church. While no material vestiges of his life exist at Mount Gulian, public tours — from mid-april through late October — interpret his relationship to the Verplancks, his life and his work among the fruits and flowers.
Following the destruction of the site by arson in 1931, the Mount Gulian Society formed to build a replica of the original Dutch manor house and reestablish its formal garden — using a contemporaneous Verplanck garden map and Brown’s journals to determine the location and types of plantings. Some of the original heirloom plants, including roses and pansies, have been restored, thanks to a Verplanck descendant who used to visit the property in the 1960s and took several plants back to her home in Connecticut.
Beyond its physical beauty and serenity, Mount Gulian provides a vital link to a resilient and talented man who achieved extraordinary career and personal success, despite overwhelming racist sentiment.
“In the period before the Civil War, freedom in the most obvious sense for a runaway meant emancipation,” said Armstead. “It also meant freedom from wage slavery, and freedom to operate in the civic sphere. We can explore the many meanings of freedom in the antebellum period through James’s diary.”