By Paul Russinoff
Those who have studied President Abraham Lincoln and his administration will immediately recognize the face of the man seated on the left in this portrait. He is John Milton Hay, Lincoln’s witty, confident and undeniably brilliant personal secretary.
Though the face may be familiar, the attire is not. In the handful of wartime photographs of Hay, he is nattily dressed in civilian clothes. But here, in this small, trimmed albumen print, he sports a bandbox fresh officer’s uniform complete with the shoulder straps of a major.
The print is the only known photograph of Hay in Union blue. In it, he sits with Lt. Col. Edward W. Smith. The portrait is a memento of a friendship. It is also a unique visual artifact connected to a highly sensitive diplomatic effort.
Hay was, in fact, an officer. His commission was rushed through Congress in early 1864. On January 11, Secretary of War Edwin M. Stanton nominated him as major and assistant adjutant general. The Senate approved the request the next day, and 25-year-old Hay accepted the commission on January 13.
Stanton arranged for the commission at the request of Lincoln, who had tapped Hay for a high stakes mission. About a month earlier, on Dec. 8, 1863, the president issued an executive order with far-reaching potential. The Proclamation of Amnesty and Reconstruction allowed a seceded state to reenter the Union if 10 percent of its voters, as calculated in the 1860 election, signed a loyalty oath to the federal government. The state’s citizens would receive a full restoration of rights and property, with the notable exception of slaves.
Lincoln’s “Ten-Percent Plan” was also a compromise to opposing policy agendas by the Radical Republicans and Peace Democrats.
Florida was targeted as the ideal place to begin. Only 1,435 signatures, 10 percent of its voters, would be necessary to fulfill the basic requirements for readmission. Lincoln tasked Hay with a trip to the state to put the plan in effect. The president also deemed it necessary that Hay should be commissioned an officer to give him authority as he navigated Union military bureaucracy along the way. To further bolster his commission, Hay was appointed as an aide-de-camp on the staff of Maj. Gen. Quincy A. Gilmore, who commanded the Army of the South and was in charge of operations in Florida.
On January 15, barely 48 hours after Hay signed his commission paper, he departed by sea for Hilton Head, S.C. Hay arrived four days later and immediately met Lt. Col. Smith. The two men had encountered one another the year before. In mid-1863, Lincoln had dispatched Hay to Florida—his first trip to the state. Here, he became acquainted with Smith, who at the time served as assistant adjutant general to Maj. Gen. David Hunter. Smith became something of a professional and social companion to Hay during the two-month tour. Hay mentioned Smith several times in his diary, including a reference to being entertained by his rendition of a number of Hymns one evening.
The meeting of the two men in Hilton Head began as a reunion, but soon turned to military matters. Smith, now on the staff of Maj. Gen. Gilmore, escorted Hay to his commander to present his orders. Gilmore, as Hay recalled in a letter to fellow secretary John Nicolay, “held the President’s letter in a helpless sort of way, like a bachelor with a baby: he didn’t know what to do with me: had a vague idea that he was to split his army up into squads and watch the polls while I carried on the election: a confused notion that he was to drop everything for the next month or two and shoulder-strike through the Everglades.”
Hay reassured Gilmore. “I wanted no soldaten—only orders. These I will get when I want them,” he wrote the German-born Nicolay. The meeting concluded and Hay bade his time in Hilton Head while details for his Florida trip were finalized.
His time was eventful. Hay came under fire by Confederate artillery as he watched the siege of Charleston grind on from the relative safety of Battery Wagner, which, by this time, had fallen into Union hands. At one point, an enemy shell landed in close proximity and forced Hay to make what he termed a “bad dodge.” He also observed the grim effects of shrapnel on soldiers after the shell exploded.
On January 21, Hay wrote Nicolay, “Last night was a glorious summer night. I got arrested and taken to the Guardhouse for being out in the moon in citizen’s clothes. Why does my tailor so delay? For tis their nature to.”
The reference implies that Hay had dressed in street clothes all along. Considering the whirlwind of activity—only eight days had passed since he accepted his commission—it perhaps comes as no surprise that he was out of uniform. Yet, the morning after he recorded his arrest in the diary, he sat for this portrait with Lt. Col. Smith in a Hilton Head photograph gallery. Though the photographer was not identified, a likely candidate is Sam A. Cooley, who operated in the area during this time.
By March 10, 1864, Hay had returned to Washington having little to show for the last two months. The Ten-Percent Plan and the Florida expedition were acknowledged as disasters.
How Hay came to be pictured in uniform, only hours after his arrest, may be explained by various scenarios. He might have ordered the blue suit before he left Washington and it serendipitously arrived in Hilton Head on the morning he sat for his portrait. It is also possible that he purchased the uniform off the rack from a Hilton Head sutler and had minor alterations made by a local tailor. The arrest may have prompted Hay to press the tardy tailor to finish the job.
But the evidence in the portrait suggests another scenario. Renowned collector Mike Cunningham suggests Hay borrowed a uniform from Smith, who appears to be about the same size and fit. Smith and most field grade officers, Cunningham notes, “routinely had two or three uniforms in their baggage.” Cunningham also observes that the two men wear matching trousers and alternate coats and hats: Hay is dressed in a frock coat and holds a slouch hat. Smith is attired in a fatigue blouse and rests a hand on a cap.
If Smith loaned the uniform, it is reasonable to assume Hay returned it when he received his from the tailor. Hay was silent on the subject in his diary and letters.
About this time, Smith informed Hay that printed copies of Lincoln’s executive order had been distributed to Confederate pickets in Florida.
Hay himself arrived in the northern Florida town of Fernandina on February 5 to lead political ground operations. He described his first day on the job in his diary. “Opened my book [of loyalty pledges] in an office over Robinson’s store, sent out my [proclamation] posters & sat like a spider. A few straggled in and swore [allegiance]. One hesitating cuss who evidently feared he was going to be tricked into the army swore, but dallied so on the signing that I shut the book & I told him to make up his mind before calling again.” On it went.
Gilmore set a military expedition in motion. He and a full division of troops arrived in Florida intent on the occupation of Jacksonville, the disruption of cattle supplies to the Confederates and the recruitment of slaves, made free by the Emancipation Proclamation, for colored regiments.
Back in Fernandina, Hay made little progress. After Union forces occupied Jacksonville, Hay moved his signature-gathering effort there. He had the same meager results.
Meanwhile, Gilmore returned to South Carolina and left command in the hands of Truman Seymour, a West Pointer who had led the failed assault at Fort Wagner the previous summer. He had received a serious wound in the engagement. His troops suffered severely—most notably Col. Robert Gould Shaw and his 54th Massachusetts Infantry, which led the ill-fated attack and famously proved that African-American soldiers could fight.
The survivors of the 54th, commanded now by Col. Edward N. Hallowell after Shaw’s death at Wagner, were in Jacksonville. Seymour struck out against orders with the 54th and the rest of his command for the Florida capital of Tallahassee. On February 20, Seymour ran into stiff resistance 50 miles west along the rail lines at Olustee Station. In the engagement that ensued, the only major battle in Florida, the federals suffered a decisive defeat.
By March 10, 1864, Hay had returned to Washington having little to show for the last two months. The Ten-Percent Plan and the Florida expedition were acknowledged as disasters.
Hay packed away his uniform and resumed his role of secretary and confidant to the president. He remained with Lincoln until his assassination 13 months later and went on to an extraordinary highly visible career of public service to four more presidents. (Assassin bullets felled two of the four presidents, James A. Garfield and William McKinley.) Hay ended his career as President Theodore Roosevelt’s Secretary of State.
Hay is perhaps best remembered for his landmark 10-volume Abraham Lincoln: A History, co-authored with John Nicolay and published in 1890. Celebrated across the globe for his many achievements, Hay died of heart disease at age 66. His wife, Clara, and three adult children survived him.
Smith, six years senior to the illustrious Hay, lived a life largely away from the spotlight. After the events in Hilton Head and Florida, including the portrait sitting, no evidence exists that the two men crossed paths again.
The son of a prominent minister and onetime president of the University of Vermont, Edward Worthington Smith began his college education under his father’s leadership. At the start of the Civil War, he was a 28-year-old lawyer with a practice in Chicago. In early 1861, he enlisted as a private in the 16th Illinois Infantry, but left the regiment a short time later to accept a first lieutenancy in the 15th U.S. Infantry. He became as aide de camp to Maj. Gen. David Hunter in August 1861. This promotion marked the beginning of a long career as a staff officer.
A talented administrator, Smith brought his considerable abilities into play as assistant adjutant general at the top levels of military command. When he sat for his portrait alongside Hay, he ranked as a lieutenant colonel of volunteers and assistant adjutant general of the 10th Corps. He ended the war as assistant adjutant general of the entire Army of the James and the Department of Virginia and North Carolina, and a brevet brigadier general.
Smith mustered out of the volunteer army in July 1866 and continued in the regular army at posts in the South, and in Western territories and states. In Atlanta, he met and married Kate Morgan Adams in 1871. Their daughter, Sybil Worthington Smith, was born two years later.
By 1876, Smith served as assistant adjutant general to Brig. Gen. Alfred H. Terry. In this role, Smith recorded and delivered to Lt. Col. George Armstrong Custer the fateful orders to attack Sitting Bull at the Little Big Horn. After Custer’s death, Smith helped Terry shape the narrative of the defeat, and worked with newspaper reporters and other investigators.
In a small way, Smith shared a literary dimension with Hay. Smith and Hay, along with other noted officers, were mentioned in the Life and Adventures, Songs, Services, and Speeches of Private Miles O’Reilly, 47th Regiment New York Volunteers, a popular book penned by his friend and fellow staffer Col. Charles G. Halpine. Smith was also an active genealogist, and, in 1878, penned and published a brief history of his paternal and maternal ancestors.
Smith went on to become major of the 22nd U.S. Infantry, and served in this capacity when, in early 1883, he experienced an intestinal issue diagnosed as gastritis, and suddenly died in May, after suffering a massive hemorrhage of the bowels. Smith was 50 years old. His remains were sent to his birth state of Vermont and buried in Greenwood Cemetery in St. Albans. A modest headstone marks his gravesite.
General Terry, on whose staff he served, provided perhaps the most fitting eulogy in a letter of recommendation written before Smith’s death. In it, Terry praised him for “a singularly long and brilliant record of service in the staff corps (to which) he adds high education, great intelligence and an unspotted reputation.”
Smith’s widow, Kate, filed a pension claim for her benefits and those for their now 10-year-old daughter. Unbeknownst to her, the post surgeon had determined Smith’s cause of death as chronic alcoholism. The pension was denied, despite no evidence that drinking impaired his service and the testimony of many fellow officers. She fought to overturn the rejection for 13 years, at which time her name disappeared from government records.
Evidence suggests this photo of her late husband and Hay belonged to the Smith family. It is possible that Kate owned the image and passed it to her daughter, who died in 1945. The picture became separated from the family at some point and landed in the collection of Robert Trownsell. The image, and many other artifacts from the Trownsell collection, was sold by Cowan’s Auctions in 2014.
References: Michael Burlingame, Inside Lincoln’s White House: The Complete Civil War Diary of John Hay; Michael Burlingame, With Lincoln in the White House: Letters, Memoranda, and other Writings of John G. Nicolay, 1860-1865; John Taliaferro, All the Great Prizes: The Life of John Hay, from Lincoln to Roosevelt; Email from Michael Cunningham, June 2, 2017; Letters Received by Commission Branch, 1863-1870, National Archives, Washington, D.C.; John M. Hay military service record, National Archives, Washington, D.C.; Edward W. Smith military service record and pension file, National Archives, Washington, D.C.; Charles G. Halpine, Life and Adventures, Songs, Services, and Speeches of Private Miles O’Reilly, 47th Regiment New York Volunteers; Edward W. Smith, A Genealogical Sketch of the Families of Rev. Worthington Smith, D.D. and Mrs. Mary Ann (Little) Smith, of St. Albans, Vermont; James Robinson, The Real Custer: From Boy General to Tragic Hero; United States Army and Navy Journal, Vol. XVI, No 22, Jan. 4, 1879.
Paul Russinoff of Baltimore, Md., has been a passionate collector and researcher of photographs from the Civil War since elementary school. A subscriber to MI since its inception, representative images from his collection appeared in the Autumn 2014 issue.
SPREAD THE WORD: We encourage you to share this story on social media and elsewhere to educate and raise awareness. If you wish to use any image on this page for another purpose, please request permission.
LEARN MORE about Military Images, America’s only magazine dedicated to showcasing, interpreting and preserving Civil War portrait photography.
VISIT OUR STORE to subscribe, renew a subscription, and more.