Officials identified the victim as Robert Matysek, 58, of Charleston, SC, reports WBAL TV.
Rescue crews pulled Matysek from the water and performed CPR. He was about a mile into the swim, the station reports.
Officials said Matysek was taken to Anne Arundel Medical Center, where he died.
The event, in its 23rd year, drew 650 participants who attempted to swim the span of the Chesapeake Bay Bridge.
Numerous swimmers were pulled from the water during the event by volunteers on boats, reports The Washington Post.