James Hylton was born on August 26, 1934 to a Pearisburg (Giles County ), Virginia family and was one of thirteen children. Hylton spent his early years in Roanoke Virginia, and his life centered primarily around farming but he soon found himself, like many other southern teenagers, immersed in the world of stock car auto racing. Hylton's career in auto racing began in the late fifties when he began working as a mechanic for the legendary Rex White. James, Rex and Louis Clements teamed to win 26 races and most importantly the 1960 NASCAR Grand National championship. In 1964, White scaled back his driving duties and James began his tenure as crew chief for the Ned Jarrett / Bondy Long team. During the 1964 season the team won 14 races and finished second in points. In 1965, the team won 12 races and won the NASCAR Grand National championship.

On July 8, 1964, Hylton made his first Grand National start at the Old Dominion 400 at Manassas, Virginia. James finished 19 and collected $100 for his efforts. Things improved dramatically in 1966, as Hylton finished second in the points chase and won the coveted NASCAR Rookie of the Year award. James also captured his first pole at Starlite Speedway in Monroe, NC. Hylton again finished second in points during the 1967 season while driving Dodge's for owner Bud Hartje. James was a model of consistency during this two year period as he had 46 top five finishes in 87 races.

In 1968, James became a car owner / driver, a dual role that continues to this day. James found his way to victory lane for the first time on March 1, 1970 at the Richmond 500, driving the familiar number 48 Ford. During the late sixties and early seventies, Hylton amassed an amazing consistency record that was rivaled only by those of Richard Petty and Cale Yarbrough.

On August 6, 1972, James forever etched his name in the history books by claiming the Talladega 500. Hylton led 106 laps of the 188 lap race and won $ 24,865 for the day. Hylton won by one car length over ARCA legend Ramo Stott.
Hylton continued driving the full NASCAR schedule until the 1982 season. James soldiered on as an owner in the NASCAR Winston Cup Series until 1993. In the nineties, he moved to the ARCA circuit where he finished 18th in the 2006 ARCA RE/MAX Series Standings.