Educated at Maesteg Grammar School, Williams was an talented track athlete, representing Wales in the Commonwealth Games in Edinburgh in 1970 and becoming Welsh sprint champion in 1971. He initially played for Bridgend, joining Llanelli RFC in 1972, and gaining his first international cap in 1973. He was rated as one of the fastest wingers in the game, and scored 12 tries in 30 appearances for Wales. He went on two British and Irish Lions tours, playing in all four tests in South Africa in 1974 and in three tests in New Zealand in 1977. He played a major role in the 1974 'invincible' series against South Africa, scoring two tries in each of the second and third tests and earning the title "The Welsh Whippet".
Williams now runs J. J. Williams Painting Services Ltd, a commercial and industrial painters based in Pyle, near Bridgend. He also fronted a consortium which offered to take over the Millennium Stadium in Cardiff. All three of his children have represented Wales at Track and Field events. His son Rhys was the Welsh track record holder in the 400 metres hurdles (49.09) in 2005.