Grayback is on Farm Road 1763 just south of its junction with Farm Road 1811, fifteen miles southeast of Vernon in south central Wilbarger County.

Much of the nearby land was bought up by the Waggoner Ranch during the 1880s, when settlement began in the area. Local oil discoveries about 1920 produced a regional boom, prompting oilmen to establish the Grayback post office near Beaver Creek. The community's population reached a reported peak of fifty during the 1950s before leveling off to twenty-five. The Grayback post office was discontinued in the mid-1950s.

This rural community was sometimes referred to as Rock Crossing, a now-extinct oil community located several miles east of Grayback. The Phillips Petroleum Company had a camp at that site and built several houses for employees there in the mid-1920s. A school founded there about 1927 lasted for a short time.

County maps made in the 1980s did not show the Rock Crossing community but indicated two businesses, a church, and a community center at Grayback, where the population was still reported as twenty-five through 2000.

Source: Wilbarger County Historical Commission, Wilbarger County (Lubbock, 1986).