Nefilim, Refaim, Anakim: Biblical Giants
The basis for the tradition of a race of giants is Gen. 6:4, the same locus classicus for the tradition of fallen angels. The angels, as you may remember, came down and took wives among mortals. The giants were either around at the time of those events, or the offspring of those unions (the phrasing of the Hebrew in Gen. 6 is ambiguous): "The nefilim [possibly meaning 'fallen ones,' it's debatable] were on the earth in those days...," and may or may not be the basis of the stories about mythological heroes "....these were the gibborim [heroes, superheroes, demigods?] of old, the men of renown." However under-defined the language of Genesis, the earliest translation of the Bible, the Septuagint, assumes Genesis is talking about gargantuanism, translating both nefilim and gibborim as "giants."
Variously known as the Nefilim [often spelled "Nephilim"] (Num. 13:32-33), Refaim (II Sam. 21:16-22), Emim (Deut. 2:1o-11, 21), Refadim (Ibid. - though this may be a simple scribal misspelling of "Refaim") or the "Children of Anak" (Deut. 9:2), the giants were prevalent enough that the spies saw them throughout their scouting of Canaan, "...we saw men of giant stature...we seemed to ourselves like grasshoppers... (Num. 13:32) and Moses repeatedly alludes to them in Deuteronomy, most notably in reference to the massive King Og of Bashan, who approached 15 feet in height (Deut. 3: 11): "Only King Og of Bashan was left remaining of the Refaim." Apparently this was not a wholly accurate report. Hundreds of years later, David's soldiers had to kill a number of them (II Samuel 21; I Chron. 20:4-8). Perhaps the different names signified different clans within the ethnos of giants.
Numerous as they are vague, these passages would be the basis for a very elaborate complex of legends about giants in post-Biblical Jewish literature.
To learn more, read the entries Giants; Goliath; Og, in the Encyclopedia of Jewish Myth, Magic, and Mysticism.