Although we are not these bodies, we base our whole life on bodily relationships centered around sex attractions. Srila Prabhupada comments:
"Attachment to women is so contaminating that one becomes attached to the condition of material life not only by the association of women but by the contaminated association of persons who are too attached to them. There are many reasons for our conditional life in the material world, but the topmost of all such causes is the association of women, as will be confirmed in the following stanzas.
In Kali-yuga, association with women is very strong. In every step of life, there is association with women. If a person goes to purchase something, the advertisements are full of pictures of women. The physiological attraction for women is very great, and therefore people are very slack in spiritual understanding. The Vedic civilization, being based on spiritual understanding, arranges association with women very cautiously. Out of the four social divisions, the members of the first order (namely brahmacarya), the third order (vanaprastha) and the fourth order (sannyasa) are strictly prohibited from female association. Only in one order, the householder, is there license to mix with women under restricted conditions. In other words, attraction for woman's association is the cause of the material conditional life, and anyone interested in being freed from this conditional life must detach himself from the association of women."
SB 3.31.36 purport