One should not be lenient when the spiritual master is offended and one should immediately give up such bad association. The more we associate with the devotees the less are the chances that we become offenders by associating with offenders. Srila Prabhupada explains:
"One should be extremely careful about associating with persons who are not respectful to the higher authorities. It is instructed, therefore, in the Vedic literature that one should always be free from the association of atheists and nondevotees and should try to associate with devotees, for by the association of a devotee one can be elevated to the platform of self-realization. This injunction is stressed in many places in Srimad-Bhagavatam; if one wants to be liberated from the clutches of material existence, then one has to associate with great souls, and if one wants to continue one's material existential life, then one may associate with persons who are materialistic. The materialistic way of life is based on sex life. Thus both becoming addicted to sex life and associating with persons who are addicted to sex life are condemned in the Vedic literature because such association will simply interfere with one's spiritual progress. However, association with great personalities, devotees who are great souls, will elevate one to the spiritual platform."
SB 4.4.26