I have had casual flings with some colleagues in the past and I would say that it is not advisable. That's because it tends to breed jealousy especially if you tend to relate with other colleagues of the opposite gender.