I believe you have valid concerns and you should not allow your daughter to go over there until you know for a fact it is more stable. Maybe invite them to your home but not the other way around.
Your daughter does not need to understand in order for you to put your foot down. It is okay to protect her. My girl gave me many grey hairs and was tougher to raise than my boys. Be strong, push back, that's what she really needs.