These materials have been used in construction for thousands of years, we know how to do ventilation and preventing water from getting there

Fire yes, but we can restrict the spread of fire to reduce the damage by not building like Norwegians