The Water-Futures project is an interdisciplinary project bringing together researches from machine learning, engineering and economics working on the next generation of urban drinking water systems. Among others, one objective is to develop Explainable Machine Learning models in non-stationary environments for complex structured and networked data to seamlessly support human decision making for smart water systems by data-driven technologies.