Engineering machine learning applications is different from engineering traditional rule-based software. We do not specify the rules ourselves, but instead deploy self-learning algorithms that discover the rules from data. So we need mechanism to explicit these learned rules. This is useful for end users affected by the algorithms but also for engineers that need to debug the algorithms. This presentation focusses on the latter. I will explain the challenges that engineers face when engineering machine learning applications and present tools and solutions available to face those challenges.