Responsive environment uses human–computer interface to improve how humans experience their surroundings. Many research aimed at different kinds of interactive environment modules with new digital tectonics or computation components. However, those new environments sometimes could be manipulated by components which are less user-friendly and complex than traditional counterparts. In this article, we implemented a real responsive interface – the Spark Wall system, which use 160 actuator modules as our responsive feedback interface and depth camera as sensing input. We built up multi-modal interface for different operating purposes allowing the user to control responsive environment with their behaviour. The user could change his or her body posture to change the pattern of the wall and moreover define touch-input area on any surface. From the user’s perspective, a responsive environment should be a simple system with understandable control modes. A responsive artefact should also be able to dynamically correspond to different methods of operation according to the user’s intentions.