The virtual world is currently limited in the graphic representation and visualization of material designs. The limitation is in part due to the limited range and the relative simplicity of spectral properties of the materials simulated in a virtual environment. Our work starts with a fast classification and retrieval for handling the large numbers of multi-scale texture samples of complex deformable materials, such as, woven and knitted fabrics. We have developed a general system that can also serve as a unified platform for other material analysis and classification and a 3D panel design system for 3D clothing modeling and draping. Multi-scale color theme indexing for image acquisition and retrieval can be more intuitively supported by a multi-touch gesture interface that is now the preferred mode of interacting with tablets, screens, and other visual communication devices. These also add to the collaborative modes of input and retrieval in fabric and fashion design for virtual agents. In this paper we describe the process of texture analysis of real fabric materials and preliminary models for the 3D texture generation for virtual clothing, color theme design, and ultimately 3D draping based on robust collision detection methods.