Vision Based Clothing Analysis And Recommendation In Smart Home

Clothing is one of essential necessities for people. As the aesthetic requirement increases, new materials, styles are introduced into clothing design and production, making the selection and laundry more and more complicated and troubling. While the daily housework oriented Smart Home technology is developing rapidly and bringing convenience to people’s life. Thus, applying intelligence into clothing related activities and reducing the housework burden is not only demanded but also a hotspot for both industry and research. To solve the key problems in clothing oriented intelligent applications, this thesis focuses on the computer vision methods on object extraction, image matching and recommendation, and proposed new models for clothing issues in household scenario. Accordingly, we proposed the approaches in three aspects:1 Visual saliency based clothing area extractionObject extraction in cluttered scene is challenging because of the following reasons. First, the background is cluttered and mixed with foreground. Second, colors and texture of clothes varies enormously, and extracting clothing area with structured principles is difficult. Third, the angles from which the images are shot are random, making the shape irregular or even incomplete. Therefore it is relatively difficult to obtain the objective clothes with certain structure model. Thus a novel approach is proposed in this thesis by combining the visual saliency and background removing method. First, the image is cut into patches with graph-based segmentation, and the saliency of each patch is computed based on an improved saliency definition involving color and location. Then the background is estimated based on GMM and the foreground probability image is employed to refine the saliency of the patches. Finally the patches with the saliency higher than threshold are extracted as clothing area.2 Clothing image matching based on visual feature setsThe surface features of clothing have two characters because of the design. For one thing, the texture in one clothes is not always homogeneous. For another, a clothing piece is consist of local part with semantic information, such as collar, buttons. These characters are distinguishable and crucial for the recognition of clothing image. However, the cognizable features above are lost in the irregular clothing images, because the wrinkles occlude and twist some of the features, which brings challenge to the recognition. Aiming at the above-mentioned difficulties, the cardinal clothing images are introduced for matching with the irregular images, converting the recognition problem into a matching problem. A novel matching model is also proposed, which describes the clothing images as the set of three types of features:color feature, texture features and local semantic features. The local semantic features are extracted from the image segments defined on the image segmentation, and the texture features are extracted from the sample of Gabor texture-based segmentation. Then the image matching rate is calculated following the Gaussian fuction, with the semantic features matching, texture features matching and global color feature matching as input. Finally the matched images are sorted in order of matching rate from high to low.3 Clothing recommendations based on a novel layered feature modelThe clothing recommendation puts forward the clothing image as results to the inqury of input image and semantic description. Feature extraction is the fundamental and key process of image content based recommendation. The high-level descriptions of clothing are complex and boundaries between the descriptions are vague, causing the recommendation based on single-level local or global statistic features fails in the clothing categories or styles. To achieve a description close to the cognition of human visual system, we proposed a four-layer feature model according to the visual process from entirety to locality. In the local semantical layer, the graph-based segmentation is employed to acquire the patches of clothing images. Then texture, geometry and color features of the patches are extracted to describe each patch. The clothing blocks are defined by clustering the patches with K-means method, features of which are combined with global features for describing the clothes. A two-step recommendation based on classifying and matching is realized using the above-mentioned features. Only the clothes of the same high-level semantic description as input clothing image are recommended.Driven by the clothing application in Smart Home, a system model for intelligent service of clothing is proposed in this thesis. The main subjects, clothing area extraction, clothing images matching and clothing recommendation are studied accordingly, and effective models for clothing image analysis and understanding is proposed.