Color recognition of design object of manual decoration element based on convolution neural network under the impact of COVID-19

2.1Inception module

The main idea of the concept module is to find a local sparse structure in the convolutional vision network and replace it with an approximate dense structure to improve the computational efficiency. Assuming that translation invariance means that the final network is composed of several convolution modules, then for the optimization work, we need to find an improved local structure, and then simply repeat it in space. Based on the Hebbian principle, Arora proposes a layer by layer construction method: in the last layer, analyze the node correlation, and aggregate the high correlation into a group, which will form the nodes of the next layer and connect with the nodes of the previous layer. It is assumed that each node in the front layer of the network always corresponds to some areas of the original image, and these nodes are integrated into a filter bank. Then the lower layer (close to the original data) will focus on the local area, which means that there will be many node groups only connected to a single area, which can be covered by 1 * 1 convolution in the later layer. Of course, the number of groups can be reduced by increasing the image area covered by the node group. In order to avoid too complex size alignment of feature map (or for the convenience of construction), only 1 * 1, 3 * 3 and 5 * 5 filters are selected in this paper. These layers with different specifications and their output filter banks will be connected to the same output vector to form the input of the latter layer. In addition, in view of the important role of pooling layer in current CNN, it should be beneficial to add another pooling part to the module, so Fig. 1 is obtained. A new convolution network can be obtained by concatenating such modules layer by layer (instead of each layer of the original CNN). It can be predicted that with the increase of network layers, the convolution ratios of 3 * 3 and 5 * 5 will increase, and the spatial constraints corresponding to the original image will be less, which is equivalent to that the learned features will be more abstract.

Fig. 1

The simple Inception.

However, such a simple improvement module will quickly lead to the collapse of computing efficiency. Because, even if the number of 5 * 5 convolutions is minimized, more filtering operations than traditional models will consume computing resources quickly. Moreover, when connected to the pooling layer, the output of the optical pooling is equal to the number of the input characteristic maps of the previous layer, which will inevitably lead to the output dimension disaster when connected with other convolution layers in parallel. Although the sparse structure of the original network is optimized, the rapidly increasing computational cost after several layers will completely offset this part of the benefits.

And the actual improvement idea is also very intuitive: add appropriate dimension reduction to all places that need a lot of calculation. The success of the embedding method proves that even low-dimensional embedding can cover the information of relatively large image area. In order to meet the experimental requirements, we hope that the embedded expression is sparse in most places, and only starts to be dense when the signal is uniformly concentrated. Therefore, 1 * 1 convolution is introduced before the 3 * 3 and 5 * 5 convolutions with large amount of calculation, and the original neurons are fully connected with the newly added neurons of 1 * 1 convolution. In this way, the number of connections is reduced from the number of original feature graphs to the number of (controllable) 1 * 1 convolutions. As shown in Fig. 2, if the original architecture is Fig. 2(a), the number of input characteristic graphs is m, and the number of 3 * 3 convolution filters is n, then the number of parameters is n (3 * 3 * m + 1); if 1 * 1 convolution filter is added in the middle during the improvement, then the number of parameters becomes l (m + 1) + n (3 * 3 * l + 1), and the number of parameters can be reduced by simply monotonic integer l < m, thus the number of connections and calculation amount can be reduced. At the same time, the added 1 * 1 convolution layer also increases the nonlinearity of the network, which makes the model more discriminative. The final Inception module is shown in Fig. 3.

Fig. 2

Dimension reduction of cascaded 1 * 1 convolution.

Fig. 3

Improved Inception.

By stacking Inception module and the conventional pooling layer described above (halving the size of the feature map), a CNN like learning network can be obtained. The Google team suggests using Inception only at the higher level, while keeping the traditional convolution layer at the lower level, which can guarantee the computing performance.

It is easy to see that the use of Inception to form a network can avoid the unlimited increase of computational complexity while increasing the number of nodes in each layer. The ubiquitous 1 * 1 in the module has been dimensionally reduced before the actual convolution operation, reducing the huge amount of input in the previous stage to the acceptable amount in the later layer. At the same time, this structure is also in line with the intuitive cognition that visual information should be processed on different scales, which together constitute the source of high-level abstraction.

2.2Schematic diagram of architecture

The CNN architecture used in this paper is shown in Fig. 3, where each “Inception” block represents a cascade module similar to Fig. 1. It is shown in Table 1 that specific convolution kernel size and other parameter settings, named colornet.

The input picture is limited to 224 * 224, RGB24 bit bitmap; the output corresponds to 11 kinds of colors (black, white, gray, red, orange, yellow, green, blue, brown, purple, pink). All neurons selected relu as the excitation function (f (x) = max (0, x)). For the Inceptiont row in the table,¬=3×3 reduce,¬=5×5 reduce, and the poor proj column represent the number of convolution filters used for dimension reduction respectively. The number of parameters of each layer (interval) can be calculated according to the method of CNN. For the Inception module, take the Inception (3a) layer as an example, and its composition is detailed as shown in Table 2:

Near the top of the network, avg pool is added to the model instead of the direct full connection layer, which reduces the dimension and reduces the number of parameters to be trained. However, in order to avoid over fitting, dropout operation is added in the end.

In addition, it is also recommended by GoogleNet to add a softmax exit above the Inception (4a) layer in Table 2.

Because of its deep network, it is not easy to converge. In layer by layer training, we can use labeled data at the middle exit to carry out a back propagation, adjust the front network, and continue to train backward after convergence, which is equivalent to giving a better initial value to the lower layer. The specific settings are shown in Table 3 below:

3.1The application of color attribute of traditional hand decorated fabric

Because of the intuitive visual experience, color has become the most important and priority element in the interior furnishing collocation, and because of its rich and variable colors and the sense of hierarchy brought by the collocation, the designers also pay special attention to decorative fabrics, especially the large area coverage of fabrics will often play a supporting role in controlling the main color of the interior, so the selection of fabric color and the collocation of interior furnishings It must be considered in the initial stage to avoid the influence of color deviation on the overall indoor environment trend. Compared with modern decorative fabrics, traditional hand-made decorative fabrics are more limited when they are matched with furnishings, but the space for designers to play is still large. Decorative fabrics cover curtains, household masks, floor laying, indoor furnishings and bedding, etc., while the traditional fabric colors are no more than five categories of red, yellow, blue, green and black, and some colors derived from them, light or deep. However, it must be realized that the traditional decorative fabrics often permeate the residents’ certain cultural feelings and aesthetic tastes, such as Huai The old generation or traditional people tend to prefer traditional colors, and their interior furnishings often exude or classical or solemn atmosphere, so when matching with this kind of furnishings, it is necessary to realize the organic blending with the hue of this kind of furnishings. For example, the fabric under the antique vase can be the dark yellow of the totem color of the Chinese nation; for the indoor furnishings with many characters and paintings, the curtains can choose multi-level light color landscape patterns; for the rooms with ancient wood furniture as the main furnishing layout, the carpet color can be mostly dark. The use of traditional decorative fabrics often reflects the appreciation of classical culture and the pursuit of refined and solemn spiritual interest of the owner of the room. Therefore, for traditional furnishings, the color selection should be as close as possible to the breath contained in them, in order to achieve mutual benefit.

3.2Pattern attribute application of traditional hand decorated fabric

The pattern of decorative fabric is the best carrier to show the regional cultural characteristics. In this regard, the traditional hand-made decorative fabric has the most advantages. For example, Persian carpet is full of rich Islamic customs, Japanese Zen cultural ornaments, and the blankets of Maya civilization in central and South America, all of which contain regional culture, national customs and even national totem, which is also a wonderful carrier for adding indoor exotic customs and highlighting inclusive culture. If the lamp shade or curtain in the study is decorated with a Japanese Zen like pattern, it can not only express the owner’s desire for peace, but also make people inadvertently look at the Japanese wooden house with the clear and crisp bagpipes. For another example, if Persian carpets are laid in the reception hall, the living room will be greatly enhanced immediately. In a word, the fabric pattern containing regional culture will set off the indoor atmosphere in an instant because of its intuitive visual impact and some kind of cultural atmosphere emitted by the pattern, so as to show the spiritual interest and aesthetic level of the host.

In addition, the material attribute is also an element that traditional hand-made decorative fabrics pay more attention to in the application of interior furnishings, especially the hand-made fabrics, whose texture and comfort are often more prominent. Of course, because of the wisdom and hard work of the stitcher, the price and the status symbol along with it are more obvious.

4.1Data acquisition and preprocessing

For the convenience of comparison, 11 colors of black, white, gray, red, orange, yellow, green, blue, brown, purple and pink were selected as the classification labels in this paper. In order to make the model close to reality and avoid tedious manual annotation, we choose to directly grab pictures from Google and other network suppliers as the training data set. The data sources used in the experiment are as follows:

1. Car data: get from Google Images, search with the keyword of “English color name + car” and grab the results with crawlers, about 800 for each category.

2. Clothing data dress; obtained from Ebay, also search under the clothing top classification with “English color name” and use reptiles to grab, about 2000 pieces for each category.

3. Integrated data misc: grab from Google Images, search and crawl the results with “English color name + color”, about 200 for each category.

All of the above data have been screened by simple manual method, and the part of obvious color inconformity has been removed.

The test data set uses the Ebay product picture set provided by PLSA-ind, including four kinds of products, including cars, dresses, shoes and pottery products. Each product has 11 kinds of colors and 12 pictures of each kind of colors, and each picture has a corresponding mask file to mark the main color area of the image (as shown in Fig. 4). In addition, the dress comp atlas contains more than 120 pictures with more than one label, that is, one garment has two distinct colors.

Fig. 4

Schematic diagram of Ebay atlas for test.

Naturally, we assume that the main body of the picture always appears in the middle, so for all pictures, we scale to the minimum side length of 256px, and then cut out the central 224 * 224 area as the actual input data.

When CNN model is implemented, cuda-convent is used to write the corresponding architecture configuration and run, and GPU operation is turned on at the same time; for PLSA-ind, Matlab code provided in the paper is used directly.

When the trained model is used for prediction, since the softmax layer outputs a probability vector, the category corresponding to the largest component is the judgment result (arg max k = 1,... kok). At the same time, in the remaining components, if any value exceeds 70% of the maximum component, it will also be returned as part of the multi tag determination. PLSA-ind has the same method for multi label determination.

4.2CNN network architecture comparison

In order to find the optimal network structure for object color recognition, this paper attempts to use CNN models of different depths, including the original 9 Inception modules and 22 layer architecture of GoogLeNet, the shallow layer network directly truncated at the middle exit of this model, and the avg pool of the last layer of ColorNet in this paper is replaced by the traditional full connect layer. Input car and dress data sets respectively for training, and then the results on the corresponding test set of Ebay are shown in Table 4.

From lines 1 and 2, we can see that using avg pool instead of full connection layer can achieve better results on the test set, which shows that it has better universality.

Comparing 1 and 3 lines, we can see that the recognition accuracy of three Inception cascaded networks is not as good as several deeper models due to their shallow layers.

Compared with 1 and 4 lines, the deeper Inception network can get better results, but in actual operation, it is found that GoogLeNet needs more convergence time than ColorNet. On NVIDIA Titan and 24 G ram tester, the convergence time of ColorNet is about 16 hours, while that of GoogLeNet is about 25 hours. Considering the improvement of its accuracy, this paper chooses the existing structure with fewer layers.