To identify the core features of the eggs, for segmenting them from the background sheet and to classify them into respective categories, a simple deep learning technique was used with four hidden layers to provide results that are much more accurate compared to conventional methods. Deep learning models are trained using TensorFlow framework to provide three different results such as foreground-background segmentation, detecting eggs, and classifying detected egg.
The core deep learning model used in our experiments is shown in Figure 2.3. The model consists of convolution layers, max-pooling layers, and fully connected layers. These layers are trained to identify the features of egg, while the last layer is modified to provide categorical or continuous data. The core CNN model is trained using a stride of (2 × 2) and kernel of (5 × 5) for convolution, with (2 × 2) max pooling and a fully connected layer consisting of 3200 neurons. The input to the CNN model is a 32 × 32 dimension image with three color channels, RGB.
Figure 2.2 Silkworm egg classes: hatched eggs and unhatched eggs.
Figure 2.3 Core CNN model.
2.3.2 Foreground-Background Segmentation
The basic requirement for accurate counting of silkworm egg is to perform foreground-background (FB) segmentation. In the previous attempts, the background was segmented based on the intensity value of the eggs [8]. The region that has no pixel values corresponding to the eggs is considered as background and discarded before the image processing stage. However, this is not ideal in all situations, since the silkworm eggs laid on the sheet may also contain urine from the silkworms that discolor the background. The urinated background dries into a white layer that resembles an egg pixel intensity value close to 230 for an 8-bit grayscale image.
The resolution of the digital data that is fed into the CNN model is another reason to perform segment the background before the class of eggs is determined. The eggs are of the size around 32 × 32 pixels after scanning, while the entire sheet of silkworm eggs is of the size 5008 × 6984 pixels (in our experiment). If the entire sheet was fed to the CNN model with an input image size of 32 × 32 pixels with a sliding window method, then the model must classify 35M images that would be computationally expensive for a system without GPU support.
To overcome this issue, the FB CNN model was trained with an input data of 128 × 128 pixels, three-channel RGB image that was fed to the core CNN model, as shown in Figure 2.3, to provide categorical output using the softmax activation function. Some of the corresponding specifications of the FB CNN model are provided in Table 2.1 along with accuracy scores. Using a 128 × 128 pixel input to the FB CNN model, the entire silkworm egg sheet is divided into square grids of 128 × 128 with a stride of 128, resulting in an image set of 2K images that must be processed for categorical results as foreground or background class. Figure 2.4 represents the segmentation of the entire silkworm egg sheet, where the foreground (presence of egg) and background (absence of egg) are represented by a green color (pixels) and red color, respectively. Further processing is only carried out for pixels represented in green color that minimizes computational time and increase final accuracy as background pixels are dropped out of the data processing cycle.
Table 2.1 Specifications of foreground-background (FB) segmentation CNN model.
| Input image | Activation/output | Training samples | Test samples | Validation samples | Test loss | Accuracy on the test set | Accuracy on the validation set |
| 128 × 128 | SoftMax 2 class-(0/1) | 142 × 103 | 64.3 × 103 | 9.6 × 103 | 0.1242 | 96.1554% | 96.6422% |
Figure 2.4 Foreground-background segmentation of entire silkworm egg sheet (a) (input) and (b) (output).
2.3.3 Egg Location Predictor
After segmentation of egg pixels from background pixels, the next step is to determine the location of the eggs. Many CNN models have been introduced such as Fast-RCNN [13] and YOLO [14] to predict the location of the object using the Intersect over Union method. There are two main reasons for not considering well-known techniques to locate the eggs. Firstly, in these methods the object/ROI dimensions are over 100 × 100, irregular in size, and the method has been trained for different bounding box dimensions. These techniques become unacceptable to determine the location of the egg, as the eggs have an average size of 28 × 28 pixels to 36 × 36 pixels. Moreover, the shape of the egg remains similar with minor deformation that can be neglected; hence, training for different dimension bounding box will not yield any good result. Secondly, these methods have the limitation of how many similar class objects can be recognized within a single bounding box, which are two objects for YOLO [14]. Since the eggs are small, many eggs will be present within a 100 × 100 grid image that may belong to the same class and hence may not be detected.
The specification of the egg location predictor CNN model is represented in Table 2.2, where the input to the core CNN model has been changed to 32 × 32 pixels, three-channel RGB image. Here, the output is a regression that provides the location of the egg center (x, y) rather than a bounding box (four corner points). The training dataset consists of both the class images, i.e.,