India is an agricultural nation with large agricultural produce. Various types of fruits and vegetables are produced throughout the year. All the agricultural produce has to be sorted and graded, and majority of it is done through manual labor. Manual labor creates various problems like subjective grading, tedious work, inconsistencies, and low productivity. Most of the above problems can be minimized using machine vision.

The sorting system used in many foods processing industry sorts using one of the following features like color, shape, weight and size and it requires specific environment for its efficient working. The background color of the conveyer system needs to be of specific color for easy segmentation of the particular fruit from its background, proper lighting, and a high performance camera. The values for grading must be manually fed into the system prior to the grading process.

In this paper Section II features the material involved, Section III the different color models and the algorithm used, Section IV the experimental results and Section V the conclusion and the future scope.

Related works

1. Developing a fruit sorting and grading using fuzzy logic. The image captured was converted to binary and boundary was extracted to determine the fruit size [1].

2. Using charge coupled device to capture only one view of the apple. Four different filters were used to acquiring the red, green, blue and the infrared regions. The two classifiers used were K-nearest neighbors and single layer perceptron. K-Nearest neighbors was used for large data, low computational cost validation technique [2].

3. Using a genetic algorithm to arrange the 32 different features extracted and derived from extracted features to grade the apples [3].

4. Developed a size and color sorting using fuzziness. The image was analyzed in the RGB color model. The classification depended on the color extracted and the degree of matching with the red, blue and green component [4].

5. Using Gaussian mixture model to determine the maturity level of mango. Blue background in the conveyer was used for easy segmentation of mango [5].

6. Proposal of a diameter and weight sorter in which all the input and output are connected through local area network. The sensed inputs are fed into MATLAB which extracts the features and relayed back to the output module [6].

7. Discussions on the best results were obtained when three features color, size and shape are used for grading. The paper also states that fuzzy logic has an efficiency of eighty-six percent [7].

8. Propose of an apple grading machine using fuzzy logic with five quality feature color, defect, shape, weight and size [8].

The image captured is through a mobile camera of 3.2 megapixels with an Android application ‘Ip webcam’ for transferring the image from the captured image to the processing software MATLAB. The algorithm was developed in MATLAB with the software designed functions. To physically represent the system an Arduino microcontroller equipped with sensors and relays are used to show how the system is designed to work practically.

There are three color models used for this sorting system.

Rgb color model

The image transferred into MATLAB is in the form of Red, Green and Blue (RGB) color model. This model uses the combination all the three components for creating a color. The segmentation using this particular model is less efficient as lighting, reflection and other factors disrupt the different and hence consistent segmentation cannot be achieved.

L*a*b* color model

This color model has three components the luminescence (L) and two chromaticity index ‘a’ and ‘b’ respectively. The luminescence is used to determine the whether the image is dark or light in color. Higher the value of luminescence the brighter the image and lower the value the darker the image. The pure color of the image is determined using the two chromaticity features ‘a’ and ‘b’. For segmentation purpose using K-Means clustering is done in this color model.

HSV color model

This color model consists of three features the Hue (H), Saturation (S) and Value (V) to represent an image. The hue component is used for representing the pure color usually varying from 0 to 360 with each value representing a color. The saturation component determines the white pigments in the color and the value component determines the brightness of the image. This color model can be used to determine the pure color with a single component. Hence for fruit and vegetable color identification the image is converted to HSV color model.

Process flow

As shown in Figure 1 the image captured from the mobile device is converted into L*a*b* color model for segmentation using K-Mean clustering. After segmentation the image is converted to HSV color model for color identification and sorting.

Figure 1: The image captured from the mobile device is converted into L*a*b*.

K-Means clustering

K-Means clustering is an in-built function in MATLAB. The image processed is in the L*a*b* color model. As shown in Figure 2 the image is segmented into three clusters. The first cluster to be formed is the blue color region. Hence for ease of segmentation a blue coin is used. The blue coin is used for size measurement purpose. The pixels occupied by the blue coin are used to determine the size of the fruit or vegetable as the position of the camera is not constant. Hence by knowing the size of the blue coin physically, a relationship with the pixel and a size measuring unit can be determined.

Figure 2: The image is segmented into three clusters.

As shown in Figure 3 different background had little effect on the segmentation of the image. The different background conditions not only refer to the background color but also to the variation in lighting.

Figure 3: Different background had little effect on the segmentation of the image.

As shown in Figure 4 various fruits and vegetables were successfully segmented. The image only represents only few of the fruits and vegetables. The segmented image is converted to HSV color model for further color extraction.

Figure 4: Various fruits and vegetables were successfully segmented.

Fuzzy logic

Two features the color and the shape are determined. Due to variation in lighting the mean value of the segmented image is taken. The shape of the image is determined by using an inbuilt function ‘regionprop’ in MATLAB.

Figure 5 and Figure 6 shows the membership function used for the determination of fruit. The combination of the color and shape is used for determining the fruit and vegetable. The regions under consideration for color are; red, orange-yellow and green while for shape are; circular, elliptical and straight. All these features are extracted from the binary image of the segmented image.

Figure 5: Membership function for color.

Figure 6: Membership function for shape.

Implementation

The basic layout of the system is shown in Figure 7. The system has a sensor which detects the motion or presence of the fruit. The image is transferred to the software which determines the fruit and the size. The motor connected in the system transports the fruit to the particular location and the relays direct it accordingly.

Figure 7: Basic layout of the control system.

The prototype of the system is shown in Figure 8 and the execution of the program is shown in Figure 9.

Figure 8: Prototype of the control system.

Figure 9: Execution of the program.

The fruit and vegetable was successfully identified. The different fruits and vegetable that was successfully identified were: tomato, orange, carrot, raw mango, mango, cucumber, banana, green apple. The success rate depended on the lighting condition. Poor lighting resulted in successful image segmentation but the identification part was flawed. A background with large spaces of different color showed poor segmentation.

The current system can only identify eight different fruits or vegetable at one possible time. To increase the number of fruits identified the features extracted should be more and relationship between these features need to be created. One way of achieving this is through genetic algorithm and learning techniques that would enable it to learn a particular fruit. Another technique that can be used is an electronic nose as a sensory input. This would greatly reduce the need to extract further features as every fruit and vegetable expels different chemicals and hence can be used for the identification of the fruit and this also reduces the storage space for the processing the features. Combining these two parameters of we can develop a near foolproof fruit sorting or vegetable sorting system.