In this article, I will show you how to create a Covide 19 virus X Ray images classification by using transfer learning and the network will be based on the series of EfficientNet , which has achieved state of the art accuracy on ImageNet while being 8.4x smaller and 6.1x faster.and MobileNetV2 Model …there are more references to MobileNetv2 you can read it
all the code in google colab
What is Covid -19 ?
Coronaviruses are a group of viruses that can cause disease, which can vary from colds and coughs to sometimes more serious diseases. Middle East Respiratory Syndrome (MERS-CoV) and SARS-CoV were severe cases the world has already faced. SARS-CoV-2 (n-coronavirus) is the new virus for the coronavirus family, first discovered in 2019, which has not been identified in humans before. It is a persistent virus that started from Wuhan in December 2019. It was later declared a pandemic by the World Health Organization because of the high prevalence rate worldwide. Currently (on May 15, 2020), this results in a total of 300,000 deaths worldwide, including 159,000 deaths alone in Europe. It becomes important to understand this spread. Here we explain the importance of deep learning in computer vision in the problem of classification of these images and knowing which model is the best in accuracy
why transfer learning ?
In this post , you will learn how to classify images of Covide-19 and virtual and normal health images for human lungs by using transfer learning from a pre-trained network.A pre-trained model is a saved network that was previously trained on a large dataset, typically on a large-scale image-classification task. You either use the pre-trained model as is or use transfer learning to customize this model to a given task.The intuition behind transfer learning for image classification is that if a model is trained on a large and general enough dataset, this model will effectively serve as a generic model of the visual world. You can then take advantage of these learned feature maps without having to start from scratch by training a large model on a large dataset.
Dataset from kaggle
What a kaggle ?
Kaggle is a platform to compete with others in competitions which are based on machine learning tasks. You may know about Codechef, Hackerrank etc., kaggle is also like them, but the key difference is the competition is only related to machine learning, data science, Deep learning or AI related. Most of the time you have been given some training and testing dataset to build some good machine learning models and when you publicize your kernel , others will review it and then give you up votes just like Quora.
Compared to other models achieving similar ImageNet accuracy, EfficientNet is much smaller. For example, the ResNet50 model has 23,534,592 parameters in total, and even though it still under performs the smallest EfficientNet, which only takes 5,330,564 parameters in total.Why is it so efficient? To answer the question, we will dive into its base model and building block. You might have heard of the building block for the classical ResNet model is identity and convolution block.For EfficientNet, its main building block is the mobile inverted bottleneck MBConv, which was first introduced in MobileNetV2. By using shortcuts directly between the bottlenecks which connects a much fewer number of channels compared to expansion layers, combined with depthwise separable convolution which effectively reduces computation by almost a factor of k2, compared to traditional layers. Where k stands for the kernel size, specifying the height and width of the 2D convolution window.
and you can see, starting from the smallest EfficientNet configuration B0 to the largest B7, accuracies are steadily increasing while maintaining a relatively small size.
Transfer Learning with EfficientNet and MobileNetV2
Transfer learning for image classification is more or less model agnostic. You can pick any other pre-trained ImageNet model such as MobileNetV2 or ResNet50 as a drop-in replacement if you want.A pre-trained network is simply a saved network previously trained on a large dataset such as ImageNet. The learned features can prove useful for many different computer vision problems, even though these new problems might involve completely different classes from those of the original task.The EfficientNet is built for ImageNet classification and contains 1000 classes labels. For our dataset, we only have 2. Which means the last few layers for classification is not useful for us. They can be excluded while loading the model by specifying the include_top argument to False, and this applies to other ImageNet models made available in Keras applications as well.We expect the model to perform well in our Covide19-X Ray dataset to classify it if it detects someone’s lungs are infected with Covid-19 or not infected by X-ray scanning images. Covid-19 X Ray classification problem with a relatively small number of samples, the easiest way to start is to open this notebook in Colab, while I will explain more details here in this post.First go to kaggle website and copy the api to the dataset. Of course you must have an account with kaggle services ….Go to this link for the data set outside
and copy api command for dataset
About the dataset
A database classification labeling for chest X-Ray images was created for three cases classified as COVID-19 positive cases in addition to regular and Viral Pneumonia images. There are 219 positive images of COVID-19, 1341 Normal images, and 1345 pictures of Viral Pneumonia.
Ok copy this API command from above image
kaggle datasets download -d tawsifurrahman/covid19-radiography-database
In google colab notebook the command will be
!kaggle datasets download -d awsifurrahman/covid19-radiography-database
and write this command
from google.colab import filesfiles.upload()!ls -lha kaggle.json!pip install -q kaggle!mkdir -p ~/.kaggle!cp kaggle.json ~/.kaggle/!chmod 600 ~/.kaggle/kaggle.json!kaggle datasets download -d awsifurrahman/covid19-radiography-database# unzip folder dataset!unzip /content/covid19-radiography-database.zip
to display some images from dataset
#load imagesdef show_images(dataset_path):# size of the image: 48*48 pixelspic_size = 48# input path for the imagesbase_path = dataset_pathplt.figure(0, figsize=(12,20))#cpt = 0for expression in os.listdir(base_path + "/"):for i in range(25):# cpt = cpt + 1plt.subplot(5,5,i+1)img = load_img(base_path + "/" + expression + "/" +os.listdir(base_path + "/" + expression)[i], target_size=(pic_size, pic_size))plt.imshow(img, cmap="gray")plt.tight_layout()plt.show()
#pip command install EfficientNet model by using!pip install efficientnet
Imported libraries and modules
#Imported libraries and modulesimport efficientnet.keras as efnfrom sklearn.metrics import classification_report,accuracy_score,f1_score,confusion_matriximport numpy as npfrom keras.preprocessing.image import load_img, img_to_arrayimport matplotlib.pyplot as pltimport osfrom keras.models import Modelfrom keras.utils import np_utilsimport tensorflow as tffrom tensorflow.keras.models import load_modelfrom keras.utils.np_utils import to_categoricalfrom keras.preprocessing.image import ImageDataGeneratorfrom keras.layers import Activation,Dense, Dropout, Flatten, Conv2Dfrom keras.layers import GlobalAveragePooling2D,BatchNormalization
loading pretrained conv base model
img_shape=224import efficientnet.keras as efn
#using here EfficientNet series BObaseModel =efn.EfficientNetB0(weights ='noisy-student', include_top=False, input_shape = (img_shape,img_shape,3))baseModel.summary()
To create our own classification layers stack on top of the EfficientNet convolutional base model. We adapt GlobalAveragePooling2D . GlobalMaxPooling2D results in a much smaller number of features compared to the Flatten layer, which effectively reduces the number of parameters.
x = baseModel.outputx = GlobalAveragePooling2D()(x)x = Dropout(0.3)(x)x = Dense(128, activation=”relu”)(x)x = Dropout(0.3)(x)x = Dense(64, activation=”relu”)(x)predictions = Dense(3, activation=”softmax”)(x)model = Model(inputs=baseModel.input, outputs=predictions)
for layer in baseModel.layers:layer.trainable = False
you can compile and train the model with Keras’s ImageDataGenerator, which adds various data augmentation options during the training to reduce the chance of overfitting.
data_gen= ImageDataGenerator(horizontal_flip=True,vertical_flip=True,rotation_range=90,width_shift_range=0.1,height_shift_range=0.1,zoom_range=.1,rescale=1/255,fill_mode=’nearest’,shear_range=0.1,brightness_range=[0.5, 1.5],validation_split=0.3)
Note that the validation data should not be augmented!
test_datagen = ImageDataGenerator(rescale=1.0 / 255)train_generator = train_datagen.flow_from_directory(# This is the target directorytrain_dir,# All images will be resized to target height and width.target_size=(height, width),batch_size=batch_size,# Since we use categorical_crossentropy loss, we need categorical labelsclass_mode=”categorical”,)validation_generator = test_datagen.flow_from_directory(validation_dir,target_size=(height, width),batch_size=batch_size,class_mode=”categorical”,)
Then you can compile and train the model again for some more epochs.
model.compile(optimizer='adam', loss="categorical_crossentropy", metrics=['accuracy'])
final fit the model
results = model.fit(train_generator,epochs=epochs,steps_per_epoch=train_generator.n/batch_size,validation_data=val_generator,validation_steps=val_generator.n/batch_size)
So we can built function for pretrained efficient model for every model from B0 to B7 for testing all of them models to our dataset
def model_efficientnetb0(no_classes):#mport Modelimg_shape=224import efficientnet.keras as efnmodel =efn.EfficientNetB0(weights ='noisy-student', include_top=False, input_shape = (img_shape,img_shape,3))x = model.outputx = GlobalAveragePooling2D()(x)x = Dropout(0.3)(x)x = Dense(128, activation="relu")(x)x = Dropout(0.3)(x)x = Dense(64, activation="relu")(x)predictions = Dense(no_classes, activation="softmax")(x)model = Model(inputs=model.input, outputs=predictions)return modeldef model_efficientnetb7(no_classes):#mport Modelimg_shape=600model =efn.EfficientNetB7(weights ='imagenet', include_top=False, input_shape = (img_shape,img_shape,3))x = model.outputx = GlobalAveragePooling2D()(x)x = Dropout(0.3)(x)x = Dense(128, activation="relu")(x)x = Dropout(0.3)(x)x = Dense(64, activation="relu")(x)predictions = Dense(no_classes, activation="softmax")(x)model = Model(inputs=model.input, outputs=predictions)return model
we can see summary for model
and also built all of these in funcion
def model_train(dataset_path,model_name,epochs,batch_size,no_class,img_shape):show_images(dataset_path)no_classes=no_classdata_gen= ImageDataGenerator(horizontal_flip=True,vertical_flip=True,rotation_range=90,width_shift_range=0.1,height_shift_range=0.1,zoom_range=.1,rescale=1/255,fill_mode='nearest',shear_range=0.1,brightness_range=[0.5, 1.5],validation_split=0.3)#load the training datatrain_generator = data_gen.flow_from_directory(dataset_path,target_size=(img_shape,img_shape),batch_size=batch_size,class_mode='categorical',subset='training',shuffle=True)#load the training dataval_suffle=Falseval_generator = data_gen.flow_from_directory(dataset_path,target_size=(img_shape,img_shape),batch_size=batch_size,class_mode='categorical',subset='validation',shuffle=val_suffle)train_generator.next().shape,train_generator.next().shapeval_generator.next().shape,val_generator.next().shapex,y = train_generator.next()plt.figure(figsize=(10,10))for i in range(8):plt.subplot(4,2,i+1)image = x[i]label = y[i]print (label)plt.imshow(image)plt.show()if model_name=='efficientnetb0':model=model_efficientnetb0(no_classes)if model_name=='efficientnetb7':model=model_efficientnetb7(no_classes)model.compile(optimizer='adam', loss="categorical_crossentropy", metrics=['accuracy'])results = model.fit(train_generator,epochs=epochs,steps_per_epoch=train_generator.n/batch_size,validation_data=val_generator,validation_steps=val_generator.n/batch_size)no_epochs=int(epochs)plt.plot(results.history['accuracy'])plt.plot(results.history['val_accuracy'])plt.title(model_name+' Model Accuracy ')plt.ylabel('accuracy')plt.xlabel('epoch')plt.legend(['train', 'test'], loc='upper left')plt.savefig(model_name+'_accuracy.png')plt.show()plt.savefig(model_name+'_accuracy.png')plt.plot(results.history['loss'])plt.plot(results.history['val_loss'])plt.title(model_name+' Model Loss')plt.ylabel('loss')plt.xlabel('epoch')plt.legend(['train', 'test'], loc='upper left')plt.savefig(model_name+'_loss.png')plt.show()plt.savefig(model_name+'_loss.png')val_generator.reset()pred=model.predict_generator(val_generator,steps=val_generator.n/batch_size,verbose=1)predicted_class_indices=np.argmax(pred,axis=1)labels = (train_generator.class_indices)labels = dict((v,k) for k,v in labels.items())y_pred=model.predict(val_generator,steps=val_generator.n/batch_size,verbose=1)y_true = val_generator.classesy_pred=np.argmax(y_pred, axis=1)from sklearn.metrics import classification_report,accuracy_score,f1_scoreprint('accuracy_score:',accuracy_score(y_true,y_pred))print('f1_score:',f1_score(y_true,y_pred,average='macro'))class_label=list(train_generator.class_indices.keys())print(classification_report(y_true, _pred,target_names=class_label))class_names = val_generator.class_indices.keys()#save modelmodel.save(model_name+'.h5')
and calculate the accuracy_score for validation dataset and prediction
val_generator.reset()pred=model.predict_generator(val_generator,steps=val_generator.n/batch_size,verbose=1)predicted_class_indices=np.argmax(pred,axis=1)labels = (train_generator.class_indices)labels = dict((v,k) for k,v in labels.items())y_pred=model.predict(val_generator,steps=val_generator.n/batch_size,verbose=1)y_true = val_generator.classesy_pred=np.argmax(y_pred, axis=1)from sklearn.metrics import classification_report,accuracy_score,f1_scoreprint('accuracy_score:',accuracy_score(y_true,y_pred))print('f1_score:',f1_score(y_true,y_pred,average='macro'))
Prediction Final Model and testing for unseen dataset
def model_predict(img_path, model_path):MODEL_PATH = model_pathprint('Model loading...')model=load_model(MODEL_PATH)print('Model loaded. Started serving...')image = cv2.imread(img_path)image = cv2.resize(image, (img_shape,img_shape))#image = image.astype("float") / 255.0image = img_to_array(image)image = np.expand_dims(image, axis=0)proba = model.predict(image)confidence = proba.max() * 100print('Class Name:',proba)print('Confidence:',confidence)
and finally you can test for another efficientneXX model series by using
dataset_path='/content/COVID-19 Radiography Database'model_name='efficientnetb0'epochs=10batch_size=48no_class=3img_shape=224model_train(dataset_path,model_name, epochs,batch_size,no_class,img_shape)
precision recall f1-score support COVID-19 0.68 1.00 0.81 65 NORMAL 0.97 0.87 0.92 402 Viral Pneumonia 0.93 0.96 0.94 403 accuracy 0.92 870 macro avg 0.86 0.94 0.89 870 weighted avg 0.93 0.92 0.92 870
using another model for MobileNetV2
dataset_path='/content/COVID-19 Radiography Database'model_name='mobilenetv2'epochs=10batch_size=16no_class=3img_shape=224
The Result for MobileNetV2
precision recall f1-score support COVID-19 0.90 0.72 0.80 65 NORMAL 0.98 0.33 0.49 402 Viral Pneumonia 0.59 1.00 0.74 403 accuracy 0.67 870 macro avg 0.82 0.68 0.68 870 weighted avg 0.79 0.67 0.63 870
Confusion Matrix for metrics (Confusion,Precision,Recall Matrix)
Normalize Confusion Matrix
get a random batch of images and predict the images on the model
we see the Efficientnet B0 is better and best than MobileNetV2
in parameters size and accuracy for both of them .
all the code in google colab