Classification Covid-19 X-Ray Images

Using Transfer learning with EfficientNet and MobileNetV2

Photo by engin akyurt on Unsplash

In this article, I will show you how to create a Covide 19 virus X Ray images classification by using transfer learning and the network will be based on the series of EfficientNet , which has achieved state of the art accuracy on ImageNet while being 8.4x smaller and 6.1x faster.and MobileNetV2 Model …there are more references to MobileNetv2 you can read it

all the code in google colab

What is Covid -19 ?

Coronaviruses are a group of viruses that can cause disease, which can vary from colds and coughs to sometimes more serious diseases. Middle East Respiratory Syndrome (MERS-CoV) and SARS-CoV were severe cases the world has already faced. SARS-CoV-2 (n-coronavirus) is the new virus for the coronavirus family, first discovered in 2019, which has not been identified in humans before. It is a persistent virus that started from Wuhan in December 2019. It was later declared a pandemic by the World Health Organization because of the high prevalence rate worldwide. Currently (on May 15, 2020), this results in a total of 300,000 deaths worldwide, including 159,000 deaths alone in Europe. It becomes important to understand this spread. Here we explain the importance of deep learning in computer vision in the problem of classification of these images and knowing which model is the best in accuracy

why transfer learning ?

In this post , you will learn how to classify images of Covide-19 and virtual and normal health images for human lungs by using transfer learning from a pre-trained network.A pre-trained model is a saved network that was previously trained on a large dataset, typically on a large-scale image-classification task. You either use the pre-trained model as is or use transfer learning to customize this model to a given task.The intuition behind transfer learning for image classification is that if a model is trained on a large and general enough dataset, this model will effectively serve as a generic model of the visual world. You can then take advantage of these learned feature maps without having to start from scratch by training a large model on a large dataset.

Dataset from kaggle

What a kaggle ?

Kaggle is a platform to compete with others in competitions which are based on machine learning tasks. You may know about Codechef, Hackerrank etc., kaggle is also like them, but the key difference is the competition is only related to machine learning, data science, Deep learning or AI related. Most of the time you have been given some training and testing dataset to build some good machine learning models and when you publicize your kernel , others will review it and then give you up votes just like Quora.

Why EfficientNet?

Compared to other models achieving similar ImageNet accuracy, EfficientNet is much smaller. For example, the ResNet50 model has 23,534,592 parameters in total, and even though it still under performs the smallest EfficientNet, which only takes 5,330,564 parameters in total.Why is it so efficient? To answer the question, we will dive into its base model and building block. You might have heard of the building block for the classical ResNet model is identity and convolution block.For EfficientNet, its main building block is the mobile inverted bottleneck MBConv, which was first introduced in MobileNetV2. By using shortcuts directly between the bottlenecks which connects a much fewer number of channels compared to expansion layers, combined with depthwise separable convolution which effectively reduces computation by almost a factor of k2, compared to traditional layers. Where k stands for the kernel size, specifying the height and width of the 2D convolution window.

and you can see, starting from the smallest EfficientNet configuration B0 to the largest B7, accuracies are steadily increasing while maintaining a relatively small size.

Transfer Learning with EfficientNet and MobileNetV2

Transfer learning for image classification is more or less model agnostic. You can pick any other pre-trained ImageNet model such as MobileNetV2 or ResNet50 as a drop-in replacement if you want.A pre-trained network is simply a saved network previously trained on a large dataset such as ImageNet. The learned features can prove useful for many different computer vision problems, even though these new problems might involve completely different classes from those of the original task.The EfficientNet is built for ImageNet classification and contains 1000 classes labels. For our dataset, we only have 2. Which means the last few layers for classification is not useful for us. They can be excluded while loading the model by specifying the include_top argument to False, and this applies to other ImageNet models made available in Keras applications as well.We expect the model to perform well in our Covide19-X Ray dataset to classify it if it detects someone’s lungs are infected with Covid-19 or not infected by X-ray scanning images. Covid-19 X Ray classification problem with a relatively small number of samples, the easiest way to start is to open this notebook in Colab, while I will explain more details here in this post.First go to kaggle website and copy the api to the dataset. Of course you must have an account with kaggle services ….Go to this link for the data set outside

and copy api command for dataset

About the dataset

A database classification labeling for chest X-Ray images was created for three cases classified as COVID-19 positive cases in addition to regular and Viral Pneumonia images. There are 219 positive images of COVID-19, 1341 Normal images, and 1345 pictures of Viral Pneumonia.

Ok copy this API command from above image

kaggle datasets download -d tawsifurrahman/covid19-radiography-database

In google colab notebook the command will be

!kaggle datasets download -d awsifurrahman/covid19-radiography-database

and write this command

from google.colab import files
files.upload()
!ls -lha kaggle.json
!pip install -q kaggle
!mkdir -p ~/.kaggle
!cp kaggle.json ~/.kaggle/
!chmod 600 ~/.kaggle/kaggle.json
!kaggle datasets download -d awsifurrahman/covid19-radiography-database
# unzip folder dataset
!unzip /content/covid19-radiography-database.zip

to display some images from dataset

#load images
def show_images(dataset_path):
# size of the image: 48*48 pixels
pic_size = 48
# input path for the images
base_path = dataset_path
plt.figure(0, figsize=(12,20))
#cpt = 0
for expression in os.listdir(base_path + "/"):
for i in range(25):
# cpt = cpt + 1
plt.subplot(5,5,i+1)
img = load_img(base_path + "/" + expression + "/" +os.listdir(base_path + "/" + expression)[i], target_size=(pic_size, pic_size))
plt.imshow(img, cmap="gray")
plt.tight_layout()
plt.show()

Install EfficientNet

#pip command install EfficientNet model by using
!pip install efficientnet

Imported libraries and modules

#Imported libraries and modules
import efficientnet.keras as efn
from sklearn.metrics import classification_report,accuracy_score,f1_score,confusion_matrix
import numpy as np
from keras.preprocessing.image import load_img, img_to_array
import matplotlib.pyplot as plt
import os
from keras.models import Model
from keras.utils import np_utils
import tensorflow as tf
from tensorflow.keras.models import load_model
from keras.utils.np_utils import to_categorical
from keras.preprocessing.image import ImageDataGenerator
from keras.layers import Activation,Dense, Dropout, Flatten, Conv2D
from keras.layers import  GlobalAveragePooling2D,BatchNormalization

loading pretrained conv base model

img_shape=224
import efficientnet.keras as efn
#using here EfficientNet series BO
baseModel =efn.EfficientNetB0(weights ='noisy-student', include_top=False, input_shape = (img_shape,img_shape,3))
baseModel.summary()

To create our own classification layers stack on top of the EfficientNet convolutional base model. We adapt GlobalAveragePooling2D . GlobalMaxPooling2D results in a much smaller number of features compared to the Flatten layer, which effectively reduces the number of parameters.

x = baseModel.output
x = GlobalAveragePooling2D()(x)
x = Dropout(0.3)(x)
x = Dense(128, activation=”relu”)(x)
x = Dropout(0.3)(x)
x = Dense(64, activation=”relu”)(x)
predictions = Dense(3, activation=”softmax”)(x)
model = Model(inputs=baseModel.input, outputs=predictions)
for layer in baseModel.layers:
layer.trainable = False

you can compile and train the model with Keras’s ImageDataGenerator, which adds various data augmentation options during the training to reduce the chance of overfitting.

data_gen= ImageDataGenerator(
horizontal_flip=True,
vertical_flip=True,
rotation_range=90,
width_shift_range=0.1,
height_shift_range=0.1,
zoom_range=.1,
rescale=1/255,
fill_mode=’nearest’,
shear_range=0.1,
brightness_range=[0.5, 1.5],
validation_split=0.3)

Note that the validation data should not be augmented!

test_datagen = ImageDataGenerator(rescale=1.0 / 255)
train_generator = train_datagen.flow_from_directory(
# This is the target directory
train_dir,
# All images will be resized to target height and width.
target_size=(height, width),
batch_size=batch_size,
# Since we use categorical_crossentropy loss, we need categorical labels
class_mode=”categorical”,
)
validation_generator = test_datagen.flow_from_directory(
validation_dir,
target_size=(height, width),
batch_size=batch_size,
class_mode=”categorical”,
)

Then you can compile and train the model again for some more epochs.

model.compile(optimizer='adam', loss="categorical_crossentropy", metrics=['accuracy'])

final fit the model

results = model.fit(train_generator,epochs=epochs,steps_per_epoch=train_generator.n/batch_size,validation_data=val_generator,validation_steps=val_generator.n/batch_size)

So we can built function for pretrained efficient model for every model from B0 to B7 for testing all of them models to our dataset

def model_efficientnetb0(no_classes):
#mport Model
img_shape=224
import efficientnet.keras as efn
model =efn.EfficientNetB0(weights ='noisy-student', include_top=False, input_shape = (img_shape,img_shape,3))
x = model.output
x = GlobalAveragePooling2D()(x)
x = Dropout(0.3)(x)
x = Dense(128, activation="relu")(x)
x = Dropout(0.3)(x)
x = Dense(64, activation="relu")(x)
predictions = Dense(no_classes, activation="softmax")(x)
model = Model(inputs=model.input, outputs=predictions)
return model
def model_efficientnetb7(no_classes):
#mport Model
img_shape=600
model =efn.EfficientNetB7(weights ='imagenet', include_top=False, input_shape = (img_shape,img_shape,3))
x = model.output
x = GlobalAveragePooling2D()(x)
x = Dropout(0.3)(x)
x = Dense(128, activation="relu")(x)
x = Dropout(0.3)(x)
x = Dense(64, activation="relu")(x)
predictions = Dense(no_classes, activation="softmax")(x)
model = Model(inputs=model.input, outputs=predictions)
return model

we can see summary for model

no_classes=3
model=model_efficientnetb0(no_classes)
model.summary()

and also built all of these in funcion

def model_train(dataset_path,model_name,epochs,batch_size,no_class,img_shape):
show_images(dataset_path)
no_classes=no_class
data_gen= ImageDataGenerator(
horizontal_flip=True,
vertical_flip=True,
rotation_range=90,
width_shift_range=0.1,
height_shift_range=0.1,
zoom_range=.1,
rescale=1/255,
fill_mode='nearest',
shear_range=0.1,
brightness_range=[0.5, 1.5],
validation_split=0.3)
#load the training data
train_generator = data_gen.flow_from_directory(
dataset_path,
target_size=(img_shape,img_shape),
batch_size=batch_size,class_mode='categorical',
subset='training',shuffle=True)
#load the training data
val_suffle=False
val_generator = data_gen.flow_from_directory(dataset_path,
target_size=(img_shape,img_shape),batch_size=batch_size,
class_mode='categorical',subset='validation',shuffle=val_suffle)
train_generator.next()[0].shape,train_generator.next()[1].shape
val_generator.next()[0].shape,val_generator.next()[1].shape
x,y = train_generator.next()
plt.figure(figsize=(10,10))
for i in range(8):
plt.subplot(4,2,i+1)
image = x[i]
label = y[i]
print (label)
plt.imshow(image)
plt.show()
if model_name=='efficientnetb0':
model=model_efficientnetb0(no_classes)
if model_name=='efficientnetb7':
model=model_efficientnetb7(no_classes)
model.compile(optimizer='adam', loss="categorical_crossentropy", metrics=['accuracy'])
results = model.fit(train_generator,epochs=epochs,steps_per_epoch=train_generator.n/batch_size,
validation_data=val_generator,validation_steps=val_generator.n/batch_size)
no_epochs=int(epochs)
plt.plot(results.history['accuracy'])
plt.plot(results.history['val_accuracy'])
plt.title(model_name+' Model Accuracy ')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.savefig(model_name+'_accuracy.png')
plt.show()
plt.savefig(model_name+'_accuracy.png')
plt.plot(results.history['loss'])
plt.plot(results.history['val_loss'])
plt.title(model_name+' Model Loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.savefig(model_name+'_loss.png')
plt.show()
plt.savefig(model_name+'_loss.png')
val_generator.reset()
pred=model.predict_generator(val_generator,steps=val_generator.n/batch_size,verbose=1)
predicted_class_indices=np.argmax(pred,axis=1)
labels = (train_generator.class_indices)
labels = dict((v,k) for k,v in labels.items())
y_pred=model.predict(val_generator,steps=val_generator.n/batch_size,verbose=1)
y_true = val_generator.classes
y_pred=np.argmax(y_pred, axis=1)
from sklearn.metrics import classification_report,accuracy_score,f1_score
print('accuracy_score:',accuracy_score(y_true,y_pred))
print('f1_score:',f1_score(y_true,y_pred,average='macro'))
class_label=list(train_generator.class_indices.keys())
print(classification_report(y_true, _pred,target_names=class_label))
class_names = val_generator.class_indices.keys()
#save model
model.save(model_name+'.h5')

and calculate the accuracy_score for validation dataset and prediction

val_generator.reset()
pred=model.predict_generator(val_generator,steps=val_generator.n/batch_size,verbose=1)
predicted_class_indices=np.argmax(pred,axis=1)
labels = (train_generator.class_indices)
labels = dict((v,k) for k,v in labels.items())
y_pred=model.predict(val_generator,steps=val_generator.n/batch_size,verbose=1)
y_true = val_generator.classes
y_pred=np.argmax(y_pred, axis=1)
from sklearn.metrics import classification_report,accuracy_score,f1_score
print('accuracy_score:',accuracy_score(y_true,y_pred))
print('f1_score:',f1_score(y_true,y_pred,average='macro'))

Prediction Final Model and testing for unseen dataset

def model_predict(img_path, model_path):
MODEL_PATH = model_path
print('Model loading...')
model=load_model(MODEL_PATH)
print('Model loaded. Started serving...')
image = cv2.imread(img_path)
image = cv2.resize(image, (img_shape,img_shape))
#image = image.astype("float") / 255.0
image = img_to_array(image)
image = np.expand_dims(image, axis=0)
proba = model.predict(image)
confidence = proba.max() * 100
print('Class Name:',proba)
print('Confidence:',confidence)

and finally you can test for another efficientneXX model series by using

dataset_path='/content/COVID-19 Radiography Database'
model_name='efficientnetb0'
epochs=10
batch_size=48
no_class=3
img_shape=224
model_train(dataset_path,model_name, epochs,batch_size,no_class,img_shape)

precision    recall  f1-score   support         COVID-19       0.68      1.00      0.81        65          NORMAL       0.97      0.87      0.92       402 Viral Pneumonia       0.93      0.96      0.94       403         accuracy                           0.92       870       macro avg       0.86      0.94      0.89       870    weighted avg       0.93      0.92      0.92       870

using another model for MobileNetV2

dataset_path='/content/COVID-19 Radiography Database'
model_name='mobilenetv2'
epochs=10
batch_size=16
no_class=3
img_shape=224

The Result for MobileNetV2

precision    recall  f1-score   support         COVID-19       0.90      0.72      0.80        65          NORMAL       0.98      0.33      0.49       402 Viral Pneumonia       0.59      1.00      0.74       403         accuracy                           0.67       870       macro avg       0.82      0.68      0.68       870    weighted avg       0.79      0.67      0.63       870

Confusion Matrix for metrics (Confusion,Precision,Recall Matrix)

Normalize Confusion Matrix

get a random batch of images and predict the images on the model

Result

we see the Efficientnet B0 is better and best than MobileNetV2
in parameters size and accuracy for both of them .

all the code in google colab

References

[1]https://www.dlology.com/blog/transfer-learning-with-efficientnet/

[2]Kaggle

[3]https://arxiv.org/abs/1905.11946

[4]https://www.kaggle.com/tawsifurrahman/covid19-radiography-database

[5]https://www.worldometers.info/coronavirus/