Classification Covid-19 X-Ray Images

Using Transfer learning with EfficientNet and MobileNetV2

Photo by engin akyurt on Unsplash

What is Covid -19 ?

Coronaviruses are a group of viruses that can cause disease, which can vary from colds and coughs to sometimes more serious diseases. Middle East Respiratory Syndrome (MERS-CoV) and SARS-CoV were severe cases the world has already faced. SARS-CoV-2 (n-coronavirus) is the new virus for the coronavirus family, first discovered in 2019, which has not been identified in humans before. It is a persistent virus that started from Wuhan in December 2019. It was later declared a pandemic by the World Health Organization because of the high prevalence rate worldwide. Currently (on May 15, 2020), this results in a total of 300,000 deaths worldwide, including 159,000 deaths alone in Europe. It becomes important to understand this spread. Here we explain the importance of deep learning in computer vision in the problem of classification of these images and knowing which model is the best in accuracy

why transfer learning ?

In this post , you will learn how to classify images of Covide-19 and virtual and normal health images for human lungs by using transfer learning from a pre-trained network.A pre-trained model is a saved network that was previously trained on a large dataset, typically on a large-scale image-classification task. You either use the pre-trained model as is or use transfer learning to customize this model to a given task.The intuition behind transfer learning for image classification is that if a model is trained on a large and general enough dataset, this model will effectively serve as a generic model of the visual world. You can then take advantage of these learned feature maps without having to start from scratch by training a large model on a large dataset.

Kaggle is a platform to compete with others in competitions which are based on machine learning tasks. You may know about Codechef, Hackerrank etc., kaggle is also like them, but the key difference is the competition is only related to machine learning, data science, Deep learning or AI related. Most of the time you have been given some training and testing dataset to build some good machine learning models and when you publicize your kernel , others will review it and then give you up votes just like Quora.

Why EfficientNet?

Compared to other models achieving similar ImageNet accuracy, EfficientNet is much smaller. For example, the ResNet50 model has 23,534,592 parameters in total, and even though it still under performs the smallest EfficientNet, which only takes 5,330,564 parameters in total.Why is it so efficient? To answer the question, we will dive into its base model and building block. You might have heard of the building block for the classical ResNet model is identity and convolution block.For EfficientNet, its main building block is the mobile inverted bottleneck MBConv, which was first introduced in MobileNetV2. By using shortcuts directly between the bottlenecks which connects a much fewer number of channels compared to expansion layers, combined with depthwise separable convolution which effectively reduces computation by almost a factor of k2, compared to traditional layers. Where k stands for the kernel size, specifying the height and width of the 2D convolution window.

and you can see, starting from the smallest EfficientNet configuration B0 to the largest B7, accuracies are steadily increasing while maintaining a relatively small size.

Transfer learning for image classification is more or less model agnostic. You can pick any other pre-trained ImageNet model such as MobileNetV2 or ResNet50 as a drop-in replacement if you want.A pre-trained network is simply a saved network previously trained on a large dataset such as ImageNet. The learned features can prove useful for many different computer vision problems, even though these new problems might involve completely different classes from those of the original task.The EfficientNet is built for ImageNet classification and contains 1000 classes labels. For our dataset, we only have 2. Which means the last few layers for classification is not useful for us. They can be excluded while loading the model by specifying the include_top argument to False, and this applies to other ImageNet models made available in Keras applications as well.We expect the model to perform well in our Covide19-X Ray dataset to classify it if it detects someone’s lungs are infected with Covid-19 or not infected by X-ray scanning images. Covid-19 X Ray classification problem with a relatively small number of samples, the easiest way to start is to open this notebook in Colab, while I will explain more details here in this post.First go to kaggle website and copy the api to the dataset. Of course you must have an account with kaggle services ….Go to this link for the data set outside

and copy api command for dataset

A database classification labeling for chest X-Ray images was created for three cases classified as COVID-19 positive cases in addition to regular and Viral Pneumonia images. There are 219 positive images of COVID-19, 1341 Normal images, and 1345 pictures of Viral Pneumonia.

Ok copy this API command from above image

kaggle datasets download -d tawsifurrahman/covid19-radiography-database

In google colab notebook the command will be

!kaggle datasets download -d awsifurrahman/covid19-radiography-database

and write this command

from google.colab import filesfiles.upload()!ls -lha kaggle.json!pip install -q kaggle!mkdir -p ~/.kaggle!cp kaggle.json ~/.kaggle/!chmod 600 ~/.kaggle/kaggle.json!kaggle datasets download -d awsifurrahman/covid19-radiography-database# unzip folder dataset!unzip /content/covid19-radiography-database.zip

to display some images from dataset

#load imagesdef show_images(dataset_path):# size of the image: 48*48 pixelspic_size = 48# input path for the imagesbase_path = dataset_pathplt.figure(0, figsize=(12,20))#cpt = 0for expression in os.listdir(base_path + "/"):for i in range(25):# cpt = cpt + 1plt.subplot(5,5,i+1)img = load_img(base_path + "/" + expression + "/" +os.listdir(base_path + "/" + expression)[i], target_size=(pic_size, pic_size))plt.imshow(img, cmap="gray")plt.tight_layout()plt.show()
#pip command install EfficientNet model by using!pip install efficientnet
#Imported libraries and modulesimport efficientnet.keras as efnfrom sklearn.metrics import classification_report,accuracy_score,f1_score,confusion_matriximport numpy as npfrom keras.preprocessing.image import load_img, img_to_arrayimport matplotlib.pyplot as pltimport osfrom keras.models import Modelfrom keras.utils import np_utilsimport tensorflow as tffrom tensorflow.keras.models import load_modelfrom keras.utils.np_utils import to_categoricalfrom keras.preprocessing.image import ImageDataGeneratorfrom keras.layers import Activation,Dense, Dropout, Flatten, Conv2Dfrom keras.layers import  GlobalAveragePooling2D,BatchNormalization
img_shape=224import efficientnet.keras as efn
#using here EfficientNet series BO
baseModel =efn.EfficientNetB0(weights ='noisy-student', include_top=False, input_shape = (img_shape,img_shape,3))baseModel.summary()

To create our own classification layers stack on top of the EfficientNet convolutional base model. We adapt GlobalAveragePooling2D . GlobalMaxPooling2D results in a much smaller number of features compared to the Flatten layer, which effectively reduces the number of parameters.

x = baseModel.outputx = GlobalAveragePooling2D()(x)x = Dropout(0.3)(x)x = Dense(128, activation=”relu”)(x)x = Dropout(0.3)(x)x = Dense(64, activation=”relu”)(x)predictions = Dense(3, activation=”softmax”)(x)model = Model(inputs=baseModel.input, outputs=predictions)
for layer in baseModel.layers:
layer.trainable = False

you can compile and train the model with Keras’s ImageDataGenerator, which adds various data augmentation options during the training to reduce the chance of overfitting.

data_gen= ImageDataGenerator(horizontal_flip=True,vertical_flip=True,rotation_range=90,width_shift_range=0.1,height_shift_range=0.1,zoom_range=.1,rescale=1/255,fill_mode=’nearest’,shear_range=0.1,brightness_range=[0.5, 1.5],validation_split=0.3)

Note that the validation data should not be augmented!

test_datagen = ImageDataGenerator(rescale=1.0 / 255)train_generator = train_datagen.flow_from_directory(# This is the target directorytrain_dir,# All images will be resized to target height and width.target_size=(height, width),batch_size=batch_size,# Since we use categorical_crossentropy loss, we need categorical labelsclass_mode=”categorical”,)validation_generator = test_datagen.flow_from_directory(validation_dir,target_size=(height, width),batch_size=batch_size,class_mode=”categorical”,)

Then you can compile and train the model again for some more epochs.

model.compile(optimizer='adam', loss="categorical_crossentropy", metrics=['accuracy'])

final fit the model

results = model.fit(train_generator,epochs=epochs,steps_per_epoch=train_generator.n/batch_size,validation_data=val_generator,validation_steps=val_generator.n/batch_size)

So we can built function for pretrained efficient model for every model from B0 to B7 for testing all of them models to our dataset

def model_efficientnetb0(no_classes):#mport Modelimg_shape=224import efficientnet.keras as efnmodel =efn.EfficientNetB0(weights ='noisy-student', include_top=False, input_shape = (img_shape,img_shape,3))x = model.outputx = GlobalAveragePooling2D()(x)x = Dropout(0.3)(x)x = Dense(128, activation="relu")(x)x = Dropout(0.3)(x)x = Dense(64, activation="relu")(x)predictions = Dense(no_classes, activation="softmax")(x)model = Model(inputs=model.input, outputs=predictions)return modeldef model_efficientnetb7(no_classes):#mport Modelimg_shape=600model =efn.EfficientNetB7(weights ='imagenet', include_top=False, input_shape = (img_shape,img_shape,3))x = model.outputx = GlobalAveragePooling2D()(x)x = Dropout(0.3)(x)x = Dense(128, activation="relu")(x)x = Dropout(0.3)(x)x = Dense(64, activation="relu")(x)predictions = Dense(no_classes, activation="softmax")(x)model = Model(inputs=model.input, outputs=predictions)return model

we can see summary for model

no_classes=3model=model_efficientnetb0(no_classes)model.summary()

and also built all of these in funcion

def model_train(dataset_path,model_name,epochs,batch_size,no_class,img_shape):show_images(dataset_path)no_classes=no_classdata_gen= ImageDataGenerator(horizontal_flip=True,vertical_flip=True,rotation_range=90,width_shift_range=0.1,height_shift_range=0.1,zoom_range=.1,rescale=1/255,fill_mode='nearest',shear_range=0.1,brightness_range=[0.5, 1.5],validation_split=0.3)#load the training datatrain_generator = data_gen.flow_from_directory(dataset_path,target_size=(img_shape,img_shape),batch_size=batch_size,class_mode='categorical',subset='training',shuffle=True)#load the training dataval_suffle=Falseval_generator = data_gen.flow_from_directory(dataset_path,target_size=(img_shape,img_shape),batch_size=batch_size,class_mode='categorical',subset='validation',shuffle=val_suffle)train_generator.next()[0].shape,train_generator.next()[1].shapeval_generator.next()[0].shape,val_generator.next()[1].shapex,y = train_generator.next()plt.figure(figsize=(10,10))for i in range(8):plt.subplot(4,2,i+1)image = x[i]label = y[i]print (label)plt.imshow(image)plt.show()if model_name=='efficientnetb0':model=model_efficientnetb0(no_classes)if model_name=='efficientnetb7':model=model_efficientnetb7(no_classes)model.compile(optimizer='adam', loss="categorical_crossentropy", metrics=['accuracy'])results = model.fit(train_generator,epochs=epochs,steps_per_epoch=train_generator.n/batch_size,validation_data=val_generator,validation_steps=val_generator.n/batch_size)no_epochs=int(epochs)plt.plot(results.history['accuracy'])plt.plot(results.history['val_accuracy'])plt.title(model_name+' Model Accuracy ')plt.ylabel('accuracy')plt.xlabel('epoch')plt.legend(['train', 'test'], loc='upper left')plt.savefig(model_name+'_accuracy.png')plt.show()plt.savefig(model_name+'_accuracy.png')plt.plot(results.history['loss'])plt.plot(results.history['val_loss'])plt.title(model_name+' Model Loss')plt.ylabel('loss')plt.xlabel('epoch')plt.legend(['train', 'test'], loc='upper left')plt.savefig(model_name+'_loss.png')plt.show()plt.savefig(model_name+'_loss.png')val_generator.reset()pred=model.predict_generator(val_generator,steps=val_generator.n/batch_size,verbose=1)predicted_class_indices=np.argmax(pred,axis=1)labels = (train_generator.class_indices)labels = dict((v,k) for k,v in labels.items())y_pred=model.predict(val_generator,steps=val_generator.n/batch_size,verbose=1)y_true = val_generator.classesy_pred=np.argmax(y_pred, axis=1)from sklearn.metrics import classification_report,accuracy_score,f1_scoreprint('accuracy_score:',accuracy_score(y_true,y_pred))print('f1_score:',f1_score(y_true,y_pred,average='macro'))class_label=list(train_generator.class_indices.keys())print(classification_report(y_true, _pred,target_names=class_label))class_names = val_generator.class_indices.keys()#save modelmodel.save(model_name+'.h5')

and calculate the accuracy_score for validation dataset and prediction

val_generator.reset()pred=model.predict_generator(val_generator,steps=val_generator.n/batch_size,verbose=1)predicted_class_indices=np.argmax(pred,axis=1)labels = (train_generator.class_indices)labels = dict((v,k) for k,v in labels.items())y_pred=model.predict(val_generator,steps=val_generator.n/batch_size,verbose=1)y_true = val_generator.classesy_pred=np.argmax(y_pred, axis=1)from sklearn.metrics import classification_report,accuracy_score,f1_scoreprint('accuracy_score:',accuracy_score(y_true,y_pred))print('f1_score:',f1_score(y_true,y_pred,average='macro'))

Prediction Final Model and testing for unseen dataset

def model_predict(img_path, model_path):MODEL_PATH = model_pathprint('Model loading...')model=load_model(MODEL_PATH)print('Model loaded. Started serving...')image = cv2.imread(img_path)image = cv2.resize(image, (img_shape,img_shape))#image = image.astype("float") / 255.0image = img_to_array(image)image = np.expand_dims(image, axis=0)proba = model.predict(image)confidence = proba.max() * 100print('Class Name:',proba)print('Confidence:',confidence)

and finally you can test for another efficientneXX model series by using

dataset_path='/content/COVID-19 Radiography Database'model_name='efficientnetb0'epochs=10batch_size=48no_class=3img_shape=224model_train(dataset_path,model_name, epochs,batch_size,no_class,img_shape)
precision    recall  f1-score   support         COVID-19       0.68      1.00      0.81        65          NORMAL       0.97      0.87      0.92       402 Viral Pneumonia       0.93      0.96      0.94       403         accuracy                           0.92       870       macro avg       0.86      0.94      0.89       870    weighted avg       0.93      0.92      0.92       870

using another model for MobileNetV2

dataset_path='/content/COVID-19 Radiography Database'model_name='mobilenetv2'epochs=10batch_size=16no_class=3img_shape=224

The Result for MobileNetV2

precision    recall  f1-score   support         COVID-19       0.90      0.72      0.80        65          NORMAL       0.98      0.33      0.49       402 Viral Pneumonia       0.59      1.00      0.74       403         accuracy                           0.67       870       macro avg       0.82      0.68      0.68       870    weighted avg       0.79      0.67      0.63       870

Confusion Matrix for metrics (Confusion,Precision,Recall Matrix)

Normalize Confusion Matrix

Result

all the code in google colab

References

[1]https://www.dlology.com/blog/transfer-learning-with-efficientnet/

[2]Kaggle

[3]https://arxiv.org/abs/1905.11946

[4]https://www.kaggle.com/tawsifurrahman/covid19-radiography-database

[5]https://www.worldometers.info/coronavirus/

Developer Programmer, in Python and deep learning. IOT Microcontroller Developer iraqprogrammer.wordpress.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store