In this article, we will provide an overview of the process and techniques involved in creating a tailored Convolutional Neural Network (CNN) model for the purpose of binary picture classification. Specifically, we will focus on constructing the model and choosing the appropriate loss function for a particular application that involves using Python programming to diagnose pneumothorax (i.e., the collapse of a lung due to the presence of air or gas in the space between the lungs and the chest wall) from patient chest X-ray images. Additionally, we will highlight the importance of medical image databases, the significance of image segmentation, and the unique challenges associated with utilising deep learning in healthcare.
Acquiring Datasets
To perform this exercise, we obtained the dataset from Kaggle. The dataset can be easily downloaded and unzipped locally, and it contains a folder with PNG images of both normal and pneumothorax chest X-rays. Additionally, we require the stage1 train images.csv and stage1 test images.csv CSV files for the training and evaluation phases, respectively.
Note: As there was a shortage of pneumothorax images in the initial dataset, it is advisable to create two copies of each training image.
The task of sorting the images into their respective categories is now at hand, as they are not pre-organised into individual folders. Fortunately, the two .csv files that accompany the images contain valuable information, such as the filenames and whether each image depicts a pneumothorax or not. In order to write the code for image tracking, this data is critical. Once this information has been obtained, the images must be sorted into their respective folders to create the necessary folder structure for the custom dataset. This will enable image classification to be carried out in the usual way.
Once completed, the folder structure should appear as follows:
Project Folder:
- train
- Non-pneumothorax (Negative)
- pneumothorax
- test
- Non-pneumothorax (Negative)
- pneumothorax
The distinct colours of each folder make it effortless to keep tabs on their hierarchy.
Due to the extensive size of the dataset, it is recommended to execute the project locally using Jupyter Notebook. In the case of using Google Colab, the Google Drive account must have enough storage capacity to store roughly 16,000 chest X-ray images.
The process has been initiated. To start, access the train CSV file, which can be read with the pandas program just like any other CSV file.
Import Pandas (in the pd format)
df=pd.read_csv(‘/content/drive/MyDrive/Pneumothorax
Project/stage 1 Train images.csv’)
PNEUMOTHORAX = 0 means there is NO PNEUMOTHORAX, while PNEUMOTHORAX = 1 means the opposite is true. A list should now be created that distinguishes between images containing pneumothorax and those that do not.
no = list(df[df[‘has pneumo’]==0][‘new filename’])
yes = list(df[df[‘has pneumo’]==1][‘new filename’])
Photos containing and not containing pneumothorax should be saved in the following locations:
‘png images/’ should be replaced with ‘/content/drive/MyDrive/Pneumothorax project/’
Image path for no collapsed lungs should be set to ‘/content/drive/MyDrive/Pneumothorax plan/exercise/no collapsed lung’
Image path for pneumothorax-containing images should be set to ‘/content/drive/MyDrive/Pneumothorax project/train/pneumothorax/’
Installation of the shutil package is necessary for copying the PNG images extracted from Kaggle to a new directory. The first step is to make sure that the directory where the extracted images are stored is specified.
Importing shutil
The code below will move all the PNG images in the ‘train’ folder that are pneumothorax-free to the ‘no pneumothorax’ folder.
To replace the letter ‘I’ with the letter ‘n’:
src_path = path + ‘i’
dest_path = img_path_no + ‘i’
For instance: temp = shutil.move(src_path, dest_path)
Print the message “No folder created and relevant images have been moved”.
Use the same method to move images containing pneumothorax to the ‘pneumothorax’ folder.
If ‘i’ represents a pneumothorax-containing image, set the source path to path + i, and the destination path to img_path_yes + i.
Replace ‘temp’ with shutil.move(src_path, dest_path).
Print the message “Pneumothorax folder created and relevant images have been moved”.
You can follow the same procedures to extract images from the ‘test’ dataset if you have the ‘test.csv’ file. The code for this is shown below:
To view the results of the initial phase of the Pneumothorax Project, run the following command in the command line: Df = pd.read_csv(“/content/drive/MyDrive/Pneumothorax Project/stage 1 Test images.csv”).
Images with no pneumothorax will be stored in the list called ‘no’, while images with pneumothorax will be stored in a list called ‘yes’.
Ensure that “png images/” is replaced with “/content/drive/MyDrive/Pneumothorax project/”.
If the image path has not been found, set img_path_no to ‘/content/drive/MyDrive/Pneumothorax project/test/no pneumothorax/’.
If the image path for pneumothorax-containing images has not been found, set img_path_yes to ‘/content/drive/MyDrive/Pneumothorax project/test/pneumothorax/’.
To replace ‘i’ with ‘n’ in the list containing no pneumothorax images, set the source path to path + i and the destination path to img_path_no + i. Then replace ‘temp’ with shutil.move(src_path, dest_path).
Print the message “No folder created and relevant images have been moved” after the images have been moved.
To replace ‘i’ with ‘s’ in the list containing pneumothorax images, set the source path to path + i and the destination path to img_path_yes + i. Then replace ‘temp’ with shutil.move(src_path, dest_path).
Print the message “Pneumothorax folder created and relevant images have been moved” after the images have been moved.
When working with the test dataset, use the ‘test’ folder instead of the ‘train’ folder.
Pre-data entry procedures
Install the required add-ons:
To start, import the following libraries from tensorflow.keras.layers: Input, Lambda, Dense, Flatten, Dropout, Conv2D, MaxPooling2D, and BatchNormalization.
Next, import the Model class from tensorflow.keras.models.
Import the Image class from tensorflow.keras.preprocessing.
To import the required evaluation metrics, import accuracy_score, classification_report, and confusion_matrix from sklearn.metrics.
Import ImageDataGenerator from ‘tensorflow.keras.preprocessing.image’.
Import the train_test_split function from sklearn.model_selection.
Import Sequential from tensorflow.keras.models.
Substitute ‘np’ with ‘import numpy’.
Import Pandas library as ‘pd’.
Import os.
Import cv2.
Use ‘import matplotlib.pyplot as plt’ to import the Python plotting library.
Resize all images to 50×50 pixels (note that the image size can be different).
Resize all images to this size.
When the image size is (50,50)
We will use OpenCV (cv2) to read images from both the train and test folders. Next, we extract the array representations of these images and store them in x_train and x_test lists. These lists will serve as inputs in our Convolutional Neural Network model.
Initialize two empty lists, x_train and x_test.
For ‘train’ path, set path=’Pneumothorax Project/Train’.
For ‘test’ path, set path=’Pneumothorax Project/Test’.
Required components for training:
While iterating through os.listdir(train_path), if a folder for which c=0 is encountered:
Set sub_path to train_path + “/” + folder.
For each image in os.listdir(sub_path):
Set image_path to sub_path + “/” + img.
Read the image from image_path using cv2.imread and store its array representation in img_arr.
Try to resize img_arr using cv2.resize to IMAGE_SIZE.
If successful, append the resized img_arr to x_train list.
If unsuccessful, increment c and continue.
Print the number of images skipped: “Number of images skipped= ” + c.
In the given test:
Initialize an empty list x_test and set c to 0.
Iterate through each folder in os.listdir(test_path):
Set sub_path to test_path + “/” + folder.
For each image in os.listdir(sub_path):
Set image_path to sub_path + “/” + img.
Read the image from image_path using cv2.imread and store its array representation in img_arr.
Try to resize img_arr using cv2.resize to IMAGE_SIZE.
If successful, append the resized img_arr to x_test list.
If unsuccessful, increment c and continue.
Print the number of images skipped: “Number of images skipped= ” + c.
To perform normalization or other significant pre-processing on image data, convert the lists to NumPy arrays and divide them by 255.
Convert x_test and x_train lists to NumPy arrays using np.array.
Divide x_train and x_test arrays by 255.0 to perform normalization.
Update the values of x_train and x_test arrays to the normalized arrays.
To obtain labels using ImageDataGenerator, follow these steps:
To obtain image labels, create an instance of ImageDataGenerator using datagen = ImageDataGenerator().
Next, set the training dataset using datagen.flow_from_directory(train_path). Ensure that class_mode is set to “binary”.
Similarly, set the test dataset using datagen.flow_from_directory(test_path).
If class_mode is “binary”, perform the following:
Use the following code to encode the labels:
Obtain class indices for the training dataset using train_dataset.class_indices.
For both the training and testing datasets, the value of a label will be 0 if pneumothorax is absent and 1 otherwise.
Assign the following categories to y_train (labels in the training dataset) and y_test (labels in the testing dataset):
Obtain the labels of the training dataset using Y_train = train_dataset.classes.
Similarly, obtain the labels of the testing dataset using Y_test = test_dataset.classes.
Ensure that the training data contains 15433 photos and the test data contains 1372 photos.
Check the shapes of x_train and y_train arrays using x_train.shape and y_train.shape respectively.
Similarly, check the shapes of x_test and y_test arrays using x_test.shape and y_test.shape respectively.
The dimensions displayed as (1372, 50, 50, 3) show that all 1372 downloaded photographs from Kaggle are coloured and have been resized to 50×50 after scaling.
Creating representations
During this process, the model will utilize the features and labels extracted from the preprocessed photos for training.
pConstruct the model like this:
Use model=Sequential() to create a sequential model, which is composed of layers that are directly linked to one another, including an input layer, a hidden layer, and an output layer. Begin constructing the model by adding convolution and pooling layers.
To add a convolutional layer, use model.add(Conv2D(64,(3,3),activation=’relu’,input_shape=(50,50,3))).
Next, add a pooling layer using model.add(MaxPooling2D(2,2)).
Add model.add(BatchNormalization()) after the pooling layer.
Add another convolutional layer using model.add(Conv2D(128,(3,3),activation=’relu’)).
Then, add another pooling layer using model.add(MaxPooling2D(2,2)).
Again, add a BatchNormalization() layer using model.add(BatchNormalization()).
Follow it by adding another convolutional layer using model.add(Conv2D(256,(3,3),activation=’relu’)).
Then add another pooling layer using model.add(MaxPooling2D(2,2)).
Finally, add BatchNormalization() using model.add(BatchNormalization()).
To enhance the performance, three convolutional layers and two batch normalization layers have been added. Complete the first layer:
The Input Layer can be created by adding model.add(Flatten()).
Add the Output Layer, which refers to the last fully connected layer:
Use model.add(Dense(1, activation=’sigmoid’)) to add the Outer/Inner Layer Notation.
Since it’s a binary classification problem, a Sigmoid function was used. A single class is represented by linear output. For a multiclass scenario, instead of “1”, the number of classes involved should be stated.
pTake a peek at the special version, here:
Use model.summary() to obtain a summary of the model.
Include the relevant loss function as part of the model compilation process.
Compile the model by using model.compile(optimizer=’adam’, loss=’binary_crossentropy’, metrics=[‘accuracy’]).
The binary cross-entropy loss function is the most suitable choice for tasks involving only two classes. When choosing the optimizer for the model, Adam is a good option because it automatically calculates the learning rate without requiring manual specification. To prevent overfitting, an early stopping technique can be used, which stops the training process if validation accuracy suddenly drops while training accuracy continues to improve.
The EarlyStopping method can be accessed through tensorflow.keras.callbacks.
Use EarlyStopping(monitor=’val_loss’, mode=’min’, verbose=1, patience=5) to define early stopping.
Try to train the model for 30 epochs.
Use history = model.fit(x_train, y_train, validation_data=(x_test, y_test), epochs=30, callbacks=[early_stop], shuffle=True) to train the model.
Training is stopped after 8 epochs to prevent overfitting.
Results
The model achieved an outstanding accuracy rate of 99.04%. To assess the model’s performance in greater depth, the confusion matrix and classification report from sklearn can be utilized to analyze the test data results.
The graphs presented indicate that the model’s performance is improving over time. It’s worth noting that the test line displays significant fluctuation, whereas training accuracy and loss improve consistently over time.
Make precise predictions for the entire test set:
Use y_pred = model.predict(x_test) and y_pred = np.argmax(y_pred, axis=1) to make predictions for the test set.
Calculate the model’s reliability rating (VALIDATION ACCURACY):
Calculate the accuracy score using accuracy_score(y_pred, y_test)
The final score is 78.86%. To obtain a detailed overview of the performance, the code below can be used to create a confusion matrix and classification report:
Print the classification report using print(classification_report(y_pred, y_test))
The model’s validation accuracy was found to be 79%. Nevertheless, this model can only be applied to images that do not contain pneumothorax, and it cannot identify the presence of the ailment in new, unseen instances.
As the subsequent confusion matrix shows:
Generate the confusion matrix using confusion_matrix(y_pred, y_test)
We could not locate any images containing pneumothorax.
pFor samples that have not yet been seen, the model consistently predicts the “no-pneumothorax” class. However, when exposed to the training data, it performs satisfactorily. It is possible that this is due to the fact that the test and training sets exhibit different characteristics as they have not been sourced from the same distribution or the same source of pictures. When the model does not perform well on the test data, this should raise a red flag and be a cause for concern.
It is conceivable that a barely visible indication of pneumothorax on a chest X-ray could be a contributing element. Nonetheless, it can be difficult to collect medical data such as X-rays, CT scans, and other information from unwell patients in contrast to healthy individuals, where such data is relatively more obtainable. Furthermore, pneumothorax is itself an infrequent occurrence.
The collection of samples from patients necessitates the utmost respect for their right to confidentiality. Acquiring data from hospitals can often be a challenging and protracted procedure. Segmentation, a method for separating an image into distinct sections based on similarities between adjacent pixels, can be utilised to improve the effectiveness of the model.
The Convolutional Neural Network (CNN) can extract information from every individual pixel of an image. However, in the present scenario, the objective is to train the model to concentrate on relevant features connected to medical imaging. The purpose of this project is to furnish the model with the ability to recognise features in the portion of the image where the pneumothorax infection is located (between the lungs and chest wall). To assist in this effort, masks have been included in the zip file of the original dataset.
Masks can be utilised to segment the chest X-ray images into separate parts. Subsequently, these images can be utilised to train a Convolutional Neural Network (CNN) model. The model can also be further enhanced if required. Alternatively, pre-trained architectures like VGG-19 or Mobile Net can be examined to determine their efficacy in producing better outcomes.
