Object Detection with TensorFlow

Object detection is pivotal in synthetic intelligence, serving because the spine for quite a few cutting-edge functions. From autonomous autos and surveillance techniques to medical imaging and augmented actuality, the flexibility to establish and find objects in pictures and movies is remodeling industries worldwide. TensorFlow’s Object Detection API, a strong and versatile instrument, simplifies constructing sturdy object detection fashions. By leveraging this API, builders can practice customized fashions tailor-made to particular wants, considerably lowering improvement time and complexity.

On this information, we’ll discover the step-by-step course of of coaching an object detection mannequin utilizing TensorFlow, specializing in integrating datasets from Roboflow Universe, a wealthy repository of annotated datasets designed to speed up AI improvement.

Studying Aims

• Be taught to arrange and configure TensorFlow‘s Object Detection API setting for environment friendly mannequin coaching.
• Perceive the way to put together and preprocess datasets for coaching, utilizing the TFRecord format.
• Acquire experience in deciding on and customizing a pre-trained object detection mannequin for particular wants.
• Be taught to regulate pipeline configuration information and fine-tune mannequin parameters to optimize efficiency.
• Grasp the coaching course of, together with dealing with checkpoints and evaluating mannequin efficiency throughout coaching.
• Perceive the way to export the skilled mannequin for inference and deployment in real-world functions.

This text was revealed as part of the Information Science Blogathon.

Step-By-Step Implementation of Object Detection with TensorFlow

On this part, we’ll stroll you thru a step-by-step implementation of object detection utilizing TensorFlow, guiding you from setup to deployment.

Step1: Setting Up the Surroundings

The TensorFlow Object Detection API requires numerous dependencies. Start by cloning the TensorFlow fashions repository:

# Clone the tensorflow fashions repository from GitHub
!pip uninstall Cython -y # Momentary repair for "No module named 'object_detection'" error
!git clone --depth 1 https://github.com/tensorflow/fashions

• Uninstall Cython: This step ensures there are not any conflicts with the Cython library throughout setup.
• Clone TensorFlow Fashions Repository: This repository incorporates TensorFlow’s official fashions, together with the Object Detection API.

Copy the Setup Information and Modify the setup.py File

# Copy setup information into fashions/analysis folder
%%bash
cd fashions/analysis/
protoc object_detection/protos/*.proto --python_out=.
#cp object_detection/packages/tf2/setup.py .

# Modify setup.py file to put in the tf-models-official repository focused at TF v2.8.0
import re
with open('/content material/fashions/analysis/object_detection/packages/tf2/setup.py') as f:
    s = f.learn()

with open('/content material/fashions/analysis/setup.py', 'w') as f:
    # Set fine_tune_checkpoint path
    s = re.sub('tf-models-official>=2.5.1',
               'tf-models-official==2.8.0', s)
    f.write(s)

Why is This Obligatory?

• Protocol Buffers Compilation: The Object Detection API makes use of .proto information to outline mannequin configurations and knowledge buildings. These have to be compiled into Python code to operate.
• Dependency Model Compatibility: TensorFlow and its dependencies evolve. Utilizing tf-models-official>=2.5.1 might inadvertently set up an incompatible model for TensorFlow v2.8.0.
• Explicitly setting tf-models-official==2.8.0 avoids potential model conflicts and ensures stability.

Putting in dependency libraries

TensorFlow fashions typically depend on particular library variations. Fixing the TensorFlow model ensures easy integration.

# Set up the Object Detection API 

# Have to do a brief repair with PyYAML as a result of Colab is not capable of set up PyYAML v5.4.1
!pip set up pyyaml==5.3
!pip set up /content material/fashions/analysis/

# Have to downgrade to TF v2.8.0 because of Colab compatibility bug with TF v2.10 (as of 10/03/22)
!pip set up tensorflow==2.8.0

# Set up CUDA model 11.0 (to keep up compatibility with TF v2.8.0)
!pip set up tensorflow_io==0.23.1
!wget https://developer.obtain.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
!mv cuda-ubuntu1804.pin /and so on/apt/preferences.d/cuda-repository-pin-600
!wget http://developer.obtain.nvidia.com/compute/cuda/11.0.2/local_installers/cuda-repo-ubuntu1804-11-0-local_11.0.2-450.51.05-1_amd64.deb
!dpkg -i cuda-repo-ubuntu1804-11-0-local_11.0.2-450.51.05-1_amd64.deb
!apt-key add /var/cuda-repo-ubuntu1804-11-0-local/7fa2af80.pub
!apt-get replace && sudo apt-get set up cuda-toolkit-11-0
!export LD_LIBRARY_PATH=/usr/native/cuda-11.0/lib64:$LD_LIBRARY_PATH

Whereas working this block it’s essential to restart the periods once more and run this block of code once more to efficiently set up all dependencies. This can set up all of the dependencies efficiently.

Putting in an acceptable model of protobuf library for resolving dependency points

!pip set up protobuf==3.20.1

Step2: Confirm Surroundings and Installations

To verify the set up works, run the next check:

# Run Mannequin Bulider Check file, simply to confirm every thing's working correctly
!python /content material/fashions/analysis/object_detection/builders/model_builder_tf2_test.py

If no errors seem, your setup is full. So now we have now accomplished the setup efficiently. 

Step3: Put together the Coaching Information

For this tutorial, we’ll use the “Individuals Detection” dataset from Roboflow Universe. Observe these steps to organize it:

Go to the dataset web page:

Fork the dataset into your workspace to make it accessible for personalisation.

Generate a model of the dataset to finalize its preprocessing configurations comparable to augmentation and resizing.

Now , Obtain it in TFRecord format, which is a binary format optimized for TensorFlow workflows. TFRecord shops knowledge effectively and permits TensorFlow to learn massive datasets throughout coaching with minimal overhead.

As soon as downloaded, place the dataset information in your Google Drive mount your code to your drive, and cargo these information within the code to make use of it.

from google.colab import drive
drive.mount('/content material/gdrive')


train_record_fname="/content material/gdrive/MyDrive/pictures/practice/practice.tfrecord"
val_record_fname="/content material/gdrive/MyDrive/pictures/check/check.tfrecord"
label_map_pbtxt_fname="/content material/gdrive/MyDrive/pictures/label_map.pbtxt"

Step4: Set Up the Coaching Configuration

Now, it’s time to arrange the configuration for the item detection mannequin. For this instance, we’ll use the efficientdet-d0 mannequin. You may select from different fashions like ssd-mobilenet-v2 or ssd-mobilenet-v2-fpnlite-320, however for this information, we’ll concentrate on efficientdet-d0.

# Change the chosen_model variable to deploy completely different fashions out there within the TF2 object detection zoo
chosen_model="efficientdet-d0"

MODELS_CONFIG = {
    'ssd-mobilenet-v2': {
        'model_name': 'ssd_mobilenet_v2_320x320_coco17_tpu-8',
        'base_pipeline_file': 'ssd_mobilenet_v2_320x320_coco17_tpu-8.config',
        'pretrained_checkpoint': 'ssd_mobilenet_v2_320x320_coco17_tpu-8.tar.gz',
    },
    'efficientdet-d0': {
        'model_name': 'efficientdet_d0_coco17_tpu-32',
        'base_pipeline_file': 'ssd_efficientdet_d0_512x512_coco17_tpu-8.config',
        'pretrained_checkpoint': 'efficientdet_d0_coco17_tpu-32.tar.gz',
    },
    'ssd-mobilenet-v2-fpnlite-320': {
        'model_name': 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8',
        'base_pipeline_file': 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.config',
        'pretrained_checkpoint': 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz',
    },
}

model_name = MODELS_CONFIG[chosen_model]['model_name']
pretrained_checkpoint = MODELS_CONFIG[chosen_model]['pretrained_checkpoint']
base_pipeline_file = MODELS_CONFIG[chosen_model]['base_pipeline_file']

We then obtain the pre-trained weights and the corresponding configuration file for the chosen mannequin:

# Create "mymodel" folder for holding pre-trained weights and configuration information
%mkdir /content material/fashions/mymodel/
%cd /content material/fashions/mymodel/

# Obtain pre-trained mannequin weights
import tarfile
download_tar="http://obtain.tensorflow.org/fashions/object_detection/tf2/20200711/" + pretrained_checkpoint
!wget {download_tar}
tar = tarfile.open(pretrained_checkpoint)
tar.extractall()
tar.shut()

# Obtain coaching configuration file for mannequin
download_config = 'https://uncooked.githubusercontent.com/tensorflow/fashions/grasp/analysis/object_detection/configs/tf2/' + base_pipeline_file
!wget {download_config}

After this, we arrange the variety of steps for coaching and batch measurement based mostly on the mannequin chosen:

# Set coaching parameters for the mannequin
num_steps = 4000

if chosen_model == 'efficientdet-d0':
  batch_size = 8
else:
  batch_size = 8

You may enhance and reduce num_steps and batch_size in keeping with your necessities.

Step5: Modify the Pipeline Configuration File

We have to customise the pipeline.config file with the paths to our dataset and mannequin parameters. The pipeline.config file incorporates numerous configurations such because the batch measurement, variety of courses, and fine-tuning checkpoints. We make these modifications by studying the template and changing the related fields:

# Set file areas and get variety of courses for config file
pipeline_fname="/content material/fashions/mymodel/" + base_pipeline_file
fine_tune_checkpoint="/content material/fashions/mymodel/" + model_name + '/checkpoint/ckpt-0'

def get_num_classes(pbtxt_fname):
    from object_detection.utils import label_map_util
    label_map = label_map_util.load_labelmap(pbtxt_fname)
    classes = label_map_util.convert_label_map_to_categories(
        label_map, max_num_classes=90, use_display_name=True)
    category_index = label_map_util.create_category_index(classes)
    return len(category_index.keys())
num_classes = get_num_classes(label_map_pbtxt_fname)
print('Whole courses:', num_classes)

# Create customized configuration file by writing the dataset, mannequin checkpoint, and coaching parameters into the bottom pipeline file
import re

%cd /content material/fashions/mymodel
print('writing customized configuration file')

with open(pipeline_fname) as f:
    s = f.learn()
with open('pipeline_file.config', 'w') as f:

    # Set fine_tune_checkpoint path
    s = re.sub('fine_tune_checkpoint: ".*?"',
               'fine_tune_checkpoint: "{}"'.format(fine_tune_checkpoint), s)

    # Set tfrecord information for practice and check datasets
    s = re.sub(
        '(input_path: ".*?)(PATH_TO_BE_CONFIGURED/practice)(.*?")', 'input_path: "{}"'.format(train_record_fname), s)
    s = re.sub(
        '(input_path: ".*?)(PATH_TO_BE_CONFIGURED/val)(.*?")', 'input_path: "{}"'.format(val_record_fname), s)

    # Set label_map_path
    s = re.sub(
        'label_map_path: ".*?"', 'label_map_path: "{}"'.format(label_map_pbtxt_fname), s)

    # Set batch_size
    s = re.sub('batch_size: [0-9]+',
               'batch_size: {}'.format(batch_size), s)

    # Set coaching steps, num_steps
    s = re.sub('num_steps: [0-9]+',
               'num_steps: {}'.format(num_steps), s)

    # Set variety of courses num_classes
    s = re.sub('num_classes: [0-9]+',
               'num_classes: {}'.format(num_classes), s)

    # Change fine-tune checkpoint kind from "classification" to "detection"
    s = re.sub(
        'fine_tune_checkpoint_type: "classification"', 'fine_tune_checkpoint_type: "{}"'.format('detection'), s)

    # If utilizing ssd-mobilenet-v2, scale back studying price (as a result of it is too excessive within the default config file)
    if chosen_model == 'ssd-mobilenet-v2':
      s = re.sub('learning_rate_base: .8',
                 'learning_rate_base: .08', s)

      s = re.sub('warmup_learning_rate: 0.13333',
                 'warmup_learning_rate: .026666', s)

    # If utilizing efficientdet-d0, use fixed_shape_resizer as an alternative of keep_aspect_ratio_resizer (as a result of it is not supported by TFLite)
    if chosen_model == 'efficientdet-d0':
      s = re.sub('keep_aspect_ratio_resizer', 'fixed_shape_resizer', s)
      s = re.sub('pad_to_max_dimension: true', '', s)
      s = re.sub('min_dimension', 'top', s)
      s = re.sub('max_dimension', 'width', s)

    f.write(s)


# (Non-obligatory) Show the customized configuration file's contents
!cat /content material/fashions/mymodel/pipeline_file.config

# Set the trail to the customized config file and the listing to retailer coaching checkpoints in
pipeline_file="/content material/fashions/mymodel/pipeline_file.config"
model_dir="/content material/coaching/"

Step6: Practice the Mannequin

Now we will practice the mannequin utilizing the customized pipeline configuration file. The coaching script will save checkpoints, which you need to use to guage the efficiency of your mannequin:

# Run coaching!
!python /content material/fashions/analysis/object_detection/model_main_tf2.py 
    --pipeline_config_path={pipeline_file} 
    --model_dir={model_dir} 
    --alsologtostderr 
    --num_train_steps={num_steps} 
    --sample_1_of_n_eval_examples=1

Step7: Save the Educated Mannequin

After coaching is full, we export the skilled mannequin so it may be used for inference. We use the exporter_main_v2.py script to export the mannequin:

!python /content material/fashions/analysis/object_detection/exporter_main_v2.py 
    --input_type image_tensor 
    --pipeline_config_path {pipeline_file} 
    --trained_checkpoint_dir {model_dir} 
    --output_directory /content material/exported_model

Lastly, we compress the exported mannequin into a zipper file for simple downloading after which you possibly can obtain the zip file containing your skilled mannequin:

import shutil
# Path to the exported mannequin folder
exported_model_path="/content material/exported_model"

# Path the place the zip file shall be saved
zip_file_path="/content material/exported_model.zip"

# Create a zipper file of the exported mannequin folder
shutil.make_archive(zip_file_path.change('.zip', ''), 'zip', exported_model_path)

# Obtain the zip file utilizing Google Colab's file obtain utility
from google.colab import information
information.obtain(zip_file_path)

You should use these downloaded mannequin information for testing it on unseen pictures or in your functions in keeping with your wants.

You may seek advice from this: collab pocket book for detailed code

Conclusion

In conclusion, this information equips you with the data and instruments needed to coach an object detection mannequin utilizing TensorFlow’s Object Detection API, leveraging datasets from Roboflow Universe for fast customization. By following the steps outlined, you possibly can successfully put together your knowledge, configure the coaching pipeline, choose the fitting mannequin, and fine-tune it to fulfill your particular wants. Furthermore, the flexibility to export and deploy your skilled mannequin opens up huge potentialities for real-world functions, whether or not in autonomous autos, medical imaging, or surveillance techniques. This workflow allows you to create highly effective, scalable object detection techniques with lowered complexity and sooner time to deployment.

Key Takeaways

• TensorFlow Object Detection API gives a versatile framework for constructing customized object detection fashions with pre-trained choices, lowering improvement time and complexity.
• TFRecord format is crucial for environment friendly knowledge dealing with, particularly with massive datasets in TensorFlow, permitting quick coaching and minimal overhead.
• Pipeline configuration information are essential for fine-tuning and adjusting the mannequin to work together with your particular dataset and desired efficiency traits.
• Pretrained fashions like efficientdet-d0 and ssd-mobilenet-v2 present strong beginning factors for coaching customized fashions, with every having particular strengths relying on use case and useful resource constraints.
• The coaching course of entails managing parameters like batch measurement, variety of steps, and mannequin checkpointing to make sure the mannequin learns optimally.
• Exporting the mannequin is crucial for utilizing the skilled object detection mannequin in a real-world mannequin that’s being packaged and prepared for deployment.

Regularly Requested Questions

The media proven on this article isn’t owned by Analytics Vidhya and is used on the Writer’s discretion.

I am Neha Dwivedi, a Information Science fanatic , Graduated from MIT World Peace College,Pune. I am obsessed with Information Science and rising developments with it. I am excited to share insights and study from this neighborhood!